Why does ExtractMpegFramesTest not work with rotated input files?

Question

I'm starting to decode/encode videos for an app and it seems that BigFlake's site is the authoritative reference for working video code. I've copied the ExtractMpegFramesTest_egl14.java into my project and modified it to use different file paths, but nothing else. It extracts properly frames from pre recorded videos I copy into the device. However, when trying to use it with a video recorded from the device's camera I get frames which are just horizontal lines:

After testing different recorded videos I realised the problem comes from rotated videos. The device's camera encodes frames as they come from the framebuffer and then slaps a rotation tag on the video. This can be extracted from the MediaFormat object:

int rotation = format.getInteger("rotation-degrees");

Unfortunately swapping the width/height of the saved image doesn't alter anything, the images still get corrupted. What has to be done to the save code to handle rotated frames correctly?

Answer 1

As hinted by the source code comment pointed out by another answer, OpenGL requires transforming the image. The original invert boolean helps with sources which have not been rotated or have a rotation of 180 degrees. But the other orientations (90 and 270) will display a flipped image, so they have to be flipped too.

Here are the changes I did to handle rotation correctly:

1. Read the rotation-degrees attribute (when available) from the source media.
2. Modify the CodecOutputSurface constructor to accept a numerical rotation parameter, which is stored in the mRotation instance variable.
3. Modify the original invert code to be used when the rotation is 0 or 180.
4. Use a different matrix transformation when the rotation is 90 or 270.

You can get the full modified version from the following gist as well as take a look at the individual changes mentioned above.