FileDocCategorySizeDatePackage
ImageReader.javaAPI DocAndroid 5.1 API27747Thu Mar 12 22:22:30 GMT 2015android.media

ImageReader

public class ImageReader extends Object implements AutoCloseable

The ImageReader class allows direct application access to image data rendered into a {@link android.view.Surface}

Several Android media API classes accept Surface objects as targets to render to, including {@link MediaPlayer}, {@link MediaCodec}, {@link android.hardware.camera2.CameraDevice}, and {@link android.renderscript.Allocation RenderScript Allocations}. The image sizes and formats that can be used with each source vary, and should be checked in the documentation for the specific API.

The image data is encapsulated in {@link Image} objects, and multiple such objects can be accessed at the same time, up to the number specified by the {@code maxImages} constructor parameter. New images sent to an ImageReader through its {@link Surface} are queued until accessed through the {@link #acquireLatestImage} or {@link #acquireNextImage} call. Due to memory limits, an image source will eventually stall or drop Images in trying to render to the Surface if the ImageReader does not obtain and release Images at a rate equal to the production rate.

Fields Summary
private static final int
ACQUIRE_SUCCESS
Returned by nativeImageSetup when acquiring the image was successful.
private static final int
ACQUIRE_NO_BUFS
Returned by nativeImageSetup when we couldn't acquire the buffer, because there were no buffers available to acquire.
private static final int
ACQUIRE_MAX_IMAGES
Returned by nativeImageSetup when we couldn't acquire the buffer because the consumer has already acquired {@maxImages} and cannot acquire more than that.
private final int
mWidth
private final int
mHeight
private final int
mFormat
private final int
mMaxImages
private final int
mNumPlanes
private final android.view.Surface
mSurface
private final Object
mListenerLock
private OnImageAvailableListener
mListener
private ListenerHandler
mListenerHandler
private long
mNativeContext
This field is used by native code, do not access or modify.
Constructors Summary
protected ImageReader(int width, int height, int format, int maxImages)

hide

        mWidth = width;
        mHeight = height;
        mFormat = format;
        mMaxImages = maxImages;

        if (width < 1 || height < 1) {
            throw new IllegalArgumentException(
                "The image dimensions must be positive");
        }
        if (mMaxImages < 1) {
            throw new IllegalArgumentException(
                "Maximum outstanding image count must be at least 1");
        }

        if (format == ImageFormat.NV21) {
            throw new IllegalArgumentException(
                    "NV21 format is not supported");
        }

        mNumPlanes = getNumPlanesFromFormat();

        nativeInit(new WeakReference<ImageReader>(this), width, height, format, maxImages);

        mSurface = nativeGetSurface();
    
Methods Summary
public ImageacquireLatestImage()

Acquire the latest {@link Image} from the ImageReader's queue, dropping older {@link Image images}. Returns {@code null} if no new image is available.

This operation will acquire all the images possible from the ImageReader, but {@link #close} all images that aren't the latest. This function is recommended to use over {@link #acquireNextImage} for most use-cases, as it's more suited for real-time processing.

Note that {@link #getMaxImages maxImages} should be at least 2 for {@link #acquireLatestImage} to be any different than {@link #acquireNextImage} - discarding all-but-the-newest {@link Image} requires temporarily acquiring two {@link Image Images} at once. Or more generally, calling {@link #acquireLatestImage} with less than two images of margin, that is {@code (maxImages - currentAcquiredImages < 2)} will not discard as expected.

This operation will fail by throwing an {@link IllegalStateException} if {@code maxImages} have been acquired with {@link #acquireLatestImage} or {@link #acquireNextImage}. In particular a sequence of {@link #acquireLatestImage} calls greater than {@link #getMaxImages} without calling {@link Image#close} in-between will exhaust the underlying queue. At such a time, {@link IllegalStateException} will be thrown until more images are released with {@link Image#close}.

return
latest frame of image data, or {@code null} if no image data is available.
throws
IllegalStateException if too many images are currently acquired

        Image image = acquireNextImage();
        if (image == null) {
            return null;
        }
        try {
            for (;;) {
                Image next = acquireNextImageNoThrowISE();
                if (next == null) {
                    Image result = image;
                    image = null;
                    return result;
                }
                image.close();
                image = next;
            }
        } finally {
            if (image != null) {
                image.close();
            }
        }
    
public ImageacquireNextImage()

Acquire the next Image from the ImageReader's queue. Returns {@code null} if no new image is available.

Warning: Consider using {@link #acquireLatestImage()} instead, as it will automatically release older images, and allow slower-running processing routines to catch up to the newest frame. Usage of {@link #acquireNextImage} is recommended for batch/background processing. Incorrectly using this function can cause images to appear with an ever-increasing delay, followed by a complete stall where no new images seem to appear.

This operation will fail by throwing an {@link IllegalStateException} if {@code maxImages} have been acquired with {@link #acquireNextImage} or {@link #acquireLatestImage}. In particular a sequence of {@link #acquireNextImage} or {@link #acquireLatestImage} calls greater than {@link #getMaxImages maxImages} without calling {@link Image#close} in-between will exhaust the underlying queue. At such a time, {@link IllegalStateException} will be thrown until more images are released with {@link Image#close}.

return
a new frame of image data, or {@code null} if no image data is available.
throws
IllegalStateException if {@code maxImages} images are currently acquired
see
#acquireLatestImage

        SurfaceImage si = new SurfaceImage();
        int status = acquireNextSurfaceImage(si);

        switch (status) {
            case ACQUIRE_SUCCESS:
                return si;
            case ACQUIRE_NO_BUFS:
                return null;
            case ACQUIRE_MAX_IMAGES:
                throw new IllegalStateException(
                        String.format(
                                "maxImages (%d) has already been acquired, " +
                                "call #close before acquiring more.", mMaxImages));
            default:
                throw new AssertionError("Unknown nativeImageSetup return code " + status);
        }
    
public ImageacquireNextImageNoThrowISE()
Don't throw IllegalStateException if there are too many images acquired.

return
Image if acquiring succeeded, or null otherwise.
hide

        SurfaceImage si = new SurfaceImage();
        return acquireNextSurfaceImage(si) == ACQUIRE_SUCCESS ? si : null;
    
private intacquireNextSurfaceImage(android.media.ImageReader$SurfaceImage si)
Attempts to acquire the next image from the underlying native implementation.

Note that unexpected failures will throw at the JNI level.

param
si A blank SurfaceImage.
return
One of the {@code ACQUIRE_*} codes that determine success or failure.
see
#ACQUIRE_MAX_IMAGES
see
#ACQUIRE_NO_BUFS
see
#ACQUIRE_SUCCESS


        int status = nativeImageSetup(si);

        switch (status) {
            case ACQUIRE_SUCCESS:
                si.createSurfacePlanes();
                si.setImageValid(true);
            case ACQUIRE_NO_BUFS:
            case ACQUIRE_MAX_IMAGES:
                break;
            default:
                throw new AssertionError("Unknown nativeImageSetup return code " + status);
        }

        return status;
    
public voidclose()
Free up all the resources associated with this ImageReader.

After calling this method, this ImageReader can not be used. Calling any methods on this ImageReader and Images previously provided by {@link #acquireNextImage} or {@link #acquireLatestImage} will result in an {@link IllegalStateException}, and attempting to read from {@link ByteBuffer ByteBuffers} returned by an earlier {@link Image.Plane#getBuffer Plane#getBuffer} call will have undefined behavior.

        setOnImageAvailableListener(null, null);
        nativeClose();
    
protected voidfinalize()

        try {
            close();
        } finally {
            super.finalize();
        }
    
public intgetHeight()
The default height of {@link Image Images}, in pixels.

The height may be overridden by the producer sending buffers to this ImageReader's Surface. If so, the actual height of the images can be found using {@link Image#getHeight}.

return
the expected height of an Image

        return mHeight;
    
public intgetImageFormat()
The default {@link ImageFormat image format} of {@link Image Images}.

Some color formats may be overridden by the producer sending buffers to this ImageReader's Surface if the default color format allows. ImageReader guarantees that all {@link Image Images} acquired from ImageReader (for example, with {@link #acquireNextImage}) will have a "compatible" format to what was specified in {@link #newInstance}. As of now, each format is only compatible to itself. The actual format of the images can be found using {@link Image#getFormat}.

return
the expected format of an Image
see
ImageFormat

        return mFormat;
    
public intgetMaxImages()
Maximum number of images that can be acquired from the ImageReader by any time (for example, with {@link #acquireNextImage}).

An image is considered acquired after it's returned by a function from ImageReader, and until the Image is {@link Image#close closed} to release the image back to the ImageReader.

Attempting to acquire more than {@code maxImages} concurrently will result in the acquire function throwing a {@link IllegalStateException}. Furthermore, while the max number of images have been acquired by the ImageReader user, the producer enqueueing additional images may stall until at least one image has been released.

return
Maximum number of images for this ImageReader.
see
Image#close

        return mMaxImages;
    
private intgetNumPlanesFromFormat()
Only a subset of the formats defined in {@link android.graphics.ImageFormat ImageFormat} and {@link android.graphics.PixelFormat PixelFormat} are supported by ImageReader. When reading RGB data from a surface, the formats defined in {@link android.graphics.PixelFormat PixelFormat} can be used, when reading YUV, JPEG or raw sensor data (for example, from camera or video decoder), formats from {@link android.graphics.ImageFormat ImageFormat} are used.

        switch (mFormat) {
            case ImageFormat.YV12:
            case ImageFormat.YUV_420_888:
            case ImageFormat.NV21:
                return 3;
            case ImageFormat.NV16:
                return 2;
            case PixelFormat.RGB_565:
            case PixelFormat.RGBA_8888:
            case PixelFormat.RGBX_8888:
            case PixelFormat.RGB_888:
            case ImageFormat.JPEG:
            case ImageFormat.YUY2:
            case ImageFormat.Y8:
            case ImageFormat.Y16:
            case ImageFormat.RAW_SENSOR:
            case ImageFormat.RAW10:
                return 1;
            default:
                throw new UnsupportedOperationException(
                        String.format("Invalid format specified %d", mFormat));
        }
    
public android.view.SurfacegetSurface()

Get a {@link Surface} that can be used to produce {@link Image Images} for this {@code ImageReader}.

Until valid image data is rendered into this {@link Surface}, the {@link #acquireNextImage} method will return {@code null}. Only one source can be producing data into this Surface at the same time, although the same {@link Surface} can be reused with a different API once the first source is disconnected from the {@link Surface}.

return
A {@link Surface} to use for a drawing target for various APIs.

        return mSurface;
    
public intgetWidth()
The default width of {@link Image Images}, in pixels.

The width may be overridden by the producer sending buffers to this ImageReader's Surface. If so, the actual width of the images can be found using {@link Image#getWidth}.

return
the expected width of an Image

        return mWidth;
    
private static native voidnativeClassInit()
We use a class initializer to allow the native code to cache some field offsets.

private native synchronized voidnativeClose()

private native synchronized android.view.SurfacenativeGetSurface()

private native synchronized intnativeImageSetup(Image i)

return
A return code {@code ACQUIRE_*}
see
#ACQUIRE_SUCCESS
see
#ACQUIRE_NO_BUFS
see
#ACQUIRE_MAX_IMAGES

private native synchronized voidnativeInit(java.lang.Object weakSelf, int w, int h, int fmt, int maxImgs)

private native synchronized voidnativeReleaseImage(Image i)

public static android.media.ImageReadernewInstance(int width, int height, int format, int maxImages)

Create a new reader for images of the desired size and format.

The {@code maxImages} parameter determines the maximum number of {@link Image} objects that can be be acquired from the {@code ImageReader} simultaneously. Requesting more buffers will use up more memory, so it is important to use only the minimum number necessary for the use case.

The valid sizes and formats depend on the source of the image data.

param
width The default width in pixels of the Images that this reader will produce.
param
height The default height in pixels of the Images that this reader will produce.
param
format The format of the Image that this reader will produce. This must be one of the {@link android.graphics.ImageFormat} or {@link android.graphics.PixelFormat} constants. Note that not all formats is supported, like ImageFormat.NV21.
param
maxImages The maximum number of images the user will want to access simultaneously. This should be as small as possible to limit memory use. Once maxImages Images are obtained by the user, one of them has to be released before a new Image will become available for access through {@link #acquireLatestImage()} or {@link #acquireNextImage()}. Must be greater than 0.
see
Image


                                                                                                                                                                                                                                                                                                                                                
               
        return new ImageReader(width, height, format, maxImages);
    
private static voidpostEventFromNative(java.lang.Object selfRef)
Called from Native code when an Event happens. This may be called from an arbitrary Binder thread, so access to the ImageReader must be synchronized appropriately.

        @SuppressWarnings("unchecked")
        WeakReference<ImageReader> weakSelf = (WeakReference<ImageReader>)selfRef;
        final ImageReader ir = weakSelf.get();
        if (ir == null) {
            return;
        }

        final Handler handler;
        synchronized (ir.mListenerLock) {
            handler = ir.mListenerHandler;
        }
        if (handler != null) {
            handler.sendEmptyMessage(0);
        }
    
private voidreleaseImage(Image i)

Return the frame to the ImageReader for reuse.

        if (! (i instanceof SurfaceImage) ) {
            throw new IllegalArgumentException(
                "This image was not produced by an ImageReader");
        }
        SurfaceImage si = (SurfaceImage) i;
        if (si.getReader() != this) {
            throw new IllegalArgumentException(
                "This image was not produced by this ImageReader");
        }

        si.clearSurfacePlanes();
        nativeReleaseImage(i);
        si.setImageValid(false);
    
public voidsetOnImageAvailableListener(android.media.ImageReader$OnImageAvailableListener listener, android.os.Handler handler)
Register a listener to be invoked when a new image becomes available from the ImageReader.

param
listener The listener that will be run.
param
handler The handler on which the listener should be invoked, or null if the listener should be invoked on the calling thread's looper.
throws
IllegalArgumentException If no handler specified and the calling thread has no looper.

        synchronized (mListenerLock) {
            if (listener != null) {
                Looper looper = handler != null ? handler.getLooper() : Looper.myLooper();
                if (looper == null) {
                    throw new IllegalArgumentException(
                            "handler is null but the current thread is not a looper");
                }
                if (mListenerHandler == null || mListenerHandler.getLooper() != looper) {
                    mListenerHandler = new ListenerHandler(looper);
                }
                mListener = listener;
            } else {
                mListener = null;
                mListenerHandler = null;
            }
        }