768 lines
31 KiB
HTML
768 lines
31 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
|
|
"http://www.w3.org/TR/html4/loose.dtd">
|
|
<html>
|
|
|
|
<head>
|
|
<title>OpenMAX AL for Android</title>
|
|
</head>
|
|
|
|
<body>
|
|
|
|
<b>Note:</b>
|
|
<a href="http://developer.android.com/reference/android/media/MediaCodec.html">android.media.MediaCodec</a>
|
|
is recommended for new applications at API level 16 and above. The NDK
|
|
equivalent (in <media/NdkMedia*.h>) is recommended for new native
|
|
applications at API level 21 and above.
|
|
|
|
<h1>OpenMAX AL for Android</h1>
|
|
|
|
This article describes the Android native multimedia APIs based on the
|
|
Khronos Group OpenMAX AL™ 1.0.1 standard, as of Android API level 14 (Android
|
|
platform version 4.0) and higher.
|
|
<p>
|
|
OpenMAX AL is a companion API to OpenSL ES, but for multimedia (video
|
|
and audio) rather than audio only.
|
|
<p>
|
|
Android 4.0 provides a direct, efficient path for low-level streaming multimedia. The path is
|
|
ideal for applications that need to maintain complete control over media data before passing it to
|
|
the platform for presentation. For example, media applications can retrieve data from any
|
|
source, apply proprietary encryption/decryption, and then send the data to the platform for display.
|
|
<p>
|
|
Applications can send processed data to the platform as a multiplexed stream of audio/video
|
|
content in MPEG-2 transport stream format. The platform de-muxes, decodes, and renders the content.
|
|
The audio track is rendered to the active audio device, while the video track is rendered to either
|
|
a Surface or a SurfaceTexture. When rendering to a SurfaceTexture, the application can apply
|
|
subsequent graphics effects to each frame using OpenGL.
|
|
<p>
|
|
OpenMAX AL provides a C language interface that is also callable from C++, and
|
|
exposes features similar to these Android APIs
|
|
callable from Java programming language code:
|
|
<ul>
|
|
<li><a href="http://developer.android.com/reference/android/media/MediaPlayer.html">
|
|
android.media.MediaPlayer</a>
|
|
</ul>
|
|
|
|
As with all of the Android Native Development Kit (NDK), the primary
|
|
purpose of OpenMAX AL for Android is to facilitate the implementation
|
|
of shared libraries to be called from Java programming language code via Java Native
|
|
Interface (JNI). NDK is not intended for writing pure C/C++
|
|
applications.
|
|
|
|
<p>
|
|
Note: though based on OpenMAX AL, the Android native multimedia API
|
|
is <i>not</i> a conforming implementation of either OpenMAX AL 1.0.1
|
|
profile (media player or media player / recorder). This is because Android does not
|
|
implement all of the features required by either of the profiles.
|
|
Any known cases where Android behaves differently than the specification
|
|
are described in section "Android extensions" below.
|
|
|
|
The Android OpenMAX AL implementation has limited features, and is
|
|
intended primarily for certain performance-sensitive native streaming
|
|
multimedia applications such as video players.
|
|
<p>
|
|
The major feature is the ability to play an MPEG-2 transport stream
|
|
containing a single program stream made up of one H.264 video elementary
|
|
stream and one AAC audio elementary stream. The application provides
|
|
the stream via an Android buffer queue data source, which is based on
|
|
the OpenSL ES buffer queue concept and Android-specific extensions.
|
|
<p>
|
|
The video sink is an <code>ANativeWindow *</code> abstract handle,
|
|
derived from an <code>android.view.Surface</code> ("surface").
|
|
A Surface from <code>SurfaceHolder.getSurface()</code> should be used when displaying
|
|
an unaltered video within a fixed SurfaceView frame. A Surface from
|
|
<code>new Surface(SurfaceTexture)</code> allows streaming the decoded
|
|
video frames to an OpenGL ES 2.0 texture, where the frames can be used
|
|
as input to a shader algorithm in the Graphics Processing Unit (GPU).
|
|
Be sure to <code>release()</code> the Surface as soon as possible after
|
|
calling <code>setSurface</code> or ANativeWindow_fromSurface.
|
|
<p>
|
|
The audio sink is always an output mix, a device-independent mixer object
|
|
similar to that of OpenSL ES.
|
|
|
|
<h2>Getting started</h2>
|
|
|
|
<h3>Example code</h3>
|
|
|
|
<h4>Recommended</h4>
|
|
|
|
Supported and tested example code, usable as a model
|
|
for your own code, is located in NDK folder
|
|
<code>platforms/android-14/samples/native-media/</code>.
|
|
|
|
<h4>Not recommended</h4>
|
|
|
|
The OpenMAX AL 1.0.1 specification contains example code in the
|
|
appendices (see section "References" below for the link to this
|
|
specification). However, the examples in Appendix D: Sample Code
|
|
use features
|
|
not supported by Android. Some examples also contain
|
|
typographical errors, or use APIs that are likely to change.
|
|
Proceed with caution in referring to these;
|
|
though the code may be helpful in understanding the full OpenMAX AL
|
|
standard, it should not be used as is with Android.
|
|
|
|
<h3>Adding OpenMAX AL to your application source code</h3>
|
|
|
|
OpenMAX AL is a C API, but is callable from both C and C++ code.
|
|
<p>
|
|
Add the following lines to your code:
|
|
<pre>
|
|
#include <OMXAL/OpenMAXAL.h>
|
|
#include <OMXAL/OpenMAXAL_Android.h>
|
|
</pre>
|
|
|
|
<h3>Makefile</h3>
|
|
|
|
Modify your Android.mk as follows:
|
|
<pre>
|
|
LOCAL_LDLIBS += libOpenMAXAL
|
|
</pre>
|
|
|
|
<h3>Multimedia content</h3>
|
|
|
|
The only supported way to supply multimedia content is via an MPEG-2
|
|
transport stream.
|
|
<p>
|
|
Finding or creating useful multimedia content for your application is
|
|
beyond the scope of this article.
|
|
<p>
|
|
Note that it is your responsibility to ensure that you are legally
|
|
permitted to play the content.
|
|
|
|
<h3>Debugging</h3>
|
|
|
|
For robustness, we recommend that you examine the <code>XAresult</code>
|
|
value which is returned by most APIs. Use of <code>assert</code>
|
|
vs. more advanced error handling logic is a matter of coding style
|
|
and the particular API; see the Wikipedia article on
|
|
<a href="http://en.wikipedia.org/wiki/Assertion_(computing)">assert</a>
|
|
for more information. In the supplied example, we have used <code>assert</code>
|
|
for "impossible" conditions which would indicate a coding error, and
|
|
explicit error handling for others which are more likely to occur
|
|
in production.
|
|
<p>
|
|
Many API errors result in a log entry, in addition to the non-zero
|
|
result code. These log entries provide additional detail which can
|
|
be especially useful for the more complex APIs such as
|
|
<code>Engine::CreateMediaPlayer</code>.
|
|
<p>
|
|
Use <a href="http://developer.android.com/guide/developing/tools/adb.html">
|
|
adb logcat</a>, the
|
|
<a href="http://developer.android.com/guide/developing/eclipse-adt.html">
|
|
Eclipse ADT plugin</a> LogCat pane, or
|
|
<a href="http://developer.android.com/guide/developing/tools/ddms.html#logcat">
|
|
ddms logcat</a> to see the log.
|
|
|
|
<h2>Supported features from OpenMAX AL 1.0.1</h2>
|
|
|
|
This section summarizes available features. In some
|
|
cases, there are limitations which are described in the next
|
|
sub-section.
|
|
|
|
<h3>Global entry points</h3>
|
|
|
|
Supported global entry points:
|
|
<ul>
|
|
<li><code>xaCreateEngine</code>
|
|
<li><code>xaQueryNumSupportedEngineInterfaces</code>
|
|
<li><code>xaQuerySupportedEngineInterfaces</code>
|
|
</ul>
|
|
|
|
<h3>Objects and interfaces</h3>
|
|
|
|
The following figure indicates objects and interfaces supported by
|
|
Android's OpenMAX AL implementation. A green cell means the feature
|
|
is supported.
|
|
|
|
<p>
|
|
<img src="chart3.png" alt="Supported objects and interfaces">
|
|
|
|
<h3>Limitations</h3>
|
|
|
|
This section details limitations with respect to the supported
|
|
objects and interfaces from the previous section.
|
|
|
|
<h4>Audio</h4>
|
|
|
|
The audio stream type cannot be configured; it is always <code>AudioManager.STREAM_MUSIC</code>.
|
|
<p>
|
|
Effects are not supported.
|
|
|
|
<h4>Dynamic interface management</h4>
|
|
|
|
<code>RemoveInterface</code> and <code>ResumeInterface</code> are not supported.
|
|
|
|
<h4>Engine</h4>
|
|
|
|
Supported:
|
|
<ul>
|
|
<li><code>CreateMediaPlayer</code>
|
|
</ul>
|
|
|
|
Not supported:
|
|
<ul>
|
|
<li><code>CreateCameraDevice</code>
|
|
<li><code>CreateRadioDevice</code>
|
|
<li><code>CreateLEDDevice</code>
|
|
<li><code>CreateVibraDevice</code>
|
|
<li><code>CreateMetadataExtractor</code>
|
|
<li><code>CreateExtensionObject</code>
|
|
<li><code>GetImplementationInfo</code>
|
|
</ul>
|
|
|
|
For <code>CreateMediaPlayer</code>, these restrictions apply:
|
|
<ul>
|
|
<li>audio sink is an output mix data locator
|
|
<li>video sink is a native display data locator
|
|
<li>soundbank, LED array, and vibra sinks must be <code>NULL</code>
|
|
</ul>
|
|
|
|
<h4>MIME data format</h4>
|
|
|
|
In the current Android implementation of OpenMAX AL, a media player
|
|
receives its source data as an MPEG-2 transport stream via a
|
|
buffer queue.
|
|
<p>
|
|
The source data locator must be <code>XA_DATALOCATOR_ANDROIDBUFFERQUEUE</code>
|
|
(see "Android extensions" below).
|
|
<p>
|
|
The source data format must be <code>XADataFormat_MIME</code>.
|
|
Initialize <code>mimeType</code> to <code>XA_ANDROID_MIME_MP2TS</code>,
|
|
and <code>containerType</code> to <code>XA_CONTAINERTYPE_MPEG_TS</code>.
|
|
<p>
|
|
The contained transport stream must have a single program with one H.264
|
|
video elementary stream and one AAC audio elementary stream.
|
|
|
|
<h4>Object</h4>
|
|
|
|
<code>Resume</code>, <code>RegisterCallback</code>,
|
|
<code>AbortAsyncOperation</code>, <code>SetPriority</code>,
|
|
<code>GetPriority</code>, and <code>SetLossOfControlInterfaces</code>
|
|
are not supported.
|
|
|
|
<h4>StreamInformation</h4>
|
|
|
|
Use the StreamInformation interface on a media player object to discover
|
|
when the video metrics (height/width or aspect ratio) are available or
|
|
changed, and to then get the sizes.
|
|
<p>
|
|
|
|
Supported:
|
|
<ul>
|
|
<li><code>RegisterStreamChangeCallback</code>
|
|
<li><code>QueryMediaContainerInformation</code>
|
|
<li><code>QueryStreamInformation</code>
|
|
<li><code>QueryStreamType</code>
|
|
</ul>
|
|
|
|
Not supported:
|
|
<ul>
|
|
<li><code>QueryActiveStreams</code>
|
|
<li><code>QueryStreamName</code>
|
|
<li><code>SetActiveStream</code>
|
|
</ul>
|
|
|
|
<h4>VideoDecoderCapabilities</h4>
|
|
|
|
This interface on the engine object reports video decoder capabilities
|
|
without interpretation, exactly as claimed by the underlying OpenMAX IL
|
|
implementation.
|
|
<p>
|
|
These fields in <code>XAVideoCodecDescriptor</code> are filled:
|
|
<ul>
|
|
<li><code>codecId</code>
|
|
<li><code>profileSetting</code>
|
|
<li><code>levelSetting</code>
|
|
</ul>
|
|
The other fields are not filled and should be ignored.
|
|
<p>
|
|
Applications should rely on the capabilities documented at
|
|
<a href="http://developer.android.com/guide/appendix/media-formats.html">Android Supported Media Formats</a>,
|
|
not the information reported by this interface.
|
|
|
|
<h3>Data structures</h3>
|
|
|
|
Android API level 14 supports these OpenMAX AL 1.0.1 data structures:
|
|
<ul>
|
|
<li>XADataFormat_MIME
|
|
<li>XADataLocator_NativeDisplay
|
|
<li>XADataLocator_OutputMix
|
|
<li>XADataSink
|
|
<li>XADataSource
|
|
<li>XAEngineOption
|
|
<li>XAInterfaceID
|
|
<li>XAMediaContainerInformation
|
|
<li>XANativeHandle
|
|
<li>XA*StreamInformation
|
|
<li>XAVideoCodecDescriptor
|
|
</ul>
|
|
|
|
<h4>XADataLocator_NativeDisplay</h4>
|
|
|
|
The native display data locator is used to specify the video sink:
|
|
<pre>
|
|
typedef struct XADataLocator_NativeDisplay_ {
|
|
XAuint32 locatorType; // XA_DATALOCATOR_NATIVEDISPLAY
|
|
XANativeHandle hWindow; // ANativeWindow *
|
|
XANativeHandle hDisplay; // NULL
|
|
} XADataLocator_NativeDisplay;
|
|
</pre>
|
|
|
|
Set the <code>hWindow</code> field to an
|
|
<code>ANativeWindow *</code> and set <code>hDisplay</code> to <code>NULL</code>.
|
|
You can get a <code>ANativeWindow *</code> handle from an <code>android.view.Surface</code>,
|
|
using this NDK function:
|
|
<pre>
|
|
#include <android/native_window_jni.h>
|
|
|
|
ANativeWindow* ANativeWindow_fromSurface(JNIEnv* env, jobject surface);
|
|
</pre>
|
|
Don't forget to free this handle in your shutdown code with <code>ANativeWindow_release</code>.
|
|
|
|
<h3>Platform configuration</h3>
|
|
|
|
OpenMAX AL for Android is designed for multi-threaded applications,
|
|
and is thread-safe.
|
|
<p>
|
|
OpenMAX AL for Android supports a single engine per application, and
|
|
up to 32 objects. Available device memory and CPU may further
|
|
restrict the usable number of objects.
|
|
<p>
|
|
<code>xaCreateEngine</code> recognizes, but ignores, these engine options:
|
|
<ul>
|
|
<li><code>XA_ENGINEOPTION_THREADSAFE</code>
|
|
<li><code>XA_ENGINEOPTION_LOSSOFCONTROL</code>
|
|
</ul>
|
|
|
|
OpenMAX AL and OpenSL ES may be used together in the same application.
|
|
In this case, there is internally a single shared engine object,
|
|
and the 32 object limit is shared between OpenMAX AL and OpenSL ES.
|
|
The application should first create both engines, then use both engines,
|
|
and finally destroy both engines. The implementation maintains a
|
|
reference count on the shared engine, so that it is correctly destroyed
|
|
at the second destroy.
|
|
|
|
<h2>Planning for future versions of OpenMAX AL</h2>
|
|
|
|
The Android native multimedia APIs at level 14 are based on Khronos
|
|
Group OpenMAX AL 1.0.1 (see section "References" below).
|
|
Khronos has released
|
|
a revised version 1.1 of the standard. The revised version
|
|
includes new features, clarifications, correction of
|
|
typographical errors, and some incompatibilities. Most of the
|
|
incompatibilities are relatively minor, or are in areas of OpenMAX AL
|
|
not supported by Android. However, even a small change
|
|
can be significant for an application developer, so it is important
|
|
to prepare for this.
|
|
<p>
|
|
The Android team is committed to preserving future API binary
|
|
compatibility for developers to the extent feasible. It is our
|
|
intention to continue to support future binary compatibility of the
|
|
1.0.1-based API, even as we add support for later versions of the
|
|
standard. An application developed with this version should
|
|
work on future versions of the Android platform, provided that
|
|
you follow the guidelines listed in section "Planning for
|
|
binary compatibility" below.
|
|
<p>
|
|
Note that future source compatibility will <i>not</i> be a goal. That is,
|
|
if you upgrade to a newer version of the NDK, you may need to modify
|
|
your application source code to conform to the new API. We expect
|
|
that most such changes will be minor; see details below.
|
|
|
|
<h3>Planning for binary compatibility</h3>
|
|
|
|
We recommend that your application follow these guidelines,
|
|
to improve future binary compatibility:
|
|
<ul>
|
|
<li>
|
|
Use only the documented subset of Android-supported features from
|
|
OpenMAX AL 1.0.1.
|
|
<li>
|
|
Do not depend on a particular result code for an unsuccessful
|
|
operation; be prepared to deal with a different result code.
|
|
<li>
|
|
Application callback handlers generally run in a restricted context,
|
|
and should be written to perform their work quickly and then return
|
|
as soon as possible. Do not do complex operations within a callback
|
|
handler. For example, within a buffer queue completion callback,
|
|
you can enqueue another buffer, but do not create a media player.
|
|
<li>
|
|
Callback handlers should be prepared to be called more or less
|
|
frequently, to receive additional event types, and should ignore
|
|
event types that they do not recognize. Callbacks that are configured
|
|
with an event mask of enabled event types should be prepared to be
|
|
called with multiple event type bits set simultaneously.
|
|
Use "&" to test for each event bit rather than a switch case.
|
|
<li>
|
|
Use prefetch status and callbacks as a general indication of progress, but do
|
|
not depend on specific hard-coded fill levels or callback sequence.
|
|
The meaning of the prefetch status fill level, and the behavior for
|
|
errors that are detected during prefetch, may change.
|
|
</ul>
|
|
|
|
<h3>Planning for source compatibility</h3>
|
|
|
|
As mentioned, source code incompatibilities are expected in the next
|
|
version of OpenMAX AL from Khronos Group. Likely areas of change include:
|
|
|
|
<ul>
|
|
<li>Addition of <code>const</code> to input parameters passed by reference,
|
|
and to <code>XAchar *</code> struct fields used as input values.
|
|
This should not require any changes to your code.
|
|
<li>Substitution of unsigned types for some parameters that are
|
|
currently signed. You may need to change a parameter type from
|
|
<code>XAint32</code> to <code>XAuint32</code> or similar, or add a cast.
|
|
<li>Additional fields in struct types. For output parameters, these
|
|
new fields can be ignored, but for input parameters the new fields
|
|
will need to be initialized. Fortunately, these are expected to all
|
|
be in areas not supported by Android.
|
|
<li>Interface
|
|
<a href="http://en.wikipedia.org/wiki/Globally_unique_identifier">
|
|
GUIDs</a> will change. Refer to interfaces by symbolic name rather than GUID
|
|
to avoid a dependency.
|
|
<li><code>XAchar</code> will change from <code>unsigned char</code>
|
|
to <code>char</code>. This primarily affects the MIME data format.
|
|
<li><code>XADataFormat_MIME.mimeType</code> will be renamed to <code>pMimeType</code>.
|
|
We recommend that you initialize the <code>XADataFormat_MIME</code>
|
|
data structure using a brace-enclosed comma-separated list of values,
|
|
rather than by field name, to isolate your code from this change.
|
|
In the example code we have used this technique.
|
|
</ul>
|
|
|
|
<h2>Android extensions</h2>
|
|
|
|
The API for Android extensions is defined in <code>OMXAL/OpenMAXAL_Android.h</code>
|
|
.
|
|
Consult that file for details on these extensions. Unless otherwise
|
|
noted, all interfaces are "explicit".
|
|
<p>
|
|
Note that use these extensions will limit your application's
|
|
portability to other OpenMAX AL implementations. If this is a concern,
|
|
we advise that you avoid using them, or isolate your use of these
|
|
with <code>#ifdef</code> etc.
|
|
<p>
|
|
The following figure shows which Android-specific interfaces and
|
|
data locators are available for each object type.
|
|
|
|
<p>
|
|
<img src="chart4.png" alt="Android extensions">
|
|
|
|
<h3>Android buffer queue data locator and interface</h3>
|
|
|
|
<h4>Comparison with OpenSL ES buffer queue</h4>
|
|
|
|
The Android buffer queue data locator and interface are based on
|
|
similar concepts from OpenSL ES 1.0.1, with these differences:
|
|
<ul>
|
|
<li>Though currently usable with only a media player and MPEG-2 transport
|
|
stream data, the Android buffer queue API is designed for flexibility
|
|
so that the API can also apply to other use cases in the future.
|
|
<li>Commands may be
|
|
optionally specified by the application at time of <code>Enqueue</code>.
|
|
Each command consists of an item key and optional item value.
|
|
Command key/value pairs are carried alongside the corresponding buffer in the queue,
|
|
and thus are processed in synchrony with the buffer.
|
|
<li>To enqueue command(s) without associated data, specify
|
|
a buffer address of NULL and buffer size of zero, along
|
|
with at least one command.
|
|
<li>Status may be
|
|
provided by the implementation during a completion callback.
|
|
Each status consists of an item key and optional item value.
|
|
Status key/value pairs are carried alongside
|
|
the corresponding buffer in the queue, and thus are received by the
|
|
application in synchrony with the completion callback.
|
|
<li>The completion callback receives additional parameters:
|
|
buffer address, buffer maximum data size, buffer actual size consumed (or filled by a future
|
|
recorder object), and a <code>void *</code> for application.
|
|
<li>The callback returns a value, which must be <code>XA_RESULT_SUCCESS</code>.
|
|
</ul>
|
|
|
|
The data locator type code is <code>XA_DATALOCATOR_ANDROIDBUFFERQUEUE</code> and
|
|
the associated structure is <code>XADataLocator_AndroidBufferQueue</code>.
|
|
<p>
|
|
The interface ID is <code>XA_IID_ANDROIDBUFFERQUEUESOURCE</code>.
|
|
|
|
<h4>Usage</h4>
|
|
|
|
A typical buffer queue configuration is 8 buffers of 1880 bytes each.
|
|
<p>
|
|
The application enqueues filled buffers of data in MPEG-2 transport
|
|
stream format. The buffer size must be a multiple of 188 bytes,
|
|
the size of an MPEG-2 transport stream packet. The buffer data must
|
|
be properly aligned on a packet boundary, and formatted per the MPEG-2
|
|
Part 1 specification.
|
|
<p>
|
|
An application may supply zero or one of these item codes
|
|
(command key/value pairs) at <code>Enqueue</code>:
|
|
<dl>
|
|
<dt>XA_ANDROID_ITEMKEY_EOS</dt>
|
|
<dd>End of stream. Informs the decode and rendering components that playback is complete.
|
|
The application must not call <code>Enqueue</code> again.
|
|
There is no associated value, so <code>itemSize</code> must be zero.
|
|
There must be no data buffer alongside the EOS command.
|
|
</dd>
|
|
<dt>XA_ANDROID_ITEMKEY_DISCONTINUITY</dt>
|
|
<dd>Discontinuity. This and following buffers have a new presentation time.
|
|
The new presentation time may be optionally specified as a parameter,
|
|
expressed in <code>itemData</code> as a 64-bit unsigned integer in units of 90 kHz clock ticks.
|
|
The <code>itemSize</code> should be either zero or 8.
|
|
The discontinuity command is intended for seeking to a new point in
|
|
the stream. The application should flush its internal data, then send
|
|
the discontinuity command prior to, or alongside of, the first buffer
|
|
corresponding to the new stream position.
|
|
The initial packets in the video elementary stream
|
|
should describe an IDR (Instantaneous Decoding Refresh) frame.
|
|
Note that the discontinuity
|
|
command is not intended for stream configuration / format changes;
|
|
for these use <code>XA_ANDROID_ITEMKEY_FORMAT_CHANGE</code>.
|
|
</dd>
|
|
<dt>XA_ANDROID_ITEMKEY_FORMAT_CHANGE</dt>
|
|
<dd>Format change. This and following buffers have a new format,
|
|
for example for MBR (Multiple Bit Rate) or resolution switching.
|
|
</dd>
|
|
</dl>
|
|
<p>
|
|
Upon notification of completion via a registered callback, the now
|
|
consumed buffer is available for the application to re-fill.
|
|
<p>
|
|
The implementation may supply zero or more of these item codes
|
|
(status key/value pairs) to the callback handler:
|
|
<dl>
|
|
<dt>XA_ANDROID_ITEMKEY_BUFFERQUEUEEVENT</dt>
|
|
<dd>Buffer queue event mask. The <code>itemSize</code> is 4, and <code>itemData</code> contains
|
|
the bit-wise "or" of zero or more of the <code>XA_ANDROIDBUFFERQUEUEEVENT_*</code>
|
|
symbols. This event mask explains the reason(s) why the callback handler
|
|
was called.</dd>
|
|
</dl>
|
|
|
|
<h3>Reporting of extensions</h3>
|
|
|
|
<code>Engine::QueryNumSupportedExtensions</code>,
|
|
<code>Engine::QuerySupportedExtension</code>,
|
|
<code>Engine::IsExtensionSupported</code> report these extensions:
|
|
<ul>
|
|
<li><code>ANDROID_SDK_LEVEL_#</code>
|
|
where # is the platform API level, 14 or higher
|
|
</ul>
|
|
|
|
<h2>Programming notes</h2>
|
|
|
|
These notes supplement the OpenMAX AL 1.0.1 specification,
|
|
available in the "References" section below.
|
|
|
|
<h3>Objects and interface initialization</h3>
|
|
|
|
Two aspects of the OpenMAX AL programming model that may be unfamiliar
|
|
to new developers are the distinction between objects and interfaces,
|
|
and the initialization sequence.
|
|
<p>
|
|
Briefly, an OpenMAX AL object is similar to the object concept
|
|
in programming languages such as Java and C++, except an OpenMAX AL
|
|
object is <i>only</i> visible via its associated interfaces. This
|
|
includes the initial interface for all objects, called
|
|
<code>XAObjectItf</code>. There is no handle for an object itself,
|
|
only a handle to the <code>XAObjectItf</code> interface of the object.
|
|
<p>
|
|
An OpenMAX AL object is first "created", which returns an
|
|
<code>XAObjectItf</code>, then "realized". This is similar to the
|
|
common programming pattern of first constructing an object (which
|
|
should never fail other than for lack of memory or invalid parameters),
|
|
and then completing initialization (which may fail due to lack of
|
|
resources). The realize step gives the implementation a
|
|
logical place to allocate additional resources if needed.
|
|
<p>
|
|
As part of the API to create an object, an application specifies
|
|
an array of desired interfaces that it plans to acquire later. Note
|
|
that this array does <i>not</i> automatically acquire the interfaces;
|
|
it merely indicates a future intention to acquire them. Interfaces
|
|
are distinguished as "implicit" or "explicit". An explicit interface
|
|
<i>must</i> be listed in the array if it will be acquired later.
|
|
An implicit interface need not be listed in the object create array,
|
|
but there is no harm in listing it there. OpenMAX AL has one more
|
|
kind of interface called "dynamic", which does not need to be
|
|
specified in the object create array, and can be added later after
|
|
the object is created. The Android implementation provides a
|
|
convenience feature to avoid this complexity; see section "Dynamic
|
|
interfaces at object creation" above.
|
|
<p>
|
|
After the object is created and realized, the application should
|
|
acquire interfaces for each feature it needs, using
|
|
<code>GetInterface</code> on the initial <code>XAObjectItf</code>.
|
|
<p>
|
|
Finally, the object is available for use via its interfaces, though
|
|
note that some objects require further setup.
|
|
Some use cases needs a bit more preparation in
|
|
order to detect connection errors. See the next section
|
|
"Media player prefetch" for details.
|
|
<p>
|
|
After your application is done with the object, you should explicitly
|
|
destroy it; see section "Destroy" below.
|
|
|
|
<h3>Media player prefetch</h3>
|
|
|
|
For a media player, <code>Object::Realize</code> allocates resources
|
|
but does not connect to the data source (i.e. "prepare") or begin
|
|
pre-fetching data. These occur once the player state is set to
|
|
either <code>XA_PLAYSTATE_PAUSED</code> or <code>XA_PLAYSTATE_PLAYING</code>.
|
|
<p>
|
|
The prefetch status interface is useful for detecting errors.
|
|
Register a callback and enable at least the
|
|
<code>XA_PREFETCHEVENT_FILLLEVELCHANGE</code> and
|
|
<code>XA_PREFETCHEVENT_STATUSCHANGE</code> events. If both of these
|
|
events are delivered simultaneously, and
|
|
<code>PrefetchStatus::GetFillLevel</code> reports a zero level, and
|
|
<code>PrefetchStatus::GetPrefetchStatus</code> reports
|
|
<code>XA_PREFETCHSTATUS_UNDERFLOW</code>, then this indicates a
|
|
non-recoverable error in the data source or in rendering to the video sink.
|
|
<p>
|
|
The next version of OpenMAX AL is expected to add more explicit
|
|
support for handling errors in the data source. However, for future
|
|
binary compatibility, we intend to continue to support the current
|
|
method for reporting a non-recoverable error.
|
|
<p>
|
|
In summary, a recommended code sequence is:
|
|
<ul>
|
|
<li>Engine::CreateMediaPlayer
|
|
<li>Object:Realize
|
|
<li>Object::GetInterface for XA_IID_PREFETCHSTATUS
|
|
<li>PrefetchStatus::SetCallbackEventsMask
|
|
<li>PrefetchStatus::RegisterCallback
|
|
<li>Object::GetInterface for XA_IID_PLAY
|
|
<li>Play::SetPlayState to XA_PLAYSTATE_PAUSED or XA_PLAYSTATE_PLAYING
|
|
<li>preparation and prefetching occur here; during this time your
|
|
callback will be called with periodic status updates and
|
|
error notifications
|
|
</ul>
|
|
|
|
<h3>Destroy</h3>
|
|
|
|
Be sure to destroy all objects on exit from your application. Objects
|
|
should be destroyed in reverse order of their creation, as it is
|
|
not safe to destroy an object that has any dependent objects.
|
|
For example, destroy in this order: audio players and recorders,
|
|
output mix, then finally the engine.
|
|
<p>
|
|
OpenMAX AL does not support automatic garbage collection or
|
|
<a href="http://en.wikipedia.org/wiki/Reference_counting">reference counting</a>
|
|
of interfaces. After you call <code>Object::Destroy</code>, all extant
|
|
interfaces derived from the associated object become <i>undefined</i>.
|
|
<p>
|
|
The Android OpenMAX AL implementation does not detect the incorrect
|
|
use of such interfaces.
|
|
Continuing to use such interfaces after the object is destroyed will
|
|
cause your application to crash or behave in unpredictable ways.
|
|
<p>
|
|
We recommend that you explicitly set both the primary object interface
|
|
and all associated interfaces to NULL as part of your object
|
|
destruction sequence, to prevent the accidental misuse of a stale
|
|
interface handle.
|
|
|
|
<h3>Callbacks and threads</h3>
|
|
|
|
Callback handlers are generally called <i>synchronously</i> with
|
|
respect to the event, that is, at the moment and location where the
|
|
event is detected by the implementation. But this point is
|
|
<i>asynchronous</i> with respect to the application. Thus you should
|
|
use a mutex or other synchronization mechanism to control access
|
|
to any variables shared between the application and the callback
|
|
handler. In the example code, such as for buffer queues, we have
|
|
omitted this synchronization in the interest of simplicity. However,
|
|
proper mutual exclusion would be critical for any production code.
|
|
<p>
|
|
Callback handlers are called from internal
|
|
non-application thread(s) which are not attached to the Dalvik virtual machine and thus
|
|
are ineligible to use JNI. Because these internal threads are
|
|
critical to the integrity of the OpenMAX AL implementation, a callback
|
|
handler should also not block or perform excessive work. Therefore,
|
|
if your callback handler needs to use JNI or do anything significant
|
|
(e.g. beyond an <code>Enqueue</code> or something else simple such as the "Get"
|
|
family), the handler should instead post an event for another thread
|
|
to process.
|
|
<p>
|
|
Note that the converse is safe: a Dalvik application thread which has
|
|
entered JNI is allowed to directly call OpenMAX AL APIs, including
|
|
those which block. However, blocking calls are not recommended from
|
|
the main thread, as they may result in the dreaded "Application Not
|
|
Responding" (ANR).
|
|
<p>
|
|
(Throughout this document, "Dalvik" can be considered to also refer to "ART").
|
|
|
|
<h3>Performance</h3>
|
|
|
|
As OpenMAX AL is a native C API, non-Dalvik application threads which
|
|
call OpenMAX AL have no Dalvik-related overhead such as garbage
|
|
collection pauses. However, there is no additional performance
|
|
benefit to the use of OpenMAX AL other than this. In particular, use
|
|
of OpenMAX AL does not result in lower audio or video latency, higher scheduling
|
|
priority, etc. than what the platform generally provides.
|
|
On the other hand, as the Android platform and specific device
|
|
implementations continue to evolve, an OpenMAX AL application can
|
|
expect to benefit from any future system performance improvements.
|
|
|
|
<h3>Security and permissions</h3>
|
|
|
|
As far as who can do what, security in Android is done at the
|
|
process level. Java programming language code can't do anything more than native code, nor
|
|
can native code do anything more than Java programming language code. The only differences
|
|
between them are what APIs are available that provide functionality
|
|
that the platform promises to support in the future and across
|
|
different devices.
|
|
<p>
|
|
Applications using OpenMAX AL must request whatever permissions they
|
|
would need for similar non-native APIs.
|
|
Applications that play
|
|
network resources need <code>android.permission.NETWORK</code>.
|
|
Note that the current Android implementation does not directly access the network,
|
|
but many applications that play multimedia receive their data via the network.
|
|
|
|
<h2>Platform issues</h2>
|
|
|
|
This section describes known issues in the initial platform
|
|
release which supports these APIs.
|
|
|
|
<h3>Dynamic interface management</h3>
|
|
|
|
<code>DynamicInterfaceManagement::AddInterface</code> does not work.
|
|
Instead, specify the interface in the array passed to Create.
|
|
|
|
<h2>References and resources</h2>
|
|
|
|
Android:
|
|
<ul>
|
|
<li><a href="http://developer.android.com/resources/index.html">
|
|
Android developer resources</a>
|
|
<li><a href="http://groups.google.com/group/android-developers">
|
|
Android developers discussion group</a>
|
|
<li><a href="http://developer.android.com/sdk/ndk/index.html">Android NDK</a>
|
|
<li><a href="http://groups.google.com/group/android-ndk">
|
|
Android NDK discussion group</a> (for developers of native code, including OpenMAX AL)
|
|
<li><a href="http://code.google.com/p/android/issues/">
|
|
Android open source bug database</a>
|
|
</ul>
|
|
|
|
Khronos Group:
|
|
<ul>
|
|
<li><a href="http://www.khronos.org/openmax/al/">
|
|
Khronos Group OpenMAX AL Overview</a>
|
|
<li><a href="http://www.khronos.org/registry/omxal/">
|
|
Khronos Group OpenMAX AL 1.0.1 specification</a>
|
|
<li><a href="http://www.khronos.org/message_boards/viewforum.php?f=30">
|
|
Khronos Group public message board for OpenMAX AL</a>
|
|
(please limit to non-Android questions)
|
|
</ul>
|
|
For convenience, we have included a copy of the OpenMAX AL 1.0.1
|
|
specification with the NDK in
|
|
<code>docs/openmaxal/OpenMAX_AL_1_0_1_Specification.pdf</code>.
|
|
|
|
<p>
|
|
Miscellaneous:
|
|
<ul>
|
|
<li><a href="http://en.wikipedia.org/wiki/Java_Native_Interface">JNI</a>
|
|
<li><a href="http://en.wikipedia.org/wiki/MPEG-2">MPEG-2</a>
|
|
<li><a href="http://en.wikipedia.org/wiki/MPEG_transport_stream">MPEG-2 transport stream</a>
|
|
<li><a href="http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC">H.264</a>
|
|
<li><a href="http://en.wikipedia.org/wiki/Advanced_Audio_Coding">AAC</a>
|
|
</ul>
|
|
|
|
</body>
|
|
</html>
|