321 lines
11 KiB
HTML
321 lines
11 KiB
HTML
<html devsite>
|
||
<head>
|
||
<title>Running the tests</title>
|
||
<meta name="project_path" value="/_project.yaml" />
|
||
<meta name="book_path" value="/_book.yaml" />
|
||
</head>
|
||
<body>
|
||
<!--
|
||
Copyright 2017 The Android Open Source Project
|
||
|
||
Licensed under the Apache License, Version 2.0 (the "License");
|
||
you may not use this file except in compliance with the License.
|
||
You may obtain a copy of the License at
|
||
|
||
http://www.apache.org/licenses/LICENSE-2.0
|
||
|
||
Unless required by applicable law or agreed to in writing, software
|
||
distributed under the License is distributed on an "AS IS" BASIS,
|
||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||
See the License for the specific language governing permissions and
|
||
limitations under the License.
|
||
-->
|
||
|
||
|
||
<p>This page provides instructions for running deqp tests in Linux and Windows
|
||
environments, using command line arguments, and working with the Android
|
||
application package.</p>
|
||
|
||
<h2 id=linux_and_windows_environments>Linux and Windows environments</h2>
|
||
|
||
<p>Start by copying the following files and directories to the target.</p>
|
||
|
||
<table>
|
||
<tr>
|
||
<th>Module</th>
|
||
<th>Directory</th>
|
||
<th>Target</th>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Execution Server</td>
|
||
<td><code>build/execserver/execserver</code></td>
|
||
<td><code><dst>/execserver</code></td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>EGL Module</td>
|
||
<td><code>build/modules/egl/deqp-egl</code></td>
|
||
<td><code><dst>/deqp-egl</code></td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td rowspan=2 style="vertical-align:middle">GLES2 Module</td>
|
||
<td><code>build/modules/gles2/deqp-gles2</code></td>
|
||
<td><code><dst>/deqp-gles2</code></td>
|
||
</tr>
|
||
|
||
|
||
<tr>
|
||
<td><code>data/gles2</code></td>
|
||
<td><code><dst>/gles2</code></td>
|
||
</tr>
|
||
|
||
|
||
|
||
<tr>
|
||
<td rowspan=2 style="vertical-align:middle">GLES3 Module</td>
|
||
<td><code>build/modules/gles3/deqp-gles3</code></td>
|
||
<td><code><dst>/deqp-gles3</code></td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td><code>data/gles3</code></td>
|
||
<td><code><dst>/gles3</code></td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td rowspan=2 style="vertical-align:middle">GLES3.1 Module</td>
|
||
<td><code>build/modules/gles31/deqp-gles31</code></td>
|
||
<td><code><dst>/deqp-gles31</code></td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td><code>data/gles31</code></td>
|
||
<td><code><dst>/gles31</code></td>
|
||
</tr>
|
||
|
||
|
||
<tr>
|
||
<td rowspan=2 style="vertical-align:middle">GLES3.2 Module</td>
|
||
<td><code>build/modules/gles32/deqp-gles32</code></td>
|
||
<td><code><dst>/deqp-gles32</code></td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td><code>data/gles32</code></td>
|
||
<td><code><dst>/gles32</code></td>
|
||
</tr>
|
||
|
||
</table>
|
||
|
||
<p>You can deploy the execution service and test binaries anywhere in the target
|
||
file system; however, test binaries expect to find data directories in the
|
||
current working directory. When ready, start the Test Execution Service on the
|
||
target device. For details on starting the service, see
|
||
<a href="/devices/graphics/port-tests.html#test_execution_service">Test
|
||
execution service</a>.</p>
|
||
|
||
<h2 id=command_line_arguments>Command line arguments</h2>
|
||
|
||
<p>The following table lists command line arguments that affect execution of all
|
||
test programs.</p>
|
||
|
||
<table width="100%">
|
||
<col style="width:50%">
|
||
<col style="width:50%">
|
||
<tr>
|
||
<th>Argument</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td><code>--deqp-case=<casename></code></td>
|
||
<td>Run cases that match a given pattern. Wildcard (*) is supported.</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td><code>--deqp-log-filename=<filename></code></td>
|
||
<td>Write test results to the file whose name you provide. The test execution
|
||
service will set the filename when starting a test.</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td><code>--deqp-stdin-caselist<br/>
|
||
--deqp-caselist=<caselist><br/>
|
||
--deqp-caselist-file=<filename></code></td>
|
||
<td>Read case list from stdin or from a given argument. The test execution
|
||
service will set the argument according to the execution request received. See
|
||
the next section for a description of the case list format.</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-test-iteration-count=<count></code></td>
|
||
<td>Override iteration count for tests that support a variable number of
|
||
iterations.</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-base-seed=<seed></code></td>
|
||
<td>Base seed for the test cases that use randomization.</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<h3 id=gles2_and_gles3-specific_arguments>GLES2 and GLES3-specific arguments</h3>
|
||
|
||
The following table lists the GLES2- and GLES3-specific arguments.
|
||
|
||
<table>
|
||
<table width="100%">
|
||
<col style="width:50%">
|
||
<col style="width:50%">
|
||
<tr>
|
||
<th>Argument</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-gl-context-type=<type></code></td>
|
||
<td>OpenGL context type. Available context types depend on the platform. On
|
||
platforms supporting EGL, the value <code>egl</code> can be used to select
|
||
the EGL context.</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-gl-config-id=<id></code></td>
|
||
<td>Run tests for the provided GL configuration ID. Interpretation is
|
||
platform-dependent. On the EGL platform, this is the EGL configuration ID.</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-gl-config-name=<name></code></td>
|
||
<td><p>Run tests for a named GL configuration. Interpretation is
|
||
platform-dependent. For EGL, the format is
|
||
<code>rgb(a)<bits>d<bits>s<bits></code>. For example, a
|
||
value of <code>rgb888s8</code> will select the first configuration where the
|
||
color buffer is RGB888 and the stencil buffer has 8 bits.</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-gl-context-flags=<flags></code></td>
|
||
<td>Creates a context. Specify <code>robust</code> or <code>debug</code>.</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-surface-width=<width><br/>
|
||
--deqp-surface-height=<height></code></td>
|
||
<td>Try to create a surface with a given size. Support for this is optional.</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-surface-type=<type></code></td>
|
||
<td>Use a given surface type as the main test rendering target. Possible
|
||
types are <code>window</code>, <code>pixmap</code>, <code>pbuffer</code>,
|
||
and <code>fbo</code>.</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>--deqp-screen-rotation=<rotation></code></td>
|
||
<td>Screen orientation in increments of 90 degrees for platforms that
|
||
support it.</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<h3 id=test_case_list_format>Test case list format</h3>
|
||
|
||
<p>The test case list can be given in two formats. The first option is to list
|
||
the full name of each test on a separate line in a standard ASCII file. As the
|
||
test sets grow, the repetitive prefixes can be cumbersome. To avoid repeating
|
||
the prefixes, use a trie (also known as a prefix tree) syntax shown below.</p>
|
||
|
||
<pre class="devsite-click-to-copy">
|
||
{nodeName{firstChild{…},…lastChild{…}}}
|
||
</pre>
|
||
|
||
<p>For example:</p>
|
||
|
||
<pre class="devsite-click-to-copy">
|
||
{dEQP-EGL{config-list,create_context{rgb565_depth_stencil}}}
|
||
</pre>
|
||
|
||
<p>Translates into the following two test cases:</p>
|
||
|
||
<pre class="devsite-click-to-copy">
|
||
dEQP-EGL.config_list
|
||
dEQP-EGL.create_context.rgb565_depth_stencil
|
||
</pre>
|
||
|
||
<h2 id=android>Android</h2>
|
||
|
||
<p>The Android application package contains all required components, including
|
||
the test execution service, test binaries, and data files. The test activity is
|
||
a <code>NativeActivity</code> that uses EGL (requires Android 3.2 or higher).</p>
|
||
|
||
<p>The application package can be installed with the following command (name
|
||
shown is the name of the APK in the Android CTS package; which name depends on
|
||
the build):</p>
|
||
<pre class="devsite-terminal devsite-click-to-copy">
|
||
adb –d install –r com.drawelements.deqp.apk
|
||
</pre>
|
||
|
||
<p>To launch the test execution service and to setup port forwarding, use the
|
||
following:</p>
|
||
<pre class="devsite-click-to-copy">
|
||
<code class="devsite-terminal">adb –d forward tcp:50016 tcp:50016</code>
|
||
<code class="devsite-terminal">adb –d shell am start –n com.drawelements.deqp/.execserver.ServiceStarter</code>
|
||
</pre>
|
||
|
||
<p>Debug prints can be enabled by executing the following before starting the
|
||
tests:</p>
|
||
|
||
<pre class="devsite-terminal devsite-click-to-copy">
|
||
adb –d shell setprop log.tag.dEQP DEBUG
|
||
</pre>
|
||
|
||
<h3 id=executing_tests_on_android_without_android_cts>Executing tests on
|
||
Android without Android CTS</h3>
|
||
|
||
<p>To manually start the test execution activity, construct an Android intent
|
||
that targets <code>android.app.NativeActivity</code>. The activities can be
|
||
found in the <code>com.drawelements.deqp</code> package. The command line must
|
||
be supplied as an extra string with key <code>"cmdLine"</code> in the Intent.</p>
|
||
|
||
<p>A test log is written to <code>/sdcard/dEQP-log.qpa</code>. If the test run
|
||
does not start normally, additional debug information is available in the device
|
||
log.</p>
|
||
|
||
<p>You can launch an activity from the command line using the <code>am</code>
|
||
utility. For example, to run <code>dEQP-GLES2.info</code> tests on a platform
|
||
supporting <code>NativeActivity,</code> use the following commands.</p>
|
||
|
||
<pre class="devsite-terminal devsite-click-to-copy">
|
||
adb -d shell am start -n com.drawelements.deqp/android.app.NativeActivity -e \
|
||
cmdLine "deqp --deqp-case=dEQP-GLES2.info.* --deqp-log-filename=/sdcard/dEQP-Log.qpa
|
||
</pre>
|
||
|
||
<h3 id=debugging_on_android>Debugging on Android</h3>
|
||
|
||
<p>To run the tests under the GDB debugger on Android, first compile and install
|
||
the debug build by running the following two scripts:</p>
|
||
|
||
<pre class="devsite-click-to-copy">
|
||
<code class="devsite-terminal">python android/scripts/build.py --native-build-type=Debug</code>
|
||
<code class="devsite-terminal">python android/scripts/install.py</code>
|
||
</pre>
|
||
|
||
<p>After the debug build is installed on the device, to launch the tests under
|
||
GDB running on the host, run the following command:</p>
|
||
|
||
<pre class="devsite-terminal devsite-click-to-copy">
|
||
python android/scripts/debug.py \
|
||
--deqp-commandline="--deqp-log-filename=/sdcard/TestLog.qpa --deqp-case=dEQP-GLES2.functional.*"
|
||
</pre>
|
||
|
||
<p>The deqp command line depends on the test cases to be executed and other
|
||
required parameters. The script adds a default breakpoint at the beginning of
|
||
the deqp execution (<code>tcu::App::App</code>).</p>
|
||
|
||
<p>The <code>debug.py</code> script accepts multiple command line arguments for
|
||
actions such as setting breakpoints for debugging, gdbserver connection
|
||
parameters, and paths to additional binaries to debug (use <code>debug.py
|
||
--help</code> for all arguments and explanations). The script also copies some
|
||
default libraries from the target device to get symbol listings.</p>
|
||
|
||
<p>To step through driver code (such as when the GDB needs to know the locations
|
||
of the binaries with full debug information), add more libraries via
|
||
<code>debug.py</code> command line parameters. This script writes out a
|
||
configuration file for the GDB starting from line 132 of the script file. You
|
||
can provide additional paths to binaries, etc., but supplying correct command
|
||
line parameters should be enough.</p>
|
||
|
||
<p class="note"><strong>Note:</strong> On Windows, the GDB binary requires
|
||
<code>libpython2.7.dll</code>. Before launching <code>debug.py</code>, add
|
||
<code><path-to-ndk>/prebuilt/windows/bin</code> to the PATH variable.</p>
|
||
|
||
<p class="note"><strong>Note:</strong> Native code debugging does not work on
|
||
stock Android 4.3; for workarounds, refer to <a href="https://issuetracker.google.com/issues/36976703">this public bug</a>.
|
||
Android 4.4 and higher do not contain this bug.</p>
|
||
|
||
</body>
|
||
</html>
|