143 lines
5.8 KiB
HTML
143 lines
5.8 KiB
HTML
<html devsite>
|
|
<head>
|
|
<title>Compatibility Test Suite</title>
|
|
<meta name="project_path" value="/_project.yaml" />
|
|
<meta name="book_path" value="/_book.yaml" />
|
|
</head>
|
|
<body>
|
|
<!--
|
|
Copyright 2017 The Android Open Source Project
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
-->
|
|
|
|
|
|
|
|
<h2 id="how-does-the-cts-work">How does the CTS work?</h2>
|
|
|
|
<p>The Compatibility Test Suite (CTS) is a free, commercial-grade test suite,
|
|
available for <a href="downloads.html">download</a>. The CTS represents the
|
|
"mechanism" of compatibility.</p>
|
|
|
|
<p>The CTS runs on a desktop machine and executes test cases directly on
|
|
attached devices or an emulator. The CTS is a set of unit tests designed to be
|
|
integrated into the daily workflow (such as via a continuous build system) of
|
|
the engineers building a device. Its intent is to reveal incompatibilities
|
|
early on, and ensure that the software remains compatible throughout the
|
|
development process.</p>
|
|
|
|
|
|
<p>The CTS is an automated testing harness that includes two major software components:</p>
|
|
<ul>
|
|
<li>
|
|
<p>The CTS tradefed test harness runs on your desktop machine and manages test execution.</p>
|
|
</li>
|
|
<li>
|
|
<p>Individual test cases are executed on the Device Under Test (DUT). The test
|
|
cases are written in Java as JUnit tests and packaged as
|
|
Android .apk files to run on the actual device target.</p>
|
|
</li>
|
|
</ul>
|
|
|
|
<p>The Compatibility Test Suite Verifier (CTS Verifier) is a supplement to the
|
|
CTS available for <a href="downloads.html">download</a>. CTS Verifier
|
|
provides tests for APIs and functions that cannot be tested on a stationary
|
|
device without manual input (e.g. audio quality, accelerometer, etc).</p>
|
|
|
|
<p>The CTS Verifier is a tool for manual testing and includes the following software components:</p>
|
|
<ul>
|
|
<li>
|
|
<p>The CTS verifier app that is executed on the DUT and collects the results.<p>
|
|
</li>
|
|
<li>
|
|
<p>The executable(s) or script(s) that are executed on the desktop machine to
|
|
provide data or additional control for some test cases in the CTS Verifier
|
|
app.</p>
|
|
</li>
|
|
</ul>
|
|
|
|
<h2 id="workflow">Workflow</h2>
|
|
|
|
<div class="attempt-right" style="width:383px">
|
|
<img src="images/cts-0.png" alt="CTS flow" height="340px" id="figure1" />
|
|
<p class="img-caption">
|
|
<strong>Figure 1.</strong> How to use CTS
|
|
</p>
|
|
</div>
|
|
|
|
<p>This diagram summarizes CTS workflow. Please refer to the subpages of this
|
|
section starting with <a href="setup.html">Setup</a> for detailed
|
|
instructions.</p>
|
|
|
|
<h2 id="types-of-test-cases">Types of test cases</h2>
|
|
<p>The CTS includes the following types of test cases:</p>
|
|
<ul>
|
|
<li>
|
|
<p><em>Unit tests</em> test atomic units of code within the Android platform; e.g. a single class, such as java.util.HashMap.</p>
|
|
</li>
|
|
<li>
|
|
<p><em>Functional tests</em> test a combination of APIs together in a higher-level use-case.</p>
|
|
</li>
|
|
</ul>
|
|
<p>Future versions of the CTS will include the following types of test cases:</p>
|
|
<ul>
|
|
<li>
|
|
<p><em>Robustness tests</em> test the durability of the system under stress.</p>
|
|
</li>
|
|
<li>
|
|
<p><em>Performance tests</em> test the performance of the system against defined benchmarks, for example rendering frames per second.</p>
|
|
</li>
|
|
</ul>
|
|
<h2 id="areas-covered">Areas covered</h2>
|
|
<p>The unit test cases cover the following areas to ensure compatibility:</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>Area</th>
|
|
<th>Description</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
<tr>
|
|
<td>Signature tests</td>
|
|
<td>For each Android release, there are XML files describing the signatures of all public APIs contained in the release. The CTS contains a utility to check those API signatures against the APIs available on the device. The results from signature checking are recorded in the test result XML file.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Platform API Tests</td>
|
|
<td>Test the platform (core libraries and Android Application Framework) APIs as documented in the SDK <a href="https://developer.android.com/reference/classes.html">Class Index</a> to ensure API correctness, including correct class, attribute and method signatures, correct method behavior, and negative tests to ensure expected behavior for incorrect parameter handling.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Dalvik Tests</td>
|
|
<td>The tests focus on testing the Dalvik Executable Format.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Platform Data Model</td>
|
|
<td>The CTS tests the core platform data model as exposed to application developers through content providers, as documented in the SDK <a href="https://developer.android.com/reference/android/provider/package-summary.html">android.provider</a> package: contacts, browser, settings, etc.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Platform Intents</td>
|
|
<td>The CTS tests the core platform intents, as documented in the SDK <a href="https://developer.android.com/guide/appendix/g-app-intents.html">Available Intents</a>.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Platform Permissions</td>
|
|
<td>The CTS tests the core platform permissions, as documented in the SDK <a href="https://developer.android.com/reference/android/Manifest.permission.html">Available Permissions</a>.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Platform Resources</td>
|
|
<td>The CTS tests for correct handling of the core platform resource types, as documented in the SDK <a href="https://developer.android.com/guide/topics/resources/available-resources.html">Available Resource Types</a>. This includes tests for: simple values, drawables, nine-patch, animations, layouts, styles and themes, and loading alternate resources.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
</body>
|
|
</html>
|