FileDocCategorySizeDatePackage
InstrumentationTestRunner.javaAPI DocAndroid 5.1 API37472Thu Mar 12 22:22:42 GMT 2015android.test

InstrumentationTestRunner

public class InstrumentationTestRunner extends android.app.Instrumentation implements TestSuiteProvider
An {@link Instrumentation} that runs various types of {@link junit.framework.TestCase}s against an Android package (application).

Developer Guides

For more information about application testing, read the Testing developer guide.

Typical Usage

  1. Write {@link junit.framework.TestCase}s that perform unit, functional, or performance tests against the classes in your package. Typically these are subclassed from:
    • {@link android.test.ActivityInstrumentationTestCase2}
    • {@link android.test.ActivityUnitTestCase}
    • {@link android.test.AndroidTestCase}
    • {@link android.test.ApplicationTestCase}
    • {@link android.test.InstrumentationTestCase}
    • {@link android.test.ProviderTestCase}
    • {@link android.test.ServiceTestCase}
    • {@link android.test.SingleLaunchActivityTestCase}
  2. Set the android:targetPackage attribute of the <instrumentation> element in the test package's manifest. You should set the attribute value to the package name of the target application under test.
  3. Run the instrumentation using "adb shell am instrument -w", with no optional arguments, to run all tests (except performance tests).
  4. Run the instrumentation using "adb shell am instrument -w", with the argument '-e func true' to run all functional tests. These are tests that derive from {@link android.test.InstrumentationTestCase}.
  5. Run the instrumentation using "adb shell am instrument -w", with the argument '-e unit true' to run all unit tests. These are tests that do notderive from {@link android.test.InstrumentationTestCase} (and are not performance tests).
  6. Run the instrumentation using "adb shell am instrument -w", with the argument '-e class' set to run an individual {@link junit.framework.TestCase}.

Running all tests: adb shell am instrument -w com.android.foo/android.test.InstrumentationTestRunner

Running all small tests: adb shell am instrument -w -e size small com.android.foo/android.test.InstrumentationTestRunner

Running all medium tests: adb shell am instrument -w -e size medium com.android.foo/android.test.InstrumentationTestRunner

Running all large tests: adb shell am instrument -w -e size large com.android.foo/android.test.InstrumentationTestRunner

Filter test run to tests with given annotation: adb shell am instrument -w -e annotation com.android.foo.MyAnnotation com.android.foo/android.test.InstrumentationTestRunner

If used with other options, the resulting test run will contain the union of the two options. e.g. "-e size large -e annotation com.android.foo.MyAnnotation" will run only tests with both the {@link LargeTest} and "com.android.foo.MyAnnotation" annotations.

Filter test run to tests without given annotation: adb shell am instrument -w -e notAnnotation com.android.foo.MyAnnotation com.android.foo/android.test.InstrumentationTestRunner

Running a single testcase: adb shell am instrument -w -e class com.android.foo.FooTest com.android.foo/android.test.InstrumentationTestRunner

Running a single test: adb shell am instrument -w -e class com.android.foo.FooTest#testFoo com.android.foo/android.test.InstrumentationTestRunner

Running multiple tests: adb shell am instrument -w -e class com.android.foo.FooTest,com.android.foo.TooTest com.android.foo/android.test.InstrumentationTestRunner

Running all tests in a java package: adb shell am instrument -w -e package com.android.foo.subpkg com.android.foo/android.test.InstrumentationTestRunner

Including performance tests: adb shell am instrument -w -e perf true com.android.foo/android.test.InstrumentationTestRunner

To debug your tests, set a break point in your code and pass: -e debug true

To run in 'log only' mode -e log true This option will load and iterate through all test classes and methods, but will bypass actual test execution. Useful for quickly obtaining info on the tests to be executed by an instrumentation command.

To generate EMMA code coverage: -e coverage true Note: this requires an emma instrumented build. By default, the code coverage results file will be saved in a /data//coverage.ec file, unless overridden by coverageFile flag (see below)

To specify EMMA code coverage results file path: -e coverageFile /sdcard/myFile.ec
in addition to the other arguments.

Fields Summary
public static final String
ARGUMENT_TEST_CLASS
public static final String
ARGUMENT_TEST_PACKAGE
public static final String
ARGUMENT_TEST_SIZE_PREDICATE
public static final String
ARGUMENT_DELAY_MSEC
private static final String
SMALL_SUITE
private static final String
MEDIUM_SUITE
private static final String
LARGE_SUITE
private static final String
ARGUMENT_LOG_ONLY
static final String
ARGUMENT_ANNOTATION
static final String
ARGUMENT_NOT_ANNOTATION
private static final float
SMALL_SUITE_MAX_RUNTIME
This constant defines the maximum allowed runtime (in ms) for a test included in the "small" suite. It is used to make an educated guess at what suite an unlabeled test belongs.
private static final float
MEDIUM_SUITE_MAX_RUNTIME
This constant defines the maximum allowed runtime (in ms) for a test included in the "medium" suite. It is used to make an educated guess at what suite an unlabeled test belongs.
public static final String
REPORT_VALUE_ID
This value, if stored with key {@link android.app.Instrumentation#REPORT_KEY_IDENTIFIER}, identifies InstrumentationTestRunner as the source of the report. This is sent with all status messages.
public static final String
REPORT_KEY_NUM_TOTAL
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the total number of tests that are being run. This is sent with all status messages.
public static final String
REPORT_KEY_NUM_CURRENT
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the sequence number of the current test. This is sent with any status message describing a specific test being started or completed.
public static final String
REPORT_KEY_NAME_CLASS
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the name of the current test class. This is sent with any status message describing a specific test being started or completed.
public static final String
REPORT_KEY_NAME_TEST
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the name of the current test. This is sent with any status message describing a specific test being started or completed.
private static final String
REPORT_KEY_RUN_TIME
If included in the status or final bundle sent to an IInstrumentationWatcher, this key reports the run time in seconds of the current test.
private static final String
REPORT_KEY_NUM_ITERATIONS
If included in the status or final bundle sent to an IInstrumentationWatcher, this key reports the number of total iterations of the current test.
private static final String
REPORT_KEY_SUITE_ASSIGNMENT
If included in the status or final bundle sent to an IInstrumentationWatcher, this key reports the guessed suite assignment for the current test.
private static final String
REPORT_KEY_COVERAGE_PATH
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the path to the generated code coverage file.
public static final int
REPORT_VALUE_RESULT_START
The test is starting.
public static final int
REPORT_VALUE_RESULT_OK
The test completed successfully.
public static final int
REPORT_VALUE_RESULT_ERROR
The test completed with an error.
public static final int
REPORT_VALUE_RESULT_FAILURE
The test completed with a failure.
public static final String
REPORT_KEY_STACK
If included in the status bundle sent to an IInstrumentationWatcher, this key identifies a stack trace describing an error or failure. This is sent with any status message describing a specific test being completed.
private static final String
DEFAULT_COVERAGE_FILE_NAME
private static final String
LOG_TAG
private final android.os.Bundle
mResults
private android.os.Bundle
mArguments
private AndroidTestRunner
mTestRunner
private boolean
mDebug
private boolean
mJustCount
private boolean
mSuiteAssignmentMode
private int
mTestCount
private String
mPackageOfTests
private boolean
mCoverage
private String
mCoverageFilePath
private int
mDelayMsec
Constructors Summary
Methods Summary
protected voidaddTestListener(junit.framework.TestListener listener)
Add a {@link TestListener}

hide

        if(mTestRunner!=null && listener!=null){
            mTestRunner.addTestListener(listener);
        }
private voidgenerateCoverageReport()

        // use reflection to call emma dump coverage method, to avoid
        // always statically compiling against emma jar
        String coverageFilePath = getCoverageFilePath();
        java.io.File coverageFile = new java.io.File(coverageFilePath);
        try {
            Class<?> emmaRTClass = Class.forName("com.vladium.emma.rt.RT");
            Method dumpCoverageMethod = emmaRTClass.getMethod("dumpCoverageData",
                    coverageFile.getClass(), boolean.class, boolean.class);

            dumpCoverageMethod.invoke(null, coverageFile, false, false);
            // output path to generated coverage file so it can be parsed by a test harness if
            // needed
            mResults.putString(REPORT_KEY_COVERAGE_PATH, coverageFilePath);
            // also output a more user friendly msg
            final String currentStream = mResults.getString(
                    Instrumentation.REPORT_KEY_STREAMRESULT);
            mResults.putString(Instrumentation.REPORT_KEY_STREAMRESULT,
                String.format("%s\nGenerated code coverage data to %s", currentStream,
                coverageFilePath));
        } catch (ClassNotFoundException e) {
            reportEmmaError("Is emma jar on classpath?", e);
        } catch (SecurityException e) {
            reportEmmaError(e);
        } catch (NoSuchMethodException e) {
            reportEmmaError(e);
        } catch (IllegalArgumentException e) {
            reportEmmaError(e);
        } catch (IllegalAccessException e) {
            reportEmmaError(e);
        } catch (InvocationTargetException e) {
            reportEmmaError(e);
        }
public junit.framework.TestSuitegetAllTests()
Override this to define all of the tests to run in your package.

        return null;
protected AndroidTestRunnergetAndroidTestRunner()

        return new AndroidTestRunner();
private java.lang.ClassgetAnnotationClass(java.lang.String annotationClassName)
Helper method to return the annotation class with specified name

param
annotationClassName the fully qualified name of the class
return
the annotation class or null

        if (annotationClassName == null) {
            return null;
        }
        try {
           Class<?> annotationClass = Class.forName(annotationClassName);
           if (annotationClass.isAnnotation()) {
               return (Class<? extends Annotation>)annotationClass;
           } else {
               Log.e(LOG_TAG, String.format("Provided annotation value %s is not an Annotation",
                       annotationClassName));
           }
        } catch (ClassNotFoundException e) {
            Log.e(LOG_TAG, String.format("Could not find class for specified annotation %s",
                    annotationClassName));
        }
        return null;
private com.android.internal.util.PredicategetAnnotationPredicate(java.lang.String annotationClassName)
Returns the test predicate object, corresponding to the annotation class value provided via the {@link ARGUMENT_ANNOTATION} argument.

return
the predicate or null

        Class<? extends Annotation> annotationClass = getAnnotationClass(annotationClassName);
        if (annotationClass != null) {
            return new HasAnnotation(annotationClass);
        }
        return null;
public android.os.BundlegetArguments()
Get the arguments passed to this instrumentation.

return
the Bundle object

        return mArguments;
private booleangetBooleanArgument(android.os.Bundle arguments, java.lang.String tag)

        String tagString = arguments.getString(tag);
        return tagString != null && Boolean.parseBoolean(tagString);
java.util.ListgetBuilderRequirements()

        return new ArrayList<Predicate<TestMethod>>();
private java.lang.StringgetCoverageFilePath()

        if (mCoverageFilePath == null) {
            return getTargetContext().getFilesDir().getAbsolutePath() + File.separator +
                   DEFAULT_COVERAGE_FILE_NAME;
        } else {
            return mCoverageFilePath;
        }
public java.lang.ClassLoadergetLoader()
Override this to provide access to the class loader of your package.

        return null;
private com.android.internal.util.PredicategetNotAnnotationPredicate(java.lang.String annotationClassName)
Returns the negative test predicate object, corresponding to the annotation class value provided via the {@link ARGUMENT_NOT_ANNOTATION} argument.

return
the predicate or null

         Class<? extends Annotation> annotationClass = getAnnotationClass(annotationClassName);
         if (annotationClass != null) {
             return Predicates.not(new HasAnnotation(annotationClass));
         }
         return null;
private com.android.internal.util.PredicategetSizePredicateFromArg(java.lang.String sizeArg)


        if (SMALL_SUITE.equals(sizeArg)) {
            return TestPredicates.SELECT_SMALL;
        } else if (MEDIUM_SUITE.equals(sizeArg)) {
            return TestPredicates.SELECT_MEDIUM;
        } else if (LARGE_SUITE.equals(sizeArg)) {
            return TestPredicates.SELECT_LARGE;
        } else {
            return null;
        }
public junit.framework.TestSuitegetTestSuite()

        return getAllTests();
public voidonCreate(android.os.Bundle arguments)


    
        
        super.onCreate(arguments);
        mArguments = arguments;

        // Apk paths used to search for test classes when using TestSuiteBuilders.
        String[] apkPaths =
                {getTargetContext().getPackageCodePath(), getContext().getPackageCodePath()};
        ClassPathPackageInfoSource.setApkPaths(apkPaths);

        Predicate<TestMethod> testSizePredicate = null;
        Predicate<TestMethod> testAnnotationPredicate = null;
        Predicate<TestMethod> testNotAnnotationPredicate = null;
        String testClassesArg = null;
        boolean logOnly = false;

        if (arguments != null) {
            // Test class name passed as an argument should override any meta-data declaration.
            testClassesArg = arguments.getString(ARGUMENT_TEST_CLASS);
            mDebug = getBooleanArgument(arguments, "debug");
            mJustCount = getBooleanArgument(arguments, "count");
            mSuiteAssignmentMode = getBooleanArgument(arguments, "suiteAssignment");
            mPackageOfTests = arguments.getString(ARGUMENT_TEST_PACKAGE);
            testSizePredicate = getSizePredicateFromArg(
                    arguments.getString(ARGUMENT_TEST_SIZE_PREDICATE));
            testAnnotationPredicate = getAnnotationPredicate(
                    arguments.getString(ARGUMENT_ANNOTATION));
            testNotAnnotationPredicate = getNotAnnotationPredicate(
                    arguments.getString(ARGUMENT_NOT_ANNOTATION));

            logOnly = getBooleanArgument(arguments, ARGUMENT_LOG_ONLY);
            mCoverage = getBooleanArgument(arguments, "coverage");
            mCoverageFilePath = arguments.getString("coverageFile");

            try {
                Object delay = arguments.get(ARGUMENT_DELAY_MSEC);  // Accept either string or int
                if (delay != null) mDelayMsec = Integer.parseInt(delay.toString());
            } catch (NumberFormatException e) {
                Log.e(LOG_TAG, "Invalid delay_msec parameter", e);
            }
        }

        TestSuiteBuilder testSuiteBuilder = new TestSuiteBuilder(getClass().getName(),
                getTargetContext().getClassLoader());

        if (testSizePredicate != null) {
            testSuiteBuilder.addRequirements(testSizePredicate);
        }
        if (testAnnotationPredicate != null) {
            testSuiteBuilder.addRequirements(testAnnotationPredicate);
        }
        if (testNotAnnotationPredicate != null) {
            testSuiteBuilder.addRequirements(testNotAnnotationPredicate);
        }

        if (testClassesArg == null) {
            if (mPackageOfTests != null) {
                testSuiteBuilder.includePackages(mPackageOfTests);
            } else {
                TestSuite testSuite = getTestSuite();
                if (testSuite != null) {
                    testSuiteBuilder.addTestSuite(testSuite);
                } else {
                    // no package or class bundle arguments were supplied, and no test suite
                    // provided so add all tests in application
                    testSuiteBuilder.includePackages("");
                }
            }
        } else {
            parseTestClasses(testClassesArg, testSuiteBuilder);
        }

        testSuiteBuilder.addRequirements(getBuilderRequirements());

        mTestRunner = getAndroidTestRunner();
        mTestRunner.setContext(getTargetContext());
        mTestRunner.setInstrumentation(this);
        mTestRunner.setSkipExecution(logOnly);
        mTestRunner.setTest(testSuiteBuilder.build());
        mTestCount = mTestRunner.getTestCases().size();
        if (mSuiteAssignmentMode) {
            mTestRunner.addTestListener(new SuiteAssignmentPrinter());
        } else {
            WatcherResultPrinter resultPrinter = new WatcherResultPrinter(mTestCount);
            mTestRunner.addTestListener(new TestPrinter("TestRunner", false));
            mTestRunner.addTestListener(resultPrinter);
            mTestRunner.setPerformanceResultsWriter(resultPrinter);
        }
        start();
public voidonStart()

        prepareLooper();

        if (mJustCount) {
            mResults.putString(Instrumentation.REPORT_KEY_IDENTIFIER, REPORT_VALUE_ID);
            mResults.putInt(REPORT_KEY_NUM_TOTAL, mTestCount);
            finish(Activity.RESULT_OK, mResults);
        } else {
            if (mDebug) {
                Debug.waitForDebugger();
            }

            ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
            PrintStream writer = new PrintStream(byteArrayOutputStream);
            try {
                StringResultPrinter resultPrinter = new StringResultPrinter(writer);

                mTestRunner.addTestListener(resultPrinter);

                long startTime = System.currentTimeMillis();
                mTestRunner.runTest();
                long runTime = System.currentTimeMillis() - startTime;

                resultPrinter.printResult(mTestRunner.getTestResult(), runTime);
            } catch (Throwable t) {
                // catch all exceptions so a more verbose error message can be outputted
                writer.println(String.format("Test run aborted due to unexpected exception: %s",
                                t.getMessage()));
                t.printStackTrace(writer);
            } finally {
                mResults.putString(Instrumentation.REPORT_KEY_STREAMRESULT,
                        String.format("\nTest results for %s=%s",
                        mTestRunner.getTestClassName(),
                        byteArrayOutputStream.toString()));

                if (mCoverage) {
                    generateCoverageReport();
                }
                writer.close();

                finish(Activity.RESULT_OK, mResults);
            }
        }
private voidparseTestClass(java.lang.String testClassName, android.test.suitebuilder.TestSuiteBuilder testSuiteBuilder)
Parse and load the given test class and, optionally, method

param
testClassName - full package name of test class and optionally method to add. Expected format: com.android.TestClass#testMethod
param
testSuiteBuilder - builder to add tests to

        int methodSeparatorIndex = testClassName.indexOf('#");
        String testMethodName = null;

        if (methodSeparatorIndex > 0) {
            testMethodName = testClassName.substring(methodSeparatorIndex + 1);
            testClassName = testClassName.substring(0, methodSeparatorIndex);
        }
        testSuiteBuilder.addTestClassByName(testClassName, testMethodName, getTargetContext());
private voidparseTestClasses(java.lang.String testClassArg, android.test.suitebuilder.TestSuiteBuilder testSuiteBuilder)
Parses and loads the specified set of test classes

param
testClassArg - comma-separated list of test classes and methods
param
testSuiteBuilder - builder to add tests to

        String[] testClasses = testClassArg.split(",");
        for (String testClass : testClasses) {
            parseTestClass(testClass, testSuiteBuilder);
        }
voidprepareLooper()
Initialize the current thread as a looper.

Exposed for unit testing.

        Looper.prepare();
private voidreportEmmaError(java.lang.Exception e)

        reportEmmaError("", e);
private voidreportEmmaError(java.lang.String hint, java.lang.Exception e)

        String msg = "Failed to generate emma coverage. " + hint;
        Log.e(LOG_TAG, msg, e);
        mResults.putString(Instrumentation.REPORT_KEY_STREAMRESULT, "\nError: " + msg);