FileDocCategorySizeDatePackage
InstrumentationTestRunner.javaAPI DocAndroid 1.5 API27220Wed May 06 22:42:02 BST 2009android.test

InstrumentationTestRunner

public class InstrumentationTestRunner extends android.app.Instrumentation implements TestSuiteProvider
An {@link Instrumentation} that runs various types of {@link junit.framework.TestCase}s against an Android package (application). Typical usage:
  1. Write {@link junit.framework.TestCase}s that perform unit, functional, or performance tests against the classes in your package. Typically these are subclassed from:
    • {@link android.test.ActivityInstrumentationTestCase}
    • {@link android.test.ActivityUnitTestCase}
    • {@link android.test.AndroidTestCase}
    • {@link android.test.ApplicationTestCase}
    • {@link android.test.InstrumentationTestCase}
    • {@link android.test.ProviderTestCase}
    • {@link android.test.ServiceTestCase}
    • {@link android.test.SingleLaunchActivityTestCase}
  2. In an appropriate AndroidManifest.xml, define the this instrumentation with the appropriate android:targetPackage set.
  3. Run the instrumentation using "adb shell am instrument -w", with no optional arguments, to run all tests (except performance tests).
  4. Run the instrumentation using "adb shell am instrument -w", with the argument '-e func true' to run all functional tests. These are tests that derive from {@link android.test.InstrumentationTestCase}.
  5. Run the instrumentation using "adb shell am instrument -w", with the argument '-e unit true' to run all unit tests. These are tests that do notderive from {@link android.test.InstrumentationTestCase} (and are not performance tests).
  6. Run the instrumentation using "adb shell am instrument -w", with the argument '-e class' set to run an individual {@link junit.framework.TestCase}.

Running all tests: adb shell am instrument -w com.android.foo/android.test.InstrumentationTestRunner

Running all small tests: adb shell am instrument -w -e size small com.android.foo/android.test.InstrumentationTestRunner

Running all medium tests: adb shell am instrument -w -e size medium com.android.foo/android.test.InstrumentationTestRunner

Running all large tests: adb shell am instrument -w -e size large com.android.foo/android.test.InstrumentationTestRunner

Running a single testcase: adb shell am instrument -w -e class com.android.foo.FooTest com.android.foo/android.test.InstrumentationTestRunner

Running a single test: adb shell am instrument -w -e class com.android.foo.FooTest#testFoo com.android.foo/android.test.InstrumentationTestRunner

Running multiple tests: adb shell am instrument -w -e class com.android.foo.FooTest,com.android.foo.TooTest com.android.foo/android.test.InstrumentationTestRunner

Including performance tests: adb shell am instrument -w -e perf true com.android.foo/android.test.InstrumentationTestRunner

To debug your tests, set a break point in your code and pass: -e debug true

To run in 'log only' mode -e log true This option will load and iterate through all test classes and methods, but will bypass actual test execution. Useful for quickly obtaining info on the tests to be executed by an instrumentation command.

To generate EMMA code coverage: -e coverage true Note: this requires an emma instrumented build. By default, the code coverage results file will be saved as /sdcard/coverage.ec, unless overridden by coverageFile flag (see below)

To specify EMMA code coverage results file path: -e coverageFile /sdcard/myFile.ec
in addition to the other arguments.

Fields Summary
public static final String
ARGUMENT_TEST_CLASS
public static final String
ARGUMENT_TEST_PACKAGE
public static final String
ARGUMENT_TEST_SIZE_PREDICATE
public static final String
ARGUMENT_INCLUDE_PERF
public static final String
ARGUMENT_DELAY_MSEC
private static final String
SMALL_SUITE
private static final String
MEDIUM_SUITE
private static final String
LARGE_SUITE
private static final String
ARGUMENT_LOG_ONLY
private static final float
SMALL_SUITE_MAX_RUNTIME
This constant defines the maximum allowed runtime (in ms) for a test included in the "small" suite. It is used to make an educated guess at what suite an unlabeled test belongs.
private static final float
MEDIUM_SUITE_MAX_RUNTIME
This constant defines the maximum allowed runtime (in ms) for a test included in the "medium" suite. It is used to make an educated guess at what suite an unlabeled test belongs.
public static final String
REPORT_VALUE_ID
This value, if stored with key {@link android.app.Instrumentation#REPORT_KEY_IDENTIFIER}, identifies InstrumentationTestRunner as the source of the report. This is sent with all status messages.
public static final String
REPORT_KEY_NUM_TOTAL
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the total number of tests that are being run. This is sent with all status messages.
public static final String
REPORT_KEY_NUM_CURRENT
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the sequence number of the current test. This is sent with any status message describing a specific test being started or completed.
public static final String
REPORT_KEY_NAME_CLASS
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the name of the current test class. This is sent with any status message describing a specific test being started or completed.
public static final String
REPORT_KEY_NAME_TEST
If included in the status or final bundle sent to an IInstrumentationWatcher, this key identifies the name of the current test. This is sent with any status message describing a specific test being started or completed.
private static final String
REPORT_KEY_RUN_TIME
If included in the status or final bundle sent to an IInstrumentationWatcher, this key reports the run time in seconds of the current test.
private static final String
REPORT_KEY_SUITE_ASSIGNMENT
If included in the status or final bundle sent to an IInstrumentationWatcher, this key reports the guessed suite assignment for the current test.
public static final int
REPORT_VALUE_RESULT_START
The test is starting.
public static final int
REPORT_VALUE_RESULT_OK
The test completed successfully.
public static final int
REPORT_VALUE_RESULT_ERROR
The test completed with an error.
public static final int
REPORT_VALUE_RESULT_FAILURE
The test completed with a failure.
public static final String
REPORT_KEY_STACK
If included in the status bundle sent to an IInstrumentationWatcher, this key identifies a stack trace describing an error or failure. This is sent with any status message describing a specific test being completed.
private static final String
DEFAULT_COVERAGE_FILE_PATH
private static final String
LOG_TAG
private final android.os.Bundle
mResults
private AndroidTestRunner
mTestRunner
private boolean
mDebug
private boolean
mJustCount
private boolean
mSuiteAssignmentMode
private int
mTestCount
private String
mPackageOfTests
private boolean
mCoverage
private String
mCoverageFilePath
private int
mDelayMsec
Constructors Summary
Methods Summary
private voidgenerateCoverageReport()

        // use reflection to call emma dump coverage method, to avoid
        // always statically compiling against emma jar
        java.io.File coverageFile = new java.io.File(getCoverageFilePath());
        try {
            Class emmaRTClass = Class.forName("com.vladium.emma.rt.RT");
            Method dumpCoverageMethod = emmaRTClass.getMethod("dumpCoverageData", 
                    coverageFile.getClass(), boolean.class, boolean.class);
            
            dumpCoverageMethod.invoke(null, coverageFile, false, false);

        } catch (ClassNotFoundException e) {
            reportEmmaError("Is emma jar on classpath?", e);
        } catch (SecurityException e) {
            reportEmmaError(e);
        } catch (NoSuchMethodException e) {
            reportEmmaError(e);
        } catch (IllegalArgumentException e) {
            reportEmmaError(e);
        } catch (IllegalAccessException e) {
            reportEmmaError(e);
        } catch (InvocationTargetException e) {
            reportEmmaError(e);
        }
public junit.framework.TestSuitegetAllTests()
Override this to define all of the tests to run in your package.

        return null;
protected AndroidTestRunnergetAndroidTestRunner()

        return new AndroidTestRunner();
private booleangetBooleanArgument(android.os.Bundle arguments, java.lang.String tag)

        String tagString = arguments.getString(tag);
        return tagString != null && Boolean.parseBoolean(tagString);
java.util.ListgetBuilderRequirements()

        return new ArrayList<Predicate<TestMethod>>();
private java.lang.StringgetCoverageFilePath()

        if (mCoverageFilePath == null) {
            return DEFAULT_COVERAGE_FILE_PATH;
        }
        else {
            return mCoverageFilePath;
        }
public java.lang.ClassLoadergetLoader()
Override this to provide access to the class loader of your package.

        return null;
private com.android.internal.util.PredicategetSizePredicateFromArg(java.lang.String sizeArg)

     
        if (SMALL_SUITE.equals(sizeArg)) {
            return TestPredicates.SELECT_SMALL;
        } else if (MEDIUM_SUITE.equals(sizeArg)) {
            return TestPredicates.SELECT_MEDIUM;
        } else if (LARGE_SUITE.equals(sizeArg)) {
            return TestPredicates.SELECT_LARGE;
        } else {
            return null;
        }
public junit.framework.TestSuitegetTestSuite()

        return getAllTests();
public voidonCreate(android.os.Bundle arguments)


    
        
        super.onCreate(arguments);

        // Apk paths used to search for test classes when using TestSuiteBuilders.
        String[] apkPaths =
                {getTargetContext().getPackageCodePath(), getContext().getPackageCodePath()};
        ClassPathPackageInfoSource.setApkPaths(apkPaths);

        Predicate<TestMethod> testSizePredicate = null;
        boolean includePerformance = false;
        String testClassesArg = null;
        boolean logOnly = false;

        if (arguments != null) {
            // Test class name passed as an argument should override any meta-data declaration.
            testClassesArg = arguments.getString(ARGUMENT_TEST_CLASS);
            mDebug = getBooleanArgument(arguments, "debug");
            mJustCount = getBooleanArgument(arguments, "count");
            mSuiteAssignmentMode = getBooleanArgument(arguments, "suiteAssignment");
            mPackageOfTests = arguments.getString(ARGUMENT_TEST_PACKAGE);
            testSizePredicate = getSizePredicateFromArg(
                    arguments.getString(ARGUMENT_TEST_SIZE_PREDICATE));
            includePerformance = getBooleanArgument(arguments, ARGUMENT_INCLUDE_PERF);
            logOnly = getBooleanArgument(arguments, ARGUMENT_LOG_ONLY);
            mCoverage = getBooleanArgument(arguments, "coverage");
            mCoverageFilePath = arguments.getString("coverageFile");

            try {
                Object delay = arguments.get(ARGUMENT_DELAY_MSEC);  // Accept either string or int
                if (delay != null) mDelayMsec = Integer.parseInt(delay.toString());
            } catch (NumberFormatException e) {
                Log.e(LOG_TAG, "Invalid delay_msec parameter", e);
            }
        }

        TestSuiteBuilder testSuiteBuilder = new TestSuiteBuilder(getClass().getName(),
                getTargetContext().getClassLoader());

        if (testSizePredicate != null) {
            testSuiteBuilder.addRequirements(testSizePredicate);
        }
        if (!includePerformance) {
            testSuiteBuilder.addRequirements(REJECT_PERFORMANCE);
        }

        if (testClassesArg == null) {
            if (mPackageOfTests != null) {
                testSuiteBuilder.includePackages(mPackageOfTests);
            } else {
                TestSuite testSuite = getTestSuite();
                if (testSuite != null) {
                    testSuiteBuilder.addTestSuite(testSuite);
                } else {
                    // no package or class bundle arguments were supplied, and no test suite 
                    // provided so add all tests in application
                    testSuiteBuilder.includePackages("");
                }
            }
        } else {
            parseTestClasses(testClassesArg, testSuiteBuilder);
        }
        
        testSuiteBuilder.addRequirements(getBuilderRequirements());

        mTestRunner = getAndroidTestRunner();
        mTestRunner.setContext(getTargetContext());
        mTestRunner.setInstrumentaiton(this);
        mTestRunner.setSkipExecution(logOnly);
        mTestRunner.setTest(testSuiteBuilder.build());
        mTestCount = mTestRunner.getTestCases().size();
        if (mSuiteAssignmentMode) {
            mTestRunner.addTestListener(new SuiteAssignmentPrinter());
        } else {
            mTestRunner.addTestListener(new TestPrinter("TestRunner", false));
            mTestRunner.addTestListener(new WatcherResultPrinter(mTestCount));
        }
        start();
public voidonStart()

        Looper.prepare();
        
        if (mJustCount) {
            mResults.putString(Instrumentation.REPORT_KEY_IDENTIFIER, REPORT_VALUE_ID);
            mResults.putInt(REPORT_KEY_NUM_TOTAL, mTestCount);
            finish(Activity.RESULT_OK, mResults);
        } else {
            if (mDebug) {
                Debug.waitForDebugger();
            }
    
            ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
            PrintStream writer = new PrintStream(byteArrayOutputStream);
            try {
                StringResultPrinter resultPrinter = new StringResultPrinter(writer);
    
                mTestRunner.addTestListener(resultPrinter);
                
                long startTime = System.currentTimeMillis();
                mTestRunner.runTest();
                long runTime = System.currentTimeMillis() - startTime;
    
                resultPrinter.print(mTestRunner.getTestResult(), runTime);
            } finally {
                mResults.putString(Instrumentation.REPORT_KEY_STREAMRESULT, 
                        String.format("\nTest results for %s=%s", 
                        mTestRunner.getTestClassName(), 
                        byteArrayOutputStream.toString()));

                if (mCoverage) {
                    generateCoverageReport();
                }
                writer.close();
                
                finish(Activity.RESULT_OK, mResults);
            }
        }
private voidparseTestClass(java.lang.String testClassName, android.test.suitebuilder.TestSuiteBuilder testSuiteBuilder)
Parse and load the given test class and, optionally, method

param
testClassName - full package name of test class and optionally method to add. Expected format: com.android.TestClass#testMethod
param
testSuiteBuilder - builder to add tests to

        int methodSeparatorIndex = testClassName.indexOf('#");
        String testMethodName = null;

        if (methodSeparatorIndex > 0) {
            testMethodName = testClassName.substring(methodSeparatorIndex + 1);
            testClassName = testClassName.substring(0, methodSeparatorIndex);
        }
        testSuiteBuilder.addTestClassByName(testClassName, testMethodName, 
                getTargetContext());
private voidparseTestClasses(java.lang.String testClassArg, android.test.suitebuilder.TestSuiteBuilder testSuiteBuilder)
Parses and loads the specified set of test classes

param
testClassArg - comma-separated list of test classes and methods
param
testSuiteBuilder - builder to add tests to

        String[] testClasses = testClassArg.split(",");
        for (String testClass : testClasses) {
            parseTestClass(testClass, testSuiteBuilder);
        }
private voidreportEmmaError(java.lang.Exception e)

        reportEmmaError("", e);
private voidreportEmmaError(java.lang.String hint, java.lang.Exception e)

        String msg = "Failed to generate emma coverage. " + hint;
        Log.e(LOG_TAG, msg, e);
        mResults.putString(Instrumentation.REPORT_KEY_STREAMRESULT, "\nError: " + msg);