1 /* 2 * Copyright 2007 Google Inc. 3 * 4 * Licensed under the Apache License, Version 2.0 (the "License"); you may not 5 * use this file except in compliance with the License. You may obtain a copy of 6 * the License at 7 * 8 * http://www.apache.org/licenses/LICENSE-2.0 9 * 10 * Unless required by applicable law or agreed to in writing, software 11 * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT 12 * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the 13 * License for the specific language governing permissions and limitations under 14 * the License. 15 */ 16 package com.google.gwt.junit.client; 17 18 /** 19 * A type of {@link com.google.gwt.junit.client.GWTTestCase} which specifically 20 * records performance results. {@link com.google.gwt.junit.client.Benchmark}s 21 * have additional functionality above and beyond GWT's JUnit support for 22 * standard <code>TestCases</code>. 23 * 24 * <ul> 25 * <li>In a single <code>JUnit</code> run, the results of all executed 26 * benchmarks are collected and stored in an XML report viewable with the 27 * <code>benchmarkViewer</code>.</li> 28 * 29 * <li>GWT automatically removes jitter from your benchmark methods by running 30 * them for a minimum period of time (150ms). GWT also optionally limits your 31 * benchmark execution to a maximum period of time (1000ms).</li> 32 * 33 * <li>GWT supports "begin" and "end" test methods that separate setup and 34 * teardown costs from the actual work being benchmarked. Simply name your 35 * functions "begin[TestMethodName]" and "end[TestMethodName]" and they will 36 * be executed before and after every execution of your test method. The 37 * timings of these setup methods are not included in the test results.</li> 38 * 39 * <li>GWT supports test methods that have parameters. GWT will execute each 40 * benchmark method multiple times in order to exhaustively test all the possible 41 * combinations of parameter values. For each parameter that your test method 42 * accepts, it should document it with the annotation, 43 * <code>@gwt.benchmark.param</code>. 44 * 45 * <p>The syntax for gwt.benchmark.param is 46 * <code><param name> = <Iterable></code>. For example, 47 * 48 * <pre> 49 * @gwt.benchmark.param where = java.util.Arrays.asList( 50 * new Position[] { Position.BEGIN, Position.END, Position.VARIED } ) 51 * @gwt.benchmark.param size -limit = insertRemoveRange 52 * public void testArrayListRemoves(Position where, Integer size) { ... } 53 * </pre></p> 54 * 55 * <p>In this example, the annotated function is executed with all the possible 56 * permutations of <code>Position = (BEGIN, END, and VARIED)</code> and 57 * <code>insertRemoveRange = IntRange( 64, Integer.MAX_VALUE, "*", 2 )</code>. 58 * </p> 59 * 60 * <p>This particular example also demonstrates how GWT can automatically limit 61 * the number of executions of your test. Your final parameter (in this example, 62 * size) can optionally be decorated with -limit to indicate to GWT that 63 * it should stop executing additional permutations of the test when the 64 * execution time becomes too long (over 1000ms). So, in this example, 65 * for each value of <code>Position</code>, <code>testArrayListRemoves</code> 66 * will be executed for increasing values of <code>size</code> (beginning with 67 * 64 and increasing in steps of 2), until either it reaches 68 * <code>Integer.MAX_VALUE</code> or the execution time for the last 69 * permutation is > 1000ms.</p> 70 * </li> 71 * </ul> 72 * 73 * <p>{@link Benchmark}s support the following annotations on each test method 74 * in order to decorate each test with additional information useful for 75 * reporting.</p> 76 * 77 * <ul> 78 * <li>@gwt.benchmark.category - The class name of the {@link Category} the 79 * benchmark belongs to. This property may also be set at the 80 * {@link com.google.gwt.junit.client.Benchmark} class level.</li> 81 * </ul> 82 * 83 * <p>Please note that {@link Benchmark}s do not currently support asynchronous 84 * testing mode. Calling 85 * {@link com.google.gwt.junit.client.GWTTestCase#delayTestFinish(int)} 86 * or {@link com.google.gwt.junit.client.GWTTestCase#finishTest()} will result 87 * in an UnsupportedOperationException.</p> 88 * 89 * <h2>Examples of benchmarking in action</h2> 90 * 91 * <h3>A simple benchmark example</h3> 92 * {@link com.google.gwt.examples.benchmarks.AllocBenchmark} is a simple example 93 * of a basic benchmark that doesn't take advantage of most of benchmarking's 94 * advanced features. 95 * 96 * {@example com.google.gwt.examples.benchmarks.AllocBenchmark} 97 * 98 * <h3>An advanced benchmark example</h3> 99 * {@link com.google.gwt.examples.benchmarks.ArrayListAndVectorBenchmark} is a more 100 * sophisticated example of benchmarking. It demonstrates the use of "begin" 101 * and "end" test methods, parameterized test methods, and automatic 102 * test execution limits. 103 * 104 * {@example com.google.gwt.examples.benchmarks.ArrayListAndVectorBenchmark} 105 */ 106 public abstract class Benchmark extends GWTTestCase { 107 108 /** 109 * The name of the system property that specifies the location 110 * where benchmark reports are both written to and read from. 111 * Its value is <code>com.google.gwt.junit.reportPath</code>. 112 * 113 * If this system property is not set, the path defaults to the user's 114 * current working directory. 115 */ 116 public static final String REPORT_PATH = "com.google.gwt.junit.reportPath"; 117 } 118