Add a proper SmokeTest list and update Android expectations.
Since the Android port only runs the --smoke option now, we can stop
using the separate android-specific TestExpectations file and just
fold expectations for the smoke test failures on Android into the main file.
(This way all the tooling should work properly).
This adds about ~1800 tests, which takes about 15 minutes of device time on
Android. There are ~130 failures and a large number of explicit skips that
represent crashes or timeouts. This gives us a baseline for tests that are
failing across all of the parts of Blink.
The tests were selected by running the layout tests on Linux, and selecting the
fastest tests from each directory tree three-directories deep or higher (e.g.,
http/tests/security and all its subdirectories), such that each tree got
roughly an equal slice of time. I will be posting the script and data used
to generate this list, but I don't expect it to be 100% reproducible. There
is no particular significance to these tests otherwise.
[email protected], [email protected], [email protected]
BUG=289883
Review URL: https://codereview.chromium.org/23600037
git-svn-id: svn://svn.chromium.org/blink/trunk@157915 bbb929c8-8fbe-4397-9dbb-9b2b20218538
diff --git a/third_party/WebKit/LayoutTests/TestExpectations b/third_party/WebKit/LayoutTests/TestExpectations
index 82d569766..b159e6a0 100644
--- a/third_party/WebKit/LayoutTests/TestExpectations
+++ b/third_party/WebKit/LayoutTests/TestExpectations
@@ -1305,3 +1305,43 @@
crbug.com/292886 [ Retina ] tables/mozilla/bugs/bug4527.html [ ImageOnlyFailure NeedsManualRebaseline ]
crbug.com/292886 [ Retina ] tables/mozilla_expected_failures/core/captions1.html [ ImageOnlyFailure NeedsManualRebaseline ]
crbug.com/292886 [ Retina ] tables/mozilla_expected_failures/core/captions2.html [ ImageOnlyFailure NeedsManualRebaseline ]
+
+# Android diffs that need rebaselines, maybe? (Needs triaging)
+crbug.com/289883 [ Android ] css1/classification/list_style_image.html [ ImageOnlyFailure ]
+crbug.com/289883 [ Android ] css1/classification/list_style_position.html [ ImageOnlyFailure ]
+crbug.com/289883 [ Android ] css1/font_properties/font_style.html [ ImageOnlyFailure ]
+crbug.com/289883 [ Android ] css1/font_properties/font_variant.html [ ImageOnlyFailure ]
+crbug.com/289883 [ Android ] css1/formatting_model/canvas.html [ ImageOnlyFailure ]
+crbug.com/289883 [ Android ] css1/formatting_model/horizontal_formatting.html [ Failure ]
+crbug.com/289883 [ Android ] css1/text_properties/text_align.html [ ImageOnlyFailure ]
+crbug.com/289883 [ Android ] fast/inline-block/overflow-clip.html [ ImageOnlyFailure ]
+crbug.com/289883 [ Android ] fast/inspector-support/matchedrules.html [ ImageOnlyFailure ]
+
+# Flaky timeouts on android
+crbug.com/289883 [ Android ] css1/cascade/important.html [ ImageOnlyFailure Timeout Pass ]
+crbug.com/289883 [ Android ] css1/classification/list_style.html [ ImageOnlyFailure Timeout ]
+crbug.com/289883 [ Android ] css1/color_and_background/background_color.html [ ImageOnlyFailure Timeout ]
+crbug.com/289883 [ Android ] css1/color_and_background/color.html [ ImageOnlyFailure Timeout ]
+crbug.com/289833 [ Android ] fast/harness/internals-profilers-heap.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/harness/override-preferences-2.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/images/text-content-crash.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/inline-block/anonymous-block-crash.html [ ImageOnlyFailure Timeout Pass ]
+crbug.com/289833 [ Android ] fast/invalid/009.html [ ImageOnlyFailure Timeout ]
+crbug.com/289833 [ Android ] fast/innerHTML/innerHTML-case.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/innerHTML/innerHTML-nbsp.xhtml [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/js/kde/const.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/harness/check-layout-error-no-attributes.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/harness/fastmallocstatistics-object.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/hidpi/clip-text-in-hidpi.html [ ImageOnlyFailure Timeout ]
+crbug.com/289833 [ Android ] fast/history/forward-during-load.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/history/history-back-within-subframe-url.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/history/history-back-forward-within-subframe-hash.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/history/window-open.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/html/imports/import-same-url.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/images/png-suite/test.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/images/style-access-during-imageChanged-crash.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/inline/inline-body-with-scrollbar-crash.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/innerHTML/005.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/invalid/invalidSVGFont.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/invalid/nestedh3s-rapidweaver.html [ Timeout Pass ]
+crbug.com/289833 [ Android ] fast/js/add-to-primitive.html [ Timeout Pass ]
