andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 1 | # Running layout tests on Linux |
| 2 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 3 | 1. Build `blink_tests` (see LinuxBuildInstructions) |
| 4 | 1. Checkout the layout tests |
| 5 | * If you have an entry in your `.gclient` file that includes |
| 6 | "LayoutTests", you may need to comment it out and sync. |
| 7 | * You can run a subset of the tests by passing in a path relative to |
| 8 | `src/third_party/WebKit/LayoutTests/`. For example, |
| 9 | `run_layout_tests.py fast` will only run the tests under |
| 10 | `src/third_party/WebKit/LayoutTests/fast/`. |
| 11 | 1. When the tests finish, any unexpected results should be displayed. |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 12 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 13 | See |
| 14 | [Running WebKit Layout Tests](http://dev.chromium.org/developers/testing/webkit-layout-tests) |
| 15 | for full documentation about set up and available options. |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 16 | |
| 17 | ## Pixel Tests |
| 18 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 19 | The pixel test results were generated on Ubuntu 10.4 (Lucid). If you're running |
| 20 | a newer version of Ubuntu, you will get some pixel test failures due to changes |
| 21 | in freetype or fonts. In this case, you can create a Lucid 64 chroot using |
| 22 | `build/install-chroot.sh` to compile and run tests. |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 23 | |
| 24 | ## Fonts |
| 25 | |
| 26 | Make sure you have all the necessary fonts installed. |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 27 | |
| 28 | ```shell |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 29 | sudo apt-get install apache2 wdiff php5-cgi ttf-indic-fonts \ |
| 30 | msttcorefonts ttf-dejavu-core ttf-kochi-gothic ttf-kochi-mincho \ |
| 31 | ttf-thai-tlwg |
| 32 | ``` |
| 33 | |
| 34 | You can also just run `build/install-build-deps.sh` again. |
| 35 | |
| 36 | ## Plugins |
| 37 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 38 | If `fast/dom/object-plugin-hides-properties.html` and |
| 39 | `plugins/embed-attributes-style.html` are failing, try uninstalling |
| 40 | `totem-mozilla` from your system: |
| 41 | |
| 42 | sudo apt-get remove totem-mozilla |
| 43 | |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 44 | |
| 45 | ## Running layout tests under valgrind on Linux |
| 46 | |
| 47 | As above, but use `tools/valgrind/chrome_tests.sh -t webkit` instead. e.g. |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 48 | |
| 49 | sh tools/valgrind/chrome_tests.sh -t webkit LayoutTests/fast/ |
| 50 | |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 51 | This defaults to using --debug. Read the script for more details. |
| 52 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 53 | If you're trying to reproduce a run from the valgrind buildbot, look for the |
| 54 | `--run_chunk=XX:YY` line in the bot's log. You can rerun exactly as the bot did |
| 55 | with the commands. |
| 56 | |
| 57 | ```shell |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 58 | cd ~/chromium/src |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 59 | echo XX > valgrind_layout_chunk.txt |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 60 | sh tools/valgrind/chrome_tests.sh -t layout -n YY |
| 61 | ``` |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 62 | |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 63 | That will run the XXth chunk of YY layout tests. |
| 64 | |
| 65 | ## Configuration tips |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 66 | |
| 67 | * Use an optimized `content_shell` when rebaselining or running a lot of |
| 68 | tests. ([bug 8475](https://crbug.com/8475) is about how the debug output |
| 69 | differs from the optimized output.) |
| 70 | |
| 71 | `ninja -C out/Release content_shell` |
| 72 | |
| 73 | * Make sure you have wdiff installed: `sudo apt-get install wdiff` to get |
| 74 | prettier diff output. |
| 75 | * Some pixel tests may fail due to processor-specific rounding errors. Build |
| 76 | using a chroot jail with Lucid 64-bit user space to be sure that your system |
| 77 | matches the checked in baselines. You can use `build/install-chroot.sh` to |
| 78 | set up a Lucid 64 chroot. Learn more about |
| 79 | [using a linux chroot](using_a_linux_chroot.md). |
| 80 | |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 81 | ## Getting a layout test into a debugger |
| 82 | |
| 83 | There are two ways: |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 84 | |
| 85 | 1. Run `content_shell` directly rather than using `run_layout_tests.py`. You |
| 86 | will need to pass some options: |
| 87 | * `--no-timeout` to give you plenty of time to debug |
| 88 | * the fully qualified path of the layout test (rather than relative to |
| 89 | `WebKit/LayoutTests`). |
| 90 | 1. Or, run as normal but with the |
| 91 | `--additional-drt-flag=--renderer-startup-dialog |
| 92 | --additional-drt-flag=--no-timeout --time-out-ms=86400000` flags. The first |
| 93 | one makes content\_shell bring up a dialog before running, which then would |
| 94 | let you attach to the process via `gdb -p PID_OF_DUMPRENDERTREE`. The others |
| 95 | help avoid the test shell and DumpRenderTree timeouts during the debug |
| 96 | session. |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 97 | |
| 98 | ## Using an embedded X server |
| 99 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 100 | If you try to use your computer while the tests are running, you may get annoyed |
| 101 | as windows are opened and closed automatically. To get around this, you can |
| 102 | create a separate X server for running the tests. |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 103 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 104 | 1. Install Xephyr (`sudo apt-get install xserver-xephyr`) |
| 105 | 1. Start Xephyr as display 4: `Xephyr :4 -screen 1024x768x24` |
| 106 | 1. Run the layout tests in the Xephyr: `DISPLAY=:4 run_layout_tests.py` |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 107 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 108 | Xephyr supports debugging repainting. See the |
| 109 | [Xephyr README](http://cgit.freedesktop.org/xorg/xserver/tree/hw/kdrive/ephyr/README) |
| 110 | for details. In brief: |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 111 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 112 | 1. `XEPHYR_PAUSE=$((500*1000)) Xephyr ...etc... # 500 ms repaint flash` |
| 113 | 1. `kill -USR1 $(pidof Xephyr)` |
| 114 | |
| 115 | If you don't want to see anything at all, you can use Xvfb (should already be |
| 116 | installed). |
| 117 | |
| 118 | 1. Start Xvfb as display 4: `Xvfb :4 -screen 0 1024x768x24` |
| 119 | 1. Run the layout tests in the Xvfb: `DISPLAY=:4 run_layout_tests.py` |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 120 | |
| 121 | ## Tiling Window managers |
| 122 | |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 123 | The layout tests want to run with the window at a particular size down to the |
| 124 | pixel level. This means if your window manager resizes the window it'll cause |
| 125 | test failures. This is another good reason to use an embedded X server. |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 126 | |
| 127 | ### xmonad |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 128 | |
| 129 | In your `.xmonad/xmonad.hs`, change your config to include a manageHook along |
| 130 | these lines: |
| 131 | |
andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 132 | ``` |
| 133 | test_shell_manage = className =? "Test_shell" --> doFloat |
| 134 | main = xmonad $ |
| 135 | defaultConfig |
| 136 | { manageHook = test_shell_manage <+> manageHook defaultConfig |
| 137 | ... |
andybons | ad92aa3 | 2015-08-31 02:27:44 | [diff] [blame] | 138 | ``` |