| Takuto Ikuta | 540ba9d | 2018-12-19 22:24:08 | [diff] [blame] | 1 | # Ninja build log collection |
| 2 | |
| 3 | [TOC] |
| 4 | |
| 5 | ## Overview |
| 6 | |
| 7 | When chromium developers use autoninja for their build, |
| 8 | |
| 9 | e.g. |
| 10 | ``` |
| 11 | $ autoninja -C out/Release chrome |
| 12 | ``` |
| 13 | |
| 14 | autoninja uploads ninja's build log for Google employees but we don't collect |
| 15 | logs from external contributors. |
| 16 | |
| 17 | We use [this page](https://chromium-build-stats-staging.appspot.com/should-upload) |
| 18 | to decide whether an autoninja user is a Googler or not. Only people with access |
| 19 | to Google's internal network can see 'Success'. |
| 20 | |
| 21 | Before uploading logs, autoninja shows a message 10 times to warn users that we |
| 22 | will collect build logs. |
| 23 | |
| 24 | autoninja users can also opt in or out by using the following commands: |
| 25 | |
| 26 | * `ninjalog_uploader_wrapper.py opt-in` |
| 27 | * `ninjalog_uploader_wrapper.py opt-out` |
| 28 | |
| 29 | ## What type of data are collected? |
| 30 | |
| 31 | The collected build log contains |
| 32 | |
| 33 | * output file of build tasks (e.g. chrome, obj/url/url/url_util.o) |
| 34 | * hash of build command |
| 35 | * start and end time of build tasks |
| 36 | |
| 37 | See [manual of ninja](https://ninja-build.org/manual.html#ref_log) for more |
| 38 | details. |
| 39 | |
| 40 | In addition to ninja's build log, we send the following data for further |
| 41 | analysis: |
| 42 | |
| 43 | * OS (e.g. Win, Mac or Linux) |
| 44 | * number of cpu cores of building machine |
| 45 | * build targets (e.g. chrome, browser_tests) |
| 46 | * parallelism passed by -j flag |
| 47 | * following build configs |
| 48 | * host\_os, host\_cpu |
| 49 | * target\_os, target\_cpu |
| 50 | * symbol\_level |
| 51 | * use\_goma |
| 52 | * is\_debug |
| 53 | * is\_component\_build |
| 54 | |
| 55 | We don't collect personally identifiable information |
| 56 | (e.g. username, ip address). |
| 57 | |
| 58 | ## Why ninja log is collected? / How the collected logs are used? |
| 59 | |
| 60 | We (goma team) collect build logs to find slow build tasks that harm developer's |
| 61 | productivity. Based on collected stats, we find the place/build tasks where we |
| 62 | need to focus on. Also we use collected stats to track chrome build performance |
| 63 | on developer's machine. We'll use this stats to measure how much we can/can't |
| 64 | improve build performance on developer's machine. |
