Posted 2 years, 4 months ago at 11:39. 3 comments
In my previous post, I outlined the four main parts in our plan of attack for making the build system faster. But how will we know how well we’re doing, and how will we know when we’re done? How fast is fast enough?
The first step was relatively easy, and involved fixing the data submission from the build automation to brasstacks. Now we can once again see how our average build times are changing on a day-to-day basis.
The graph looks especially nice right now due to turning on pymake on August 30th which netted us a ~25% reduction in build times on Windows (32-bit). (Here is a static image of the graph if you happen to be visiting from the future.) sid0 has a patch in progress to also use pymake for Window 64-bit builds. I look forward to enabling that soon.
The next step is going to take a little more work.
Greg (gps) would like to start tracking CPU utilization as a measure of the efficiency of the build system. This will help address the “how fast is fast enough?” question by providing an initial plateau: get CPU utilization as close to 100% as possible on the various build platforms (how close to 100% is close enough? shhh, stop asking hard questions). As Greg notes,
“…end-to-end build times aren’t going to decrease if the CPU utilization during the build doesn’t go up. The problem today is getting build actions/processes to spawn sooner/faster/on all cores. If we are at 100% CPU utilization on all cores, the problem will be getting them to finish faster. Those are two very different problems.”
One of the tools that Greg has been working on, mach, can report on CPU (and disk I/O) utilization, but it relies on having python 2.7 available on all the build platforms. I’ve picked up bug 602908 to help drive that forward. Having a more recent version of python available will also help other efforts to speed-up end-to-end times, such as nascent work to try running unittests in parallel where possible.
Some progress already made, and lots of work in flight here. Please check out the thread in mozilla.dev.builds where Greg et al. have recently decided to sandboxed Python files for the extracted Makefile data (Part 4 from my previous post).