You are currently browsing the archives for the Software category.
Posted 2 months, 2 weeks ago at 11:42. 0 comments
Releng systems are complex, with many moving parts. Making changes to these systems used to be fraught with peril. Even for people who had been working with buildbot for many years, it was hard to write a patch and be certain that wouldn’t break some downstream system.
Posted 1 year, 1 month ago at 15:16. 4 comments
Armen has a blog post up about the cost savings Mozilla has been able to realize in its continuous integration infrastructure in Amazon over just the last 3 months. This has been a bit of a sea change for release engineering, who have historically been conservative with regards to changing core infrastructure and practices. We’re all coming to grips with the new world order, but I’m quite excited about the possibilities.
Some quick back-of-the-envelope calculations based on other recent numbers from Armen:
- starting with a low-ball estimate of 7,000 pushes/month, if we project the rate of spending from December ($19/push) over an entire year, we end up with $1,596,000.
- at the new rate ($6/push), a year of AWS time will cost only $504,000.
- that’s a yearly savings of $1,092,000.
If history has taught us anything, continued growth will eat in to at least part of that savings, but think of what Mozilla could do with an extra million dollars. Depending on where we hire them, that money could easily buy 5-10 more engineers to continue driving the mission forward.
Posted 1 year, 5 months ago at 19:47. 1 comment
Mozilla is graciously giving its staff a two-week break over the holidays. What does this mean for service groups like release engineering?
We too will be away spending time with our families, but we will also continue to monitor the general health of the continuous integration infrastructure. If something happens that knocks an entire platform or datacenter offline, we will stand things back up, but we won’t be worrying about regular day-to-day issues like rebooting individual slaves or fulfilling loan requests.
Barring a chemspill, we also won’t be producing releases over the two-week span, with the exception of Nightly and Aurora daily releases which should continue to happily hum along.
Mozilla staff know how to get in connect with release engineering should it become necessary. For other Mozillians, your best bet is to contact us in the #releng channel on IRC. Ping coop (that’s me), catlee, or hwine if necessary, but please be patient. There may be eggnog involved.
Posted 1 year, 7 months ago at 11:46. 8 comments
Greg Szorc has been tireless in pushing for improvements in the build system. This past summer, he added automatic psutil system usage reporting for all Mozilla automation jobs run by mozharness. Since release engineering is actively moving all jobs to mozharness, we should soon have efficiency metrics for all jobs.
Unfortunately, Greg tried to parlay that work into a larger analysis of overall infrastructure machine efficiency. His analysis is not wrong, per se, but his post presents the state of machine efficiency with the assumption that it is that way by accident. Greg did post an addendum at the bottom of the entry, but I don’t think that edit ever found the same traction that the original piece did. I’d like to try to address why our machine efficiency numbers look the way they do.
Posted 1 year, 9 months ago at 09:56. 0 comments
- timechange to 11am PDT, 2:30pm PDT?
- do 11am PDT in two weeks time; check with EST folks during next BuildConfig meeting, maybe move after that.
- action items from last mtg
- (coop) talk to RyanVM re: MozillaBuild – NOT DONE
- (coop) move Windows release repacks to Win64 – IN PROGRESS
- (bsmedberg) talk to gps re: resource monitoring
- (mshal, joey) return with estimates
- Round table
- q3 goals status – 1month gone; 2 months to go
- making progress, but no measure of how far to completion; separate meeting to resolve
- scope of “export tier” goal? Estimate to completion?
- revised “draft” goals list never published – l10n and RelEng
- details + nextsteps around RelEng l10n blowout this week in FF24.0b1 (l10n/pymake/mock/vc8/vc10?)
- questions, estimates, adjustments and fodder for Q3..
- Conversion estimates: http://releng.etherpad.mozilla.org/conversion-estimate
- Elapsed time based on ultra-simple passthrough conversions with try coverage on all platforms has been an actual 2 month avg.
- 77 vars remaining to convert, down 3 from last month (from 07/17)
- Current trend will be 154 months
- unthreaded, single user
- simple passthrough conversions, ignoring overhead for code reviews, etc
- how should stats be adjusted for logic based conversions ?
- Any feedback on Fennec patch?
- mshal is now a build peer \o/
- action items:
- meeting to decide scope of Q3 goal
- revisit fennec-nop-builds after touching base with kats, blassey, margaret, mfinkle – what else after the
- new tracking bug for Q3 issues to fix? Or keep on with 748452?
- NOTE: next meeting at 11am PDT / 2pm EST. gps to send out invite
Current Tunes: Arty - Together We Are: 052 [Live Set from Glow Club, Echo Stage - Washington] | Filed under Build/Release, Firefox, Meeting Notes, Mozilla, Open Web, Software
Posted 2 years, 8 months ago at 11:39. 3 comments
In my previous post, I outlined the four main parts in our plan of attack for making the build system faster. But how will we know how well we’re doing, and how will we know when we’re done? How fast is fast enough?
Current Tunes: The Offspring - It'll Be A Long Time | Filed under Build/Release, Firefox, Mozilla, Software
Posted 3 years, 11 months ago at 17:53. 0 comments
It seems I’ve been remiss in welcoming a new hire to my subgroup of Mozilla Release Engineering.
Joey Armstrong is a Makefile whiz, and he’s going to be taking a stab at improving turnaround time on incremental builds and generally simplifying our Makefile story. John rightly gave up waiting for me to blog about Joey starting, and has written a post of his own explaining in more depth why Joey’s work is important. I’ll just add that we’ve done a lot of work recently in RelEng trying to identify long poles in our current build/test process. This effort has largely focused on re-assorting jobs and using hardware in smarter ways. It is awesome to have someone on the team now who is specifically tasked with trying to shorten the build process itself.
Posted 4 years, 10 months ago at 10:38. 0 comments
A small caveat (and workaround) if you’re using doStepIf with buildbot:
The Mozilla RelEng team recently upgraded to buildbot 0.8.0. This has allowed me to start using conditional buildbot steps by specifying the doStepIf parameter and a small helper function to check whether given properties have been set. I love it, and it opens up a whole new ways of working with our builds.
Unfortunately, the doStepIf implementation is new-ish and still broken in some ways. If steps are skipped, the finished() code is called without start() ever having been run, leaving you with potentially uninitialized variables. In my case, this manifests if I don’t specify a description for my conditional build step. Buildbot attempts to build a description for me by walking the properties, some of which may not be set if doStepIf is false.
The workaround is, of course, to always provide a description for any conditional build steps (easy), or wrap everything in a try block.
Current Tunes: Above and Beyond - Trance Around The World 329 - Boom Jinx - 2010-07-16 | Filed under Build/Release, Mozilla, Software
Posted 5 years ago at 15:34. 3 comments
I realized the other day that I hadn’t done a post about my favorite add-ons since Firefox 1.5. The add-on landscape has changed a lot since that time, sufficiently so that I think it merits an updated post with a short blurb about why I love/need each extension. I’m using some add-ons for Thunderbird too, so I’ve included those as well.
Posted 5 years, 9 months ago at 10:33. 2 comments
High on the lists of things you probably don’t want to try: moving your iTunes music library to an external hard drive while your iPod is connected. The library move (using Consolidate Library) works just fine, UNTIL the next time you try to sync your iPod.