You are currently browsing the archives for the Build/Release category.
Posted 1 month, 2 weeks ago at 10:38. 0 comments
A small caveat (and workaround) if you’re using doStepIf with buildbot:
The Mozilla RelEng team recently upgraded to buildbot 0.8.0. This has allowed me to start using conditional buildbot steps by specifying the doStepIf parameter and a small helper function to check whether given properties have been set. I love it, and it opens up a whole new ways of working with our builds.
Unfortunately, the doStepIf implementation is new-ish and still broken in some ways. If steps are skipped, the finished() code is called without start() ever having been run, leaving you with potentially uninitialized variables. In my case, this manifests if I don’t specify a description for my conditional build step. Buildbot attempts to build a description for me by walking the properties, some of which may not be set if doStepIf is false.
The workaround is, of course, to always provide a description for any conditional build steps (easy), or wrap everything in a try block.
Current Tunes: Above and Beyond - Trance Around The World 329 - Boom Jinx - 2010-07-16 | Filed under Build/Release, Mozilla, Software
Posted 1 month, 2 weeks ago at 12:26. 0 comments
I posted this followup to the newsgroups a few weeks back, but things got kinda hectic with the end of quarter and the Summit so I never circled back here.
The cleanup crontabs for all products are in place on stage.mozilla.org now. We did see a little bit of bustage with the initial set of MAR expiry commands being improperly limited, but luckily KaiRo didn’t lose anything irreplaceable.
We didn’t reclaim as much space as originally estimated because we preserved the contrib dirs largely intact. We did still manage to recover almost 350G of space across all products, bringing our %used number on stage down to 79%. It was at 95% the day before we put these crontabs in place.
I’ve documented the new policy in the wiki: ReleaseEngineering:StageCleanupPolicy
We’ll still have to grow the disk eventually, or shuffle stuff around, but at least we have *something* in place policy-wise now.
Thanks to everyone who weighed-in on this longstanding issue.
Posted 3 months, 2 weeks ago at 15:15. 0 comments
Another positive outcome from the recent work to generate nightly updates on the buildslaves is that we now have update generation tools that work on Linux, Mac, and Windows. There been some interest in the past from other software projects about using our update tools, but I know that getting all of the mozilla source setup to build just these tools is non-trivial, especially if your software isn’t Mozilla-based to begin with.
I’ve packaged up the update tools for each platform from a recent mozilla-central nightly and put them under the xulrunner directory on ftp:
I’ve also updated our wiki documentation about how we make our nightly updates for Firefox to give people a fighting chance of figuring out how to use these tools for their own purposes.
Enjoy!
Current Tunes: Radiohead - Everything In It's Wrong Place (remix) | Filed under Build/Release, Firefox, Mozilla
Posted 3 months, 3 weeks ago at 22:30. 4 comments
For those who want to skip to my specific proposals — there are 6 — for reclaiming space on stage.mozilla.org, please skip ahead to “Redux”, but if you’re going to comment, please read the whole thing.
Everyday we produce up to 17G worth of new nightly builds for Firefox across all branches. This includes all opt+debug builds in 75+ locales, each on 4 operating systems (OSes), each on 9 different project branches. We do reclaim much of this 17G as we retire l10n builds older than 1 week, but we are still creating 1.3G of new nightly en-US builds that we need to store (essentially) indefinitely. nthomas did some cleanup recently as part of bug 562261 and that has bought us some more time, but this inexorable increase will eventually overrun our disk capacity on the staging server. If we add to this disk usage by nightlies from other products which may have worse nightly hygiene habits and the expected increase in space requirements every night as we add 4 new OSes (adding Linux 64bit, Windows 64bit, OSX10.6 64bit and Android), the problem is magnified.
Continue Reading…
Current Tunes: Violent Femmes - American Music | Filed under Build/Release, Firefox, Mozilla
Posted 5 months ago at 02:46. 4 comments
I landed a patch last week that represents the culmination of several months worth of effort by me, but that also brings to fruition the full promise of l10n nightly updates, work that was begun by Armen almost a year ago.
Continue Reading…
Posted 6 months, 3 weeks ago at 19:49. 1 comment
Let it never be said that we don’t listen.
In response to the comments on my previous entry about Tackling the Release Engineering:Future queue and a number of conversations I’ve had out-of-band over the past few months, the release engineering team has decided to do away with the Future component. We will be merging all of the bugs therein back into the regular Release Engineering component.
Aside from the simplicity inherent in having fewer components to worry about, there were three main reasons we chose to do this: Continue Reading…
Current Tunes: New Order - Blue Monday | Filed under Build/Release, Mozilla
Posted 6 months, 3 weeks ago at 17:22. 4 comments
The Release Engineering:Future bugzilla component alternately inspires feelings of sadness, loathing, and contempt…and that’s just within the RelEng team!
I’m certain most developers first response to having their bug moved to the Future queue is, “Oh, look, my bug has fallen down a well.” Historically speaking, that may not be far from the truth.
Continue Reading…
Current Tunes: Above and Beyond - Trance Around The World 306 - Andy Moor - 2010-02-05 | Filed under Build/Release, Mozilla
Posted 7 months ago at 15:37. 0 comments
This is a follow-up to Ben’s blog post about the RelEng Sheriff back in October. His post described clearly what the RelEng Sheriff (and more generally, the RelEng team) can do to help developers.
Since we implemented the RelEng sheriff (or “buildduty” as it is more informally called) last spring, developers have been getting better about using the buildduty person as the first point of contact for RelEng issues. I’d like to implore people to continue doing so. There are a few exceptions, of course:
Continue Reading…
Posted 10 months, 2 weeks ago at 11:03. 9 comments
Recognizing some existing internal group dynamics and the need for the entire Mozilla release engineering (RelEng) team to grow in a more sustainable manner, we recently started an internal reorganization of the RelEng group. John will likely blog about the new structure of the group at some point, but I’ll relate the bits that are pertinent to me.
Continue Reading…
Current Tunes: Digitalism - Echoes | Filed under Build/Release, Mozilla
Posted 10 months, 2 weeks ago at 11:05. 1 comment
The Mozilla release engineering (RelEng) team has grown substantially over the past two years. Some members of the team have certain domain-specific focuses (e.g. talos, mobile), but one of our primary team goals has been to get our release automation to the point where anyone on our team can handle a release. Given the emerging branch picture for Mozilla code where we might see simultaneous releases on 3+ code branches (a.k.a release-apalooza), this is becoming increasingly important.
I’ve done my share of releases over the past few quarters, and until the current release (3.5.4, still in progress as of writing), I would have said that I understood the process pretty well.
Continue Reading…
