You are currently browsing the archives for the Software category.

Better releng patch contribution workflow

Posted 2 months, 2 weeks ago at 11:42. 0 comments

Releng systems are complex, with many moving parts. Making changes to these systems used to be fraught with peril. Even for people who had been working with buildbot for many years, it was hard to write a patch and be certain that wouldn’t break some downstream system.
Continue Reading…

Current Tunes: Nothing | Filed under Build/Release, Firefox, Mozilla, Software

If I had a million dollars

Posted 1 year, 1 month ago at 15:16. 4 comments

Kraft DinnerArmen has a blog post up about the cost savings Mozilla has been able to realize in its continuous integration infrastructure in Amazon over just the last 3 months. This has been a bit of a sea change for release engineering, who have historically been conservative with regards to changing core infrastructure and practices. We’re all coming to grips with the new world order, but I’m quite excited about the possibilities.

Some quick back-of-the-envelope calculations based on other recent numbers from Armen:

  • starting with a low-ball estimate of 7,000 pushes/month, if we project the rate of spending from December ($19/push) over an entire year, we end up with $1,596,000.
  • at the new rate ($6/push), a year of AWS time will cost only $504,000.
  • that’s a yearly savings of $1,092,000.

If history has taught us anything, continued growth will eat in to at least part of that savings, but think of what Mozilla could do with an extra million dollars. Depending on where we hire them, that money could easily buy 5-10 more engineers to continue driving the mission forward.

Current Tunes: Nothing | Filed under Build/Release, Firefox, Mozilla, Open Web, Software

Release engineering holiday coverage

Posted 1 year, 5 months ago at 19:47. 1 comment

Mozilla is graciously giving its staff a two-week break over the holidays. What does this mean for service groups like release engineering?

We too will be away spending time with our families, but we will also continue to monitor the general health of the continuous integration infrastructure. If something happens that knocks an entire platform or datacenter offline, we will stand things back up, but we won’t be worrying about regular day-to-day issues like rebooting individual slaves or fulfilling loan requests.

Barring a chemspill, we also won’t be producing releases over the two-week span, with the exception of Nightly and Aurora daily releases which should continue to happily hum along.

Mozilla staff know how to get in connect with release engineering should it become necessary. For other Mozillians, your best bet is to contact us in the #releng channel on IRC. Ping coop (that’s me), catlee, or hwine if necessary, but please be patient. There may be eggnog involved.


Current Tunes: Jaytech Music Podcast 071 | Filed under Build/Release, Firefox, Mozilla, Software

Wait times: why measuring machine efficiency is hard, and why we still need to try

Posted 1 year, 7 months ago at 11:46. 8 comments

Dilbert.comGreg Szorc has been tireless in pushing for improvements in the build system. This past summer, he added automatic psutil system usage reporting for all Mozilla automation jobs run by mozharness. Since release engineering is actively moving all jobs to mozharness, we should soon have efficiency metrics for all jobs.

Unfortunately, Greg tried to parlay that work into a larger analysis of overall infrastructure machine efficiency. His analysis is not wrong, per se, but his post presents the state of machine efficiency with the assumption that it is that way by accident. Greg did post an addendum at the bottom of the entry, but I don’t think that edit ever found the same traction that the original piece did. I’d like to try to address why our machine efficiency numbers look the way they do.
Continue Reading…

Current Tunes: Audien - Wayfarer | Filed under Build/Release, Firefox, Hardware, Mozilla, Software

Build::Config meeting notes for 2013-08-08

Posted 1 year, 9 months ago at 09:56. 0 comments

2013-08-08 Agenda:

  • logistics:
    • timechange to 11am PDT, 2:30pm PDT?
    • do 11am PDT in two weeks time; check with EST folks during next BuildConfig meeting, maybe move after that.
  • action items from last mtg
    • (coop) talk to RyanVM re: MozillaBuild – NOT DONE
    • (coop) move Windows release repacks to Win64 – IN PROGRESS
    • (bsmedberg) talk to gps re: resource monitoring
    • (mshal, joey) return with estimates
  • Round table
  • action items:
    • meeting to decide scope of Q3 goal
    • revisit fennec-nop-builds after touching base with kats, blassey, margaret, mfinkle – what else after the 
      • new tracking bug for Q3 issues to fix? Or keep on with 748452?
    • NOTE: next meeting at 11am PDT / 2pm EST. gps to send out invite

Current Tunes: Arty - Together We Are: 052 [Live Set from Glow Club, Echo Stage - Washington] | Filed under Build/Release, Firefox, Meeting Notes, Mozilla, Open Web, Software

Reviving BuildFaster: Measuring progress

Posted 2 years, 8 months ago at 11:39. 3 comments

Pull thread to unravelIn my previous post, I outlined the four main parts in our plan of attack for making the build system faster. But how will we know how well we’re doing, and how will we know when we’re done? How fast is fast enough?
Continue Reading…

Current Tunes: The Offspring - It'll Be A Long Time | Filed under Build/Release, Firefox, Mozilla, Software

Welcome, Joey

Posted 3 years, 11 months ago at 17:53. 0 comments

It seems I’ve been remiss in welcoming a new hire to my subgroup of Mozilla Release Engineering.

Joey Armstrong is a Makefile whiz, and he’s going to be taking a stab at improving turnaround time on incremental builds and generally simplifying our Makefile story. John rightly gave up waiting for me to blog about Joey starting, and has written a post of his own explaining in more depth why Joey’s work is important. I’ll just add that we’ve done a lot of work recently in RelEng trying to identify long poles in our current build/test process. This effort has largely focused on re-assorting jobs and using hardware in smarter ways. It is awesome to have someone on the team now who is specifically tasked with trying to shorten the build process itself.

So…welcome, Joey!

Current Tunes: The Prodigy - Colours | Filed under Build/Release, Mozilla, Software

Setting descriptions to avoid buildbot exceptions from doStepIf

Posted 4 years, 10 months ago at 10:38. 0 comments

NutA small caveat (and workaround) if you’re using doStepIf with buildbot:

The Mozilla RelEng team recently upgraded to buildbot 0.8.0. This has allowed me to start using conditional buildbot steps by specifying the doStepIf parameter and a small helper function to check whether given properties have been set. I love it, and it opens up a whole new ways of working with our builds.

Unfortunately, the doStepIf implementation is new-ish and still broken in some ways. If steps are skipped, the finished() code is called without start() ever having been run, leaving you with potentially uninitialized variables. In my case, this manifests if I don’t specify a description for my conditional build step. Buildbot attempts to build a description for me by walking the properties, some of which may not be set if doStepIf is false.

The workaround is, of course, to always provide a description for any conditional build steps (easy), or wrap everything in a try block.

Current Tunes: Above and Beyond - Trance Around The World 329 - Boom Jinx - 2010-07-16 | Filed under Build/Release, Mozilla, Software

My add-ons

Posted 5 years ago at 15:34. 3 comments

puzzle pieceI realized the other day that I hadn’t done a post about my favorite add-ons since Firefox 1.5. The add-on landscape has changed a lot since that time, sufficiently so that I think it merits an updated post with a short blurb about why I love/need each extension. I’m using some add-ons for Thunderbird too, so I’ve included those as well.
Continue Reading…

Current Tunes: Nothing | Filed under Firefox, Software, Thunderbird

Don’t try this at home

Posted 5 years, 9 months ago at 10:33. 2 comments

Sad Mac :(High on the lists of things you probably don’t want to try: moving your iTunes music library to an external hard drive while your iPod is connected. The library move (using Consolidate Library) works just fine, UNTIL the next time you try to sync your iPod.
Continue Reading…

Current Tunes: DowJonez - Stay In Your Place feat. Drake & Gemstonez | Filed under Music, Software