I *think* all the relevant docs have been migrated now, and those that were not migrated have been moved back to the wiki. In general, the how-to-use Litmus (including Admin) information is available on QMO, and the how-to-install Litmus information is back in the wiki.

All the information is now properly linked from the Litmus wiki, but here are some quick links for the impatient:

• Litmus Tutorial
• Litmus Admin Tutorial
• Litmus Install FAQ

I’m not actively maintaining Litmus any more myself, but I encourage interested parties to participate in the QMO discussion about Improving Litmus.

We encourage people who are already knowledgeable about certain failure types to add to the page as needed. I’ve seeded the page with some basic unit test error modes, and Alice has already added a whole whack of Talos failure info.

I wasn’t really planning on giving a session, but I was pleased to see it well attended. It was nice to meet Wayne (whose recently picked up the Thunderbird QA torch from Gary), Henrik who has been very active in Bugzilla as of late, and Emily, who has been doing fantastic work testing for Mozilla while working for Sun China over the last few years.

It’s also probably a good time to mention that I’m going to be phasing out my involvement with Litmus over the next few months. I’ve officially been a member of the release team for the better part of a year now, and Litmus support and development has suffered as a result, especially since Zach has also been busy with school. I’m pleased to announce that the Mozilla Webdev team will be taking over Litmus development in my stead.

This is bittersweet for me. I’ve been involved with Litmus since it’s inception — we borrowed the name from Zach’s WIP tool, but it really bears no resemblance to that original Testrunner-derivative — and it was one of the prime reasons for my joining Mozilla 3 years ago. However, nothing is worse than a stagnant tool. These days I’m full up with release duties (2.0.0.17 anyone?), unittest work, and now graph server bugs, and don’t have any time to make forward progress on bug fixes, much less features. Despite heavy personal and professional investiture in the code, I’m glad to have Litmus move under the auspices of the Webdev team where it will finally see some development love. If only we’d had a Webdev team 3 years ago!

To properly set expectations here, nothing much will change with Litmus in the near-term. The Webdev team is very busy, and likely won’t have time to pickup Litmus bugs for a few months. In the meantime, I’ll continue to address critical and blocker bugs, but the Mozilla QA team (specifically Marcia) is going to take over Litmus bug triage, effective immediately.

She’ll be using a different set of criteria than I was for triage, namely what features would actually be useful for the QA team, rather than what changes are actually possible for me to fix with very limited time. If you’ve been holding off on filing a Litmus bug due to lack of forward progress, it might be a good time to revisit those requests so that the Webdev team can get a better picture of what people need once they are ready to take on that work.

In the past, testing with Selenium has entailed firing up Firefox 2 because the Selenium IDE had not yet been updated to work with the Firefox 3 betas/release candidates. I was pleasantly surprised yesterday to find a new version 1.0 beta 2 available for download that has been updated to support Firefox 3. Not a moment too soon either!

Aside from minimizing my need to switch between major revs of my favorite browser, the 1.0 betas of the IDE have introduced a wealth of new functionality that make my webdev testing life easier. Of key importance to me are two features:

1. Dynamic/automatic addition of test commands to the right-click context menu as I use them. I’m likely to use the same commands again and again based on the way I write tests, and this may not be the same as other people, so not having a static list is great.
2. Ability to edit/run test suites directly. No more firing up selenium-core or switching between the IDE and a browser tab. This is a HUGE win for me.

Thanks to the Selenium team for your hard work on a great tool. I’m very interested to see how Bromine shapes up.

All the clobbering self-help instructions can be found in the wiki.

There are 4 new unit test buildbot slaves in the mix:

• qm-leak-centos5-01
• qm-leak-macosx-01
• qm-leak-winxp01
• qm-leak-w2k3-01

These machines are a little different: they’re producing debug builds and are running the standard suite of unit tests (reftest, mochitest, mochichrome, and browser chrome), but they are doing so with trace-malloc, bloat, and leak logging enabled as well.

These unit test build machines are currently reporting to the MozillaTest tinderbox tree. Jeff Walden has had a preliminary look at the output, but the tests still remain chronically red/orange. What we need now are more eyeballs to compare the failures we’re seeing on these boxes with the output of the same tests on the unit test machines reporting to the Firefox tree to tease out the legitimate failures and get bugs filed.

If you are interested in helping, you can find more information about debugging memory leaks at the Mozilla Developer Center.

Tomcat says that some of the European tech press picked up our testday announcement, and it certainly showed. Lots of new faces in #testday, and over 1300 test results filed by the community over the course of the day (another 270 if you add results from Mozilla QA staff). This was even more impressive considering that this was the second test day we held last week. In the past we’ve seen an attendance drop-off if we hold events too close together.

People seem genuinely stoked about Firefox 3. Lots of testing remains to be done, so I hope to see a lot of those folks back to help some more. Keep it locked to QMO for details.

This is a great open source conference put on by the people at Seneca college in Toronto. The conference is now in its second year, and is priced right to actually allow many smaller open source ventures or individuals to attend. I even got the opportunity to talk to a few high school students who were able to attend because the price point didn’t overwhelm their nascent interest. But I digress…

One of the major themes this year was building community and the various problems inherent therein. Even Microsoft sent a speaker, Bryan Kirschner, to discuss how Microsoft is trying to build a community around its own open source efforts. Regardless of whether you trust Microsoft’s initiatives or not, I respect Bryan a lot for coming into what has traditionally been a very hostile environment and trying to rally support without being defensive.

A lot of projects — e.g. Microsoft, OpenKomodo, Miro — are just starting (relatively speaking) to build their communities. They’re dealing with issues that are a little different than ours: how do I attract people to my project vs. project X? Some of these projects have the added benefit of being newer and/or flashier than us (Miro). Some of the projects will be effectively capped by their target audience (OpenKomodo). Microsoft just has to get past 25 years of bad blood.

Mozilla shares the recruitment problem of these other projects, but increasingly our problem is also one of retention: how do we keep the testers we already have (many of the newer projects *are* seductive), and how do we embrace new people who do show up to help and not scare them away?

The best session I saw concerning this was entitled Community Management as Open Source’s Core Competency, as given by David Eaves. I don’t have any formal background in negotiation or debate and yet I am increasing required to interact with a large community of users and testers. I found David’s talk the most eye-opening in terms of how to improve our interactions with the community and to encourage them to come back and help us more.

We (MozQA) are getting a bit better at this. IMO, the biggest step forward we’ve made in this area recently was hiring Tomcat. As someone who came directly from the community, Tomcat is filling those communication gaps that used to exist (e.g. sending notice well in advance of test days and bug days), and he is actively engaging people during QA events.

There are still some pain points though. Many people I talked to cite Bugzilla as a major impediment for joining the project. Bugzilla is often referred to as “opaque,” and it’s apparently still quite common for new people who file new bugs to be dismissed, sometimes harshly. If we can somehow find a way to *not* make Bugzilla the entry gate into the Mozilla community, that in itself would be a huge win.

We already have a few potential solutions in place. Hendrix puts a friendlier face on Bugzilla, and QMO is the new quality-focused community portal. At the end of the day though, this is not a tools issue. There has to be person and a connection happening at the other end of the tool.

And that’s really the crux of our current problem: how can we grow our community with only a small number of community “managers” on our end and still have it feel like a real community to everyone participating? Right now we’re faking it pretty well, especially considering that very few of us have formal training in this, but I fear that if it ever smells fake to our community members, they will pick up their toys and go home. As I mentioned before, they have a lot of other open source projects they could be contributing their time and effort too. If we want them to stick around, we have to engage them and make them feel like a real part of the Mozilla process.

Let’s figure out how to do that better.

What will I be doing? Well, first and foremost, the Mozilla QA team has a testday scheduled for the Friday, Oct. 26th with a focus on OS Integration on Vista & Mac, with a secondary focus on Installer testing. Between sessions, I’ll be plugging into testday, and hopefully leading an impromptu remote testday in conjunction with Club Moz @ Seneca. It’s no secret that Seneca has also been a fruitful recruiting ground for us in the past, so I’ll also be chatting with students and generally trying to get people excited about plugging in to the Mozilla community.

Hope to see lots of folks there.

Tinderbox is a bit of a beast, have no doubt. Much of the base code remains unchanged since I first interacted with it in 1998, but we’ve made some key improvements “recently.” Things like automatic config updating from CVS and CLOBBER support have reduced our Tinderbox maintenance substantially, and need to find their way into our Buildbot setup ASAP (bugs are on file for as much). Buildbot allows us to do some things in a much smarter way than Tinderbox, but our current Buildbot setup seems like a bit of a step backwards for me maintenance-wise.

I’m happy to bring my experiences with Tinderbox to bear on the problem. Hopefully we can apply those hard-learned lessons and quickly get to a better maintenance state.