You are currently browsing the archives for the QA category.

Recent mobile testing improvements

Posted 2 years, 9 months ago at 09:38. 0 comments

Firefox for AndroidThere’s a Mozilla mobile workweek happening in Boston this week, and while I’m sure there will be plenty of exciting news to come out of it, I want to take a moment to highlight something that happened a few weeks ago.

In case you weren’t paying attention to Google groups, wlach, jmaher, Callek and the other folks involved with Mobile testing landed a tegra reboot fix 10 days ago that reduced the intermittent failure rate for tegra tests from 39% to 13%. Subsequent test fixes making their way through the system have now brought that failure rate down to 7%.

While this improvement is impressive enough on its face, these fixes finally bring Android test results into the range where developers can (and should) pay attention to their tegra test results when landing changes. This takes some of the onus off of sheriffs to be able to diagnose and requeue these intermittent failures, freeing them up to do more proactive work.

Given the continued focus of Mozilla on mobile development, this can only be a good thing.

Current Tunes: Above and Beyond - Trance Around The World 437 - 2012-08-10 | Filed under Build/Release, Mozilla, QA

IRC response time

Posted 3 years, 1 month ago at 18:41. 3 comments

Pro-tip: if you ping me on IRC to ask me something and don’t ask your question as part of the ping, you have added an asynchronous cycle of indeterminate length to our interaction. Don’t do that.

Current Tunes: OceanLab vs. Passive Progressive - Sky Falls Away | Filed under Build/Release, Mozilla, QA

Litmus tutorial documentation help needed

Posted 4 years, 1 month ago at 12:08. 0 comments

The long-missing Litmus tutorial has been resurrected. Unfortunately, it isn’t nearly as useful as it could be since it hasn’t been updated since 2007.

If you’re a current Litmus user, you’re already more awesome than most. If you’d like to take a stab at updating the Litmus Tutorial to reflect the current state of Litmus usage, I’m more than happy to help out any way I can with proofreading, etc.

Current Tunes: The Instance Podcast #226 | Filed under Litmus, Mozilla, QA

Where to find Litmus documentation

Posted 6 years ago at 12:38. 0 comments

QMO logoWith the “recent” changes to QMO, a lot of the Litmus documentation went missing. It was still available at an archived site if you knew to look there, but there was no automatic migration or visible linkages in place.

I *think* all the relevant docs have been migrated now, and those that were not migrated have been moved back to the wiki. In general, the how-to-use Litmus (including Admin) information is available on QMO, and the how-to-install Litmus information is back in the wiki.

All the information is now properly linked from the Litmus wiki, but here are some quick links for the impatient:

I’m not actively maintaining Litmus any more myself, but I encourage interested parties to participate in the QMO discussion about Improving Litmus.

Current Tunes: Metric - Gold Guns Girls | Filed under Litmus, Mozilla, QA

Diagnosing Tinderbox Red/Orange

Posted 6 years, 7 months ago at 16:34. 0 comments

As an offshoot of bug 442720, we now have a wiki page to track the various failure modes that can cause Tinderbox builds to go orange or red. This should hopefully help sheriffs and other interested parties to figure out where builds are going off the rails.

We encourage people who are already knowledgeable about certain failure types to add to the page as needed. I’ve seeded the page with some basic unit test error modes, and Alice has already added a whole whack of Talos failure info.

Current Tunes: Oasis - Don't Look Back In Anger | Filed under Build/Release, Mozilla, QA

Litmus/QAC session notes from the Summit

Posted 6 years, 10 months ago at 11:32. 0 comments

LitmusI’ve added the notes from the Litmus/QAC session directly to the Session Proposal page.

I wasn’t really planning on giving a session, but I was pleased to see it well attended. It was nice to meet Wayne (whose recently picked up the Thunderbird QA torch from Gary), Henrik who has been very active in Bugzilla as of late, and Emily, who has been doing fantastic work testing for Mozilla while working for Sun China over the last few years.
Continue Reading…

Current Tunes: Above & Beyond - Trance Around The World 277 - Anhken - 2008-08-01 | Filed under Firefox, Litmus, Mozilla, QA

Selenium IDE updated for Firefox 3

Posted 6 years, 11 months ago at 10:53. 2 comments

I’ve spent the last day or so updating the existing Selenium test suite for Litmus in advance of landing the new cloning code for bug 362489.

In the past, testing with Selenium has entailed firing up Firefox 2 because the Selenium IDE had not yet been updated to work with the Firefox 3 betas/release candidates. I was pleasantly surprised yesterday to find a new version 1.0 beta 2 available for download that has been updated to support Firefox 3. Not a moment too soon either!
Continue Reading…

Current Tunes: The Irresistible Force - It's Tomorrow Already | Filed under Litmus, Mozilla, QA, Software

Unit test clobber support

Posted 7 years, 4 months ago at 21:30. 1 comment

ThingIn case you weren’t following the stream of curses in IRC last Thursday — alternately, robcee and myself cursing at buildbot slaves, and developers cursing at us — clobber support for the buildbot unit test machines has landed. My apologies for the disruption it caused. The payoff is that developers can now clobber unit test machines via CVS in a similar manner to the build machines. Each improvement makes these very fragile machines a little less touchy, and allows us to be more hands-off with them.

All the clobbering self-help instructions can be found in the wiki.

Current Tunes: Digitalism - Zdarlight | Filed under Build/Release, Mozilla, QA

It’s no use carrying an umbrella if your shoes are leaking

Posted 7 years, 4 months ago at 20:57. 0 comments

Drip!Ah, there’s always an Irish proverb to fit the bill (especially if the bill happens to be a bar tab).

There are 4 new unit test buildbot slaves in the mix:

  • qm-leak-centos5-01
  • qm-leak-macosx-01
  • qm-leak-winxp01
  • qm-leak-w2k3-01

These machines are a little different: they’re producing debug builds and are running the standard suite of unit tests (reftest, mochitest, mochichrome, and browser chrome), but they are doing so with trace-malloc, bloat, and leak logging enabled as well.

These unit test build machines are currently reporting to the MozillaTest tinderbox tree. Jeff Walden has had a preliminary look at the output, but the tests still remain chronically red/orange. What we need now are more eyeballs to compare the failures we’re seeing on these boxes with the output of the same tests on the unit test machines reporting to the Firefox tree to tease out the legitimate failures and get bugs filed.

If you are interested in helping, you can find more information about debugging memory leaks at the Mozilla Developer Center.

Current Tunes: Gabriel & Dresden - Essential Mix - 2003-03-09 | Filed under Build/Release, Mozilla, QA

Community a go-go

Posted 7 years, 6 months ago at 22:35. 0 comments

I’m not even supposed to be here today, but I just had to write a little something about the Firefox 3 Beta 1 test day that we ran last week.

Tomcat says that some of the European tech press picked up our testday announcement, and it certainly showed. Lots of new faces in #testday, and over 1300 test results filed by the community over the course of the day (another 270 if you add results from Mozilla QA staff). This was even more impressive considering that this was the second test day we held last week. In the past we’ve seen an attendance drop-off if we hold events too close together.

People seem genuinely stoked about Firefox 3. Lots of testing remains to be done, so I hope to see a lot of those folks back to help some more. Keep it locked to QMO for details.

Current Tunes: The Orb - Bedouin | Filed under Litmus, Mozilla, QA