The roads I take...

One thing is closely following after the other, now that I have synched up even the master configuration the the SeaMonkey buildbots with what Firefox has.

First I got L10n nightlies triggering again (and in a much better way, actually), then they did even gain automated updates, and after that I started experimenting with running what we call "packaged tests".

What that means is that all files for running our test suites get packaged up and uploaded to FTP, from where any machine can pull and run them. With this configuration, we can make test suites run in parallel if idle machines are available, and therefore shorten the time until all of them are run complete. Also, they show up as distinct columns on tinderbox so one test suite failing doesn't turn the whole unit test column orange but only the one for that suite, and failures indifferent suite can more easily be tracked individually.

Our only tree running tests right now is the SeaMonkey2.0 one, and I now have turned on running tests that way there - with the exception of xpcshell tests, as mailnews tests have a remaining issue that needs to be solved first. The old "unit test" columns now only do the builds, package up those and the tests, trigger the other test runs, and then run "make check" (binary tests, cannot be packaged) and the xpcshell tests. Everything else runs in different columns.

As a future step, we might be even able to not create special builds for the test runs, but have them run on builds that are done for the other columns anyhow.

This is a random collection of thoughts going around in my head - probably all of them would warrant a full blog post, but I don't come around to doing them, so I'll put down the short summaries.

• Geolocation: Why do we have to rely on Google magically knowing where someone might be? Wouldn't it be better to have a community-driven database we all could contribute to, which could give you croudsourced location data?
  For example, I know where I am, I can give precise location of both my wifi IDs as well as a range of IPs up to the building, but Google Location Service (what Firefox uses for the internal geolocation module) just tells me I'm somewhere in Vienna. They don't have a way for me telling them my info and improving that information, but an open, community-based service could. And OpenStreetMap would even know full address data for this location. How dull that all our modern technology just tells me I'm somewhere in a multi-kilometer radius around the Vienna City Center.
• SeaMonkey Meetup: The SeaMonkey project has some amount of donations stacked up at Mozilla Foundation, and I think it would be cool to use that to finance a SeaMonkey meetup, paying for accommodation and travel of major contributors as well as the (if needed) the place to do the meetings. Would this be a good idea? Who would come, maybe even if we can't pay for him, in what city should we do this event?
• Contribution Statistics: I have thought a few times about doing a script that parses our Mercurial pushlog at least between releases, and gathers data on how many changesets and +/- codelines people have created and/or reviewed, to get a view of which people are how active in the community. The same could be done for bug triage. The result would be something like Jonathan Corbet's Linux Kernel Developer Statistics.
• SeaMonkey QA: We're really missing someone to lead and coordinate SeaMonkey QA work - Andrew Schultz is quite busy with some strange thing called "real life" nowadays and can't really do that work right now. I'm sure we'd all be quite happy if he could pass the torch and we'd support anyone who tries to do it with all help we can provide.
• Web-based Help and Support Resources: We have some weekness in SeaMonkey help and support resources on the web. While our in-product help is fine, it would be good if a Google search would turn up something helpful and if we could point people to URLs. One partial solution would be to have a script that periodically converts our inline help to usable web pages, and better solution would be to set up a copy of SUMO for SeaMonkey, with a knowledge base and possibly even a web forum - but someone needs to drive that. Any volunteers?
• SeaMonkey Marketing: Even though I'm theoretically responsible for marketing right now, I badly fail on getting anything done, starting from putting a page with a collection of logos and web buttons up, and moving on with all other possibilities of fostering community marketing. This is another area where I'd be happy to have someone come on board and drive this. I'm happy to support any efforts from a technical and project organization POV, but we probably need someone else to lead those efforts.
• Mozmill: Thunderbird is starting to automate tests on Mozmill now, Firefox QA people start using it for smoketests, could someone get it to run for SeaMonkey so we can do those things as well?
• SMILE, Weave, Jetpack: There are more things out there that probably need help: the equivalent to FUEL and STEEL, which we call SMILE, getting Weave to work for SeaMonkey, and last not least, getting Jetpack to work (which probably needs SMILE).
• Parallels: Why does it need to be so painful to run OSX in VMs? And does nobody else run a larger number of VMs, including OSX machines, on Parallels? I don't understand how we can have basic problem like not being able to run more than 8 VMs on one host and OSX VMs being unstable esp. if they have access to more then 1 CPU core and both thing not getting much traction from Parallels devs. It can't be that we are the only customer who see those problems.
• Statistics: I would love to have a lot more statistics on users, downloads, etc. for SeaMonkey and esp. SeaMonkey 2 but it seems to be hard to get the data and tooling that exists inside Mozilla systems out in a way we can use it. I guess Mozilla Corporation is not as open as it could and should be in some areas.
• openSUSE: With the inclusion of SeaMonkey 2.0 Alpha 3 (soon to be Beta 1, they already have Thunderbird 3.0 Beta 3), in the current openSUSE Factory, it looks very much like openSUSE 11.2 will be the first distribution to have SeaMonkey 2 in their official package repositories. Thanks to their package manager Wolfgang Rosenauer for making this possible!
• Moving: I'll finally be moving from a student dorm to my own flat in August, lots of stuff to think about there. Also, the machines building SeaMonkey 1.x nightlies and releases are about to be moved to a different location, Linux and Windows being unavailable there recently is connected to this, I had to clear up how to do this, and some missing responsiveness on the side of my provider contributed a lot to finally deciding to switch providers in that process.
• Vacation: I have already booked the flight for my vacation this year, I'll be away for three weeks in November, traveling through the US gulf region, circling from Houston via New Orleans, Pensacola, Atlanta, Nashville, Memphis, Dallas back to Houston. It will be quite a distance to travel, but it should be manageable and be a good distraction from my usual work, and lots of things to see and experience. I easily get excited when talking about this.
• Music: Sometimes I'd love to be a signer in a local Blues/Rock/Country band, but I hardly find the time to practice the guitar any bit or type the lyrics of my recently written songs into the web database I have for them. At least I come around to some Karaoke singing every week.
• Space: How come that the great things NASA does is not worth more to the public than the half cent of every US tax dollar it actually gets? How come that it isn't worth more to other countries as well? Isn't exploration of new frontiers, world-wide cooperation to do amazing things not because they are easy, but because they are hard, aren't all those things one of the main drivers of what makes humanity great? Are we losing focus in that we are only caring about our own small biotopes and internal affairs and forgetting to expand our knowledge and horizons?
• Test Coverage: It would be so nice to increase coverage of SeaMonkey code with automated tests, but it's proving even hard to require tests for new things added, as we also don't want to slow down progress - esp. when we are already behind the schedules we hoped to follow a few months ago.
• Mozilla 1.9.2 and SeaMonkey/Thunderbird: Mozilla platform maintainers decided to do a 1.9.2 branch very soon now and base Fennec 1.0 on it as well as a Firefox 3.6, not featuring lots of application changes, but some good platform improvements. Some of those changes in the platform would be good to have for SeaMonkey and Thunderbird, but we also know of some problems we'll have there due to doing our experimental builds with mozilla-central all along. Moving over to the branch now would probably delay our stable releases for a few weeks more, but we are already running behind the schedules we wanted to have, so we think it's better to stick with 1.9.1 for now and get SM 2.0 and TB 3.0 out before even thinking of what to do about 1.9.2 - we could either ignore it completely or do smaller-step 2.1 and 3.1 releases on top of it just like FF does with 3.6, but we're not sure what's best. For now, we'll watch it but not actually do anything about it.
• MailNews API refactorings: It would be really nice if we could port the JS-driven folder pane and the various refactorings done for gloda search from Thunderbird to SeaMonkey UI, as those would sync our APIs with theirs and make life easier for add-ons, next to making work with folder and thread panes easier internally as well. Once again, what we're missing is someone to do the work - we are a volunteer open source project after all, and people here tend to work on those things that are fun for them, and of course only in the little free time they have.
• Local Communities: Every face-to-face meeting i had locally with open source developers around here in Vienna was very rewarding, and I should engage much more with those communities. Also, my recent talk for IT businessmen on "project management in open source" was a very exciting and successful thing, I believe I could, with the help of a few fellow open source community people, dampen a lot of FUD that arises with people used to traditional IT business but who are still interested in how thing work "on our side" - which I hope to have proven to not actually be that much different as they often think. By the way, and I got got comments like "Oh, so the suite is still being developed? Nice, I need to try SeaMonkey then!"

You see, lots and lots of things are going around in my head all the time - this is still only an excerpt, but I thought a should get a few words on most of them out to you some time, so here's those bits. More thoughts will come in the future, I'm sure - i'm a thinker, you know

When I read about a test that runs well in SeaMonkey being moved from testing/mochitest/ to browser/ yesterday (as it tests stuff that is actually internal to browser/ JS), I decided it would be good if SeaMonkey would still run it. And as I was already looking into moving/porting a browser test, I decided I would look into what other tests from browser/base/content we should have.

Following that, I spent the whole day trying tests and porting those that are useful to suite/browser - finding a number of things to add to the Firefox to SeaMonkey porting list as a number of tests were for features we haven't implemented yet (helpwanted!) as well as a few bugs in SeaMonkey, which I either fixed right in the patches for adding the plain mochitests and browser-chrome tests or marked todo() relying on followup bugs to fix them.

In the end this should result in easier extensible browser context menus and better accesskeys within them, improved feed detection, and plugins handling improvements - in addition to detecting more regressions in the tested areas when we're doing future work.

What I really like with that work though is that it uncovers existing bugs (that are sometime easy to fix) in addition to early detecting future ones.

That's why I proposed increasing automated test coverage as a Summer of Code project for SeaMonkey - if you are a student and want to learn Mozilla code, please consider applying for this one, you'd even get paid for it!

Well, almost.

The "open browser and close again" as well as the "open mail and close again" tests our (Linux) bloat test box is doing both reports 0 refcnt leaks now, even without the slight hack we had in there until today (not loading the EM RDF datasource and not populate the "View > Apply Theme" menu with actual themes).

And even the chrome mochitest suite reports "no leaks" on Linux and Windows.

Unfortunately, we somehow still leak 1112 bytes on Mac for those tests. For the plain mochitest suite, where we run much more tests, we report 8 bytes leaked on Windows and Linux, on Mac OS, we report those 8 plus the previously mentioned 1112, making up 1120 bytes total. And for the browser chrome mochitests, we leak 1112 bytes on Linux and Windows, and 1232 on Mac. Somehow this 1112 number is something we like - interestingly, we get it in those places where one non-browser window is open in addition to the standard browser window (the "hidden window" on Mac or the additional window for browser chrome tests), I wonder if that's somehow connected. Also, we're leaking the RDFServiceImpl and an InMemoryDataSource there, which could be an indication what to look for, I guess (unfortunately I don't know the code too well myself).

By the way, I just worked a long-standing workaround to our EM RDF datasource leak into an actual patch, removing the datasource in the unload handler of the browser window, the real fix would be in cycle collection though.

Oh, and have patches waiting for review to get the actual test failures on SeaMonkey machines out of the way and hopefully turn them green for the holidays.

As many of you know, one of the new addition to Firefox development between the Gecko-1.8.1-based Firefox 2 and the Gecko-1.9-based Firefox 3 releases was performing automated tests on all kinds of functionality in this software. This greatly improves software quality by having constant checking of a good part of this complex system and being able to early spot problems introduced in code and revert/correct those.

Starting this week, we also have machines that run a broad range of automated tests on SeaMonkey trunk (the Gecko-1.9-based code that will end up in SeaMonkey 2). Currently, we have Windows and Linux boxes (bug for a Mac box is filed - unfortunately, Apple doesn't support those being run as VMs), reporting to the SeaMonkey-Ports tinderbox waterfall page - and failing a number of tests.

The test failures we see with those machines have different causes:

• The mochitest framework has some focus problems, and generating a keypress event doesn't have the intended effect when the wrong window has focus. Applying the patch from bug 431464 helped with the vast amount of those problems (thanks, Andrew, for figuring that out).
• The remaining mochitests failures on Windows might be because not the whole iframe is in focus and some generated mouse events don't point into it, maybe due to a too small window size. We need to look into improving that.
• Failures in browser chrome tests due to multiple problems, including use of Firefox-only functionality in platform tests, testing a download manager that is not (yet) built for SeaMonkey, and some errors in SeaMonkey sanitizer tests. I could create a patch for those which is currently under review.

The machines will be switched to report to the main SeaMonkey tinderbox waterfall page and be required not to break in future development, once we have fixed those problems and all tests pass.

We have run the unit tests for Linux in our main tinderbox for a while now, but without detailed reporting of numbers, and without coverage of reftests, crashtests, mochitests, chrome or browser tests or other platforms. We now have all those test suites running on two platforms, hopefully adding Mac "soon".

Even a number of about 51500 passing tests doesn't mean that a lot of SeaMonkey-specific code is tested, though. What we're currently are running, are mainly platform tests we share with Firefox - we only have a total of 1 suite and 14 mailnews unit tests being run, and the sanitizer tests account for 31 passed browser-chrome tests (with my patch applies so they actually pass). All other SeaMonkey application code is still not covered by such tests.

What this means is that even though we have machines running the tests, we badly need tests for SeaMonkey code (including mailnews [hint for Thunderbird people reading this]) to be checked in so they get run. Still, we know that the platform still works fine in the configuration used by SeaMonkey, which is good to know and already a major improvement to earlier times.

I'm convinced that amount of automated testing will even improve the jewel of product that SeaMonkey 2 will be. That said, automated testing can only augment manual testing by our community, and never replace it, so please keep on testing our nightly builds - those tests will help to make us not have bad breakage in those nightlies though (at least not for long - it's always a good idea to check for green tinderboxes before downloading and using a nightly, as I hope you know).