The roads I take...

I was pointed on IRC to "Code Rush" being available online, and as I never saw it before, just viewed it.
It really was a crazy time 10 years ago when Mozilla saw the light of the day as an open source project. "We're doomed" is something one hears again and again in this documentary about those days at Netscape. I know that feeling. I wasn't there from the start in 1998, my first download was M5 somewhere in 1999, and the first fully localized German build I released was M11 or so on January 1, 2000. I didn't see all of that happen, but just was on the way into the project where this report ends. It also was a hard time for all of us Netscape fans back then, and I also wondered sometimes if that effort was doomed. But I believed in the technology, because it was sexy in a technological way.
And here we are, about 10 years later - almost an eternity in this market - and look where we are! Mozilla is alive! And not only that, it's probably more alive than ever. We have a multi-million-download browser as well as a number of other projects, like a mail client, a complete Internet suite, calendaring software and whatnot. And we're going stronger than ever.
Yes, it's really a crazy time - still.

Oh, and if you haven't seen "Code Rush" yet, go and watch it. As a bonus, you'll even get to see how Stuart hacked around in mozilla/xpfe/ before he even joined Netscape!

After my recent post about SeaMonkey and Thunderbird on Mercurial, discussions started about possible solutions in both our projects as well as the Calendar project. I worked on the preliminary solution I had a bit further, and this week, we had a meeting of our projects with hg and build system experts, concluding that we'll create a new Mercurial repository shared between all three projects, and we'll already start off with more or less our own build system in place.
At the beginning, we'll import static snapshots of SeaMonkey and Thunderbird code, calendar will be pulled from cvs for now and added as another static snapshot once the calendar team is ready to switch to trunk for development. ("Static snapshot" in this case means current state of code without history, cvs/bonsai can be consulted for the history and work is ongoing to integrate the annotate view of hgweb with a database of older history to make it more feature-complete.)

I volunteered to create a test repository and work on getting things to build on local computers, eventually even work on test/build machines so that we have a good migration path.
For this purpose, I rewrote the SeaMonkey:hg-based_build page on the wiki to describe how to build with that test repository, and as of now, the instructions there (should) lead to fully working builds of SeaMonkey and hopefully also Thunderbird!

Please let me know of any problems when trying to build this or run the resulting binaries.

The next steps I'll work on are getting Lighting to build with Thunderbird, make tests run correctly, drive the two needed patches into mozilla-central and get my changes (build system, mainly) reviewed.

It looks like we can soon end up with a working code repository for our projects - what we still need is a good name for it. My first temporary name was "calemaisu" (calendar-mail-suite), internally, I'm currently using "momomo" (Mozilla Messaging, i.e. "MoMo" + SeaMonkey), which was a proposal in #maildev. There's still some need to discuss this topic between our projects, I guess.

I've just announced our first SeaMonkey status meeting on the newsgroups. This meeting, is set to take place on July 1st at 12:00 UTC in the #seamonkey IRC channel.

We're planning to do such meetings every other week from now to get a good overview of the current status of the SeaMonkey project and where it's headed.

I hope everyone interested will be there. If you can't join this time, please tell me why and what you'd like to contribute.

Here's a summary of SeaMonkey/Mozilla-related work I've done in week 25/2008 (June 16 - 22, 2008):

• Version Control and SeaMonkey Development:
  Again, I spent quite some time refining my build setup and discussing Mercurial for SeaMonkey and Thunderbird, and calendar entered the scene as a possible additional project sharing this repository.
  Nothing has been decided yet, but a lot of things are out there to decide and we'll follow along.
  I also filed some bugs about fixes we probably need in Lightning independent of what repository calendar ends up in.
• Automated Testing:
  I filed a bug about the problem we have with xpfe autocomplete failing Mac tests.
• SeaMonkey Status Meetings:
  I've sent out mail to the Council and a number of active developers, and I got a few replies, a status meetings wiki page has been set up as well with a few pieces of information, something's definitely starting to get going here.
• SeaMonkey 1.1.10 Release:
  The release process is going forward, but there were a few last-minute security fixes in Gecko we needed to take, so we made a respin of the 1.1.10 candidates and rescheduled the release for July 1.
• Help Updates:
  I finally updated the patch for the helper applications help again.
• Various Discussions:
  Firefox 3 release and download day, session (re)store, toolbar customization, treelines, Thunderbird/mailnews development, mozilla-central/1.9.1, bidi support, etc.

While Firefox has neared and released a major new version with really great marketing and resulting download numbers, and Thunderbird development has picked up so they could release their first alpha and are preparing the next one, news have been a bit silent on the SeaMonkey side recently. When seeing what the collection of status item in the meeting agenda is revealing, I start to get more optimistic again and feeling that we are on the right track after all. We surely need all the help we can get, but I hope at least over the summer we can get to a stage where we can show off a first testing version of a finally toolkit-based Internet suite to the community, which will be a great milestone.

That said, we of course want to integrate some things that are new, try out how well some things developed for Firefox 3 can work for us and test some changed default settings. We will surely not going to ever include a clone of Firefox as our browser, but sometimes we'll introduce changes in an alpha just to see how a wider audience than the hardcore nightly users reacts. Please don't kill off efforts to do that before we have the chance to get such feedback. Oh, and if you know concrete things you like better in SeaMonkey than in Firefox, please let us know, preferably on the newsgroups / mailing lists - not that we want to fight our friend Firefox, but we want to know how we can target our niche of the market even better.

One of the fun things of operating a page like the German SeaMonkey (and Mozilla suite) default start page is that one gets interesting statistics on Internet usage, in my case at least from the German-speaking part of the Internet.

And sometimes this reveals quite interesting stuff:
The pictures below are snapshots of my statistics from June 16, 2008. Your quiz questions for looking at those graphs: When did the European Championship soccer game between Germany and Austria start? When was the (15min) half-time break? When did the match end?


Server's network connections


CBSM system sessions (5min granularity)


seamonkey.at sessions (15min granularity)


(Notes regarding the numbers: seamonkey.at serves the default home page of German SeaMonkey and Mozilla suite installations and is traffic-wise the major domain running on the CBSM system, which is one of two larger PHP-based web systems on the server. CBSM sessions time out from the statistics after 5min, both connections and sessions are snapshots of what was considered open by netstat or CBSM at a given time.)

It took me a while, but finally I uploaded a "best of" containing 100 photos from my USA trip (California plus Las Vegas, see my Google map for waypoints, green and special symbols for visited spots) up to the galleries on my site.
I even added comments to all images so everyone should know where they were made. Once again, thanks to preed for the great Bay Area plane tour! Oh, and don't look for Mozilla pictures, I figured the experiences there were better than my pictures.

Yes, Firefox 3 has been released and did set the world record with probably (judges need to determine the official number still) 8 million downloads in 24 hours. For one thing, congratulations to Mozilla Corporation for that marketing boost and to the whole Firefox community for delivering an improved and still damn good browser for the masses.

So, the world is good, the sun is shining and everything is fine. Or is it?

For one thing, I can't hear the word "awesome" any more. I think some people should listen to the song "Now That's Awesome" by Bill Engvall and realize that this inflational use of that word is probably removing its real meaning. Thanks.

Then, something that has to be said is while the mass products make the big headlines (Firefox does that well), the nice products fill up the gaps in the market that those mass products can't fill. That's what e.g. SeaMonkey or Camino are trying to do, and I hope we have our success with that.

And then, sure, Firefox is showing that Mozilla and open source can produce and even market mass products. Good. But is that all we are here for? I don't think so, and I think it's quite interesting that Marc Surman from the Shuttleworth foundation has written up a blog post about his thoughts that the Mozilla Foundation should do more than what it does right now and try to encourage users to become participants/contributors in our vision of the open Internet, making this a really good place for the future and enriching the whole world this way. Let's start our quest for improving ourselves and mankind as a whole with a really open Internet (oh, Captain Picard and his crew from Star Trek's "The Next Generation" would love me now if they were real)!

All in all, there's still some way we need to go until everything will be fine - and I think every step in that direction is worth making. Mozilla Foundation can play a good part there, I hope SeaMonkey can as well in its "corner" of the net, and if we make it, then - and only then - it will be awesome. And I mean that.

Here's a summary of SeaMonkey/Mozilla-related work I've done in week 24/2008 (June 9 - 15, 2008):

• Version Control and SeaMonkey Development:
  I again spent a lot of time this week with figuring out SeaMonkey and Thunderbird building with mozilla-central, blogging about it and posting on the newsgroups about this topic, as well as fiddling out more details, like removing (almost) all the hacks I had in my build setup in the beginning. And, of course, I discussed the topic with people on IRC.
  Additionally, I got Lightning to build (but not install yet), which I still need to document (and which needs a few fixes build-wise when calendar goes hg). From discussions on IRC, I feel we badly need to discuss the whole topic with calendar people directly, as our views of the future world diverge a bit there.
  The L10n story is still something I need to figure out, it might still need some tweaks.
• SeaMonkey Status Meetings:
  We had a small bit of discussion about this on IRC, we might go for biweekly meetings, not sure if we'll do them on the phone or on IRC though.
• SeaMonkey 1.1.10 Release:
  It's been some time, but finally, another security release is for 1.1.x is due later this month - I took the first steps on the release process, i.e. version update and tagging on the release branch, as well as starting the generation of 1.1.10 candidates.
• Various Discussions:
  OS X address book, cleaning out old L10n nightlies, session (re)store, toolbar improvements, treelines, tab preview, Thunderbird/mailnews development, mozilla-central/1.9.1, etc.

A discussion about the status of SeaMonkey Composer came up on newsgroups, and it's pretty interesting: I'm reading a lot about different people and groups using and promoting SeaMonkey Composer for designing simple HTML pages, I heard of teachers using it in classes, and recently, even a German publishing house sent a request for putting SeaMonkey on their book CD as they described how to use it in the book.
Composer is becoming one of the strong arguments for SeaMonkey nowadays, it seems, especially in the light of all the spinoffs either being unmaintained (Nvu), unfinished ("Mozilla Composer") or still based on old code (KompoZer). We have the only stable release of Composer code that's based on Gecko 1.8(.1), and the only development series of it that works with 1.9.x.
Unfortunately, the Composer-specific code is also not actively developed (a lot of the functionality is shared with mailcompose and Midas though), while we still maintain it to a state that it stays working and can be released even in SeaMonkey 2. If someone comes up and works on improving this code, possibly even porting improvements from Nvu/KompoZer to our code, it would be very much appreciated and find a good base of users.

It might seem that Mozilla people suddenly turned into chemists and are embracing the only liquid-under-normal-conditions metal, a toxic substance known as Hg or Mercury. Or they may just have learned that distributed version control systems (DVCS) have advantages over the old, file-based, centralized cvs system, and turn to a tool called "Mercurial", or "hg" for short, as a reasonable compromise between feature sets, well-working, well-maintained software and being able to run it decently on all our development platforms (including, even very prominently, a proprietary OS from a small company based in the town of Redmond, WA, USA).

The "trunk" of Mozilla development has moved to this tool at least, to a repository we call "mozilla-central" (people are working on getting this "hgweb" display nearer to feature-parity with the bonsai tool we used with cvs, including bug links and such stuff), the code in which is also web-browsable via MXR. With cvs HEAD now being basically the "1.9.0 branch" of Mozilla development and mozilla-central not including code outside of the Mozilla platform and Firefox, projects like SeaMonkey and Thunderbird need some changes if they still want to follow trunk development. As even Mozilla 1.9.1 will come off mozilla-central and the two mentioned projects may want to release their next big releases (Thunderbird 3 and SeaMonkey 2) off that (no firm decisions have been made yet), this topic gets even more pressing that originally imagined, when the move to hg was supposed to be for "Mozilla 2" only.

Over the last two weeks, I have intensely been investigating how we could do such a move, based on "option A" of a repository options doc that Benjamin Smedberg has thankfully written up for us, as most SeaMonkey and Thunderbird developers I talked to seemed to agree with Benjamin that this "option A" would be the best solution to go with, if it's feasible. Because of that, I worked on a proof-of-concept of this option - and amazingly, with a lot of help by Ted Mielczarek, I could figure that out more easily than I would have imagined.

And here's what I did:

Let's start with what this "option A" actually means. The basic building stone of this new build structure is that the mailnews/ directory with shared mail/news backend code as well as the Thunderbird-specific mail/ and SeaMonkey-specific suite/ directories can live in a shared hg repository, with the easy possibility of single changesets affecting all three (which we have frequently when a mailnews/ backend change needs e.g. locale string changes, which are done in both mail/ and suite/). While we can nest hg repos in a way that one subdirectory inside one repo is actually another repo that is being pulled in, we can't have a selection of subdirectories being in a different repo than another selection on the same level. What this means is that mail/, mailnews/ and suite/ can't live in the toplevel like in (the shared) cvs.

So, in this option, the new repository, which I call "mailsuite" for now, contains the said directories and pulls in mozilla-central into a mozilla/ subdirectory in parallel to them (we can automate that with a client.py script, of which I have a working version on my disk). As Ted proposed, we can still use the Mozilla build system to build our applications with one central tweak: In your mozconfig (or ./configure call), set --enable-application=../suite (or ../mail). This means that the top-level source directory for the Mozilla build system is actually the mozilla/ subdirectory of our mailsuite repository! Because of that, the specified objdir also needs to end in "/mozilla" - both those tweaks can be hidden with client.mk/configure scripts in the mailsuite repository in the future.

Starting with that, we need a list of rather boring changes to our "mailsuite" files to deal with the situation that they now live below $(topsrcdir)/../ instead of $(topsrcdir)/ from the view of the Mozilla build system, as well as some tweaks to set our own build variables ourselves, when they were set by e.g. Mozilla's configure script previously. Benjamin did a very fast patch to introduce app-config.mk/app-rules.mk which we can use to do the latter.

A full list of how one can get SeaMonkey or Thunderbird building with that structure is available in a wiki doc I wrote up, most of this will be done by e.g. client.* in our own mailsuite repository (including pulling in external extensions like ChatZilla or venkman) in the future or just checked into files in that repository.
Still, there are a few things left I still need to figure out:

• LDAP - directory/ needs to be either in mozilla-central or reworked to live in and build off mailsuite
• Dealing with SeaMonkey's built-in extensions, i.e. making them optional at build time but not require symlinking them into mozilla/extensions/
• Get make-makefile to work, Ted has promised to look into this
• Building Calendar (Lightning) from the same source structure, as a built-in extension to Thunderbird/SeaMonkey
• L10n: localized builds, L10n repackaging

The main thing here is though that I have working builds of SeaMonkey and Thunderbird built off that structure on my computer - I actually even could successfully run unit tests on SeaMonkey!

I personally think we can and should move to this structure (and to 1.9.1) for Thunderbird 3 and SeaMonkey 2, esp. as I'm told that there's the target to keep the 1.9.1 trunk in a shippable state at all times. We probably need to do some reconfiguration of our build and testing machines, but I think that's manageable.

The biggest open question (apart from the mostly technical issues listed above) is how we do the transition the this new repository. I could import a static snapshot of our cvs directories right now for testing and apply my local changes as a first changeset, but I'd rather have firm decisions if Thunderbird and SeaMonkey really want to go with that solution in the future and import the whole history into the final repo right away, as well as putting that transition changeset of mine right there.
For the time where we have both repositories (cvs and hg) in action, we still need to figure out if people should manually commit to both or if we use a script that merges in the changes from cvs (which was done for mozilla-central in for some time). And, of course, we need to figure out timeframes for all the needed steps.

So, all in all, we have still some lose ends to tie up, but we have a sketched out good way to go towards hosting SeaMonkey and Thunderbird in Mercurial - and I hope and am pretty sure it turns out less toxic than what the name "hg" suggests to me as an actual chemist.

Here's a summary of SeaMonkey/Mozilla-related work I've done in week 23/2008 (June 2 - 8, 2008):

• Automated Testing:
  TUnit on the Linux tinderbox is off now, the new buildbots are happily running that instead.
  Unfortunately, we still have no clue why a few autocomplete tests fail only on the Mac unit test machine but not on the other platforms. Help would be appreciated.
• Version Control and SeaMonkey Development:
  I spent most of my time this week on trying to get a proof of concept on "option A" of Benjamin's repository options doc, actually with nice success. A bug report for tracking this is now open, app-config.mk/app-rules.mk have been introduced to help us there (thanks Benjamin!), and I filed a bug on the make-makefile issue with that, with a hacky patch for now, but Ted is looking into a better solution.
  In the end, I got SeaMonkey and Thunderbird to generate working build off that structure, a doc about how to get it to work is up on the wiki, but it still describes a few hacks (mainly the symlinks into mozilla/). We can ride on the Mozilla build system even with our repository being the parent of mozilla/ - we just need to prefix the --enable-application value with ../ and have the objdir path end in /mozilla, but we probably can hide both of those via top-level build system entry files (client.mk/configure) in our own repository once we have one. I also (locally) have a client.py that pulls all the source we need.
  All in all, there are a few things we still need to figure out, but there's a good proof-of-concept of the structure, opening the possibility to chose this option for our future. I may possibly do another post with more details in the next few days.
• SeaMonkey Status Meetings:
  This is still on my TODO list, I'd really like to get something going there.
• Various Discussions:
  Mail client cert prompts, about: privileges, OS X address book, MoFo ED Search, Thunderbird/mailnews development, mozilla-central/1.9.1, etc.

While I'm working on getting us to a new build structure that allows us to have SeaMonkey and Thunderbird source in a new, shared hg repository and build with mozilla-central/1.9.1, I'm getting more and more convinced that both our projects should ditch 1.9.0.x and go straight for 1.9.1 instead, leaving cvs behind for our development. There's still a bit of work to do before everything runs smoothly that way, but I think the sooner we do this the easier we can deliver a release on the new version control infrastructure - and the easier we still can get platform changes into Mozilla 1.9.1 where we come across needing those. Note though that this is completely my personal opinion, there has been no decision in either of our two projects about this.

Here's a (very late, sorry) summary of SeaMonkey/Mozilla-related work I've done in week 22/2008 (May 26 - June 1, 2008):

• Automated Testing:
  The new Mac unit test machine is up and running, but failing some address book unit tests (missing OS X integration) and some autocomplete mochitests (reason still unknown).
  Also, there were no negative comments about turning off the TUnit tests on the Linux tinderbox, so I'll do that soon.
• Preference Migration:
  The advanced pref panel is checked in now, so all top-level panels are now migrated!
• Windows cygwin builds:
  As I have an old cvs trunk tinderbox running with cygwin, I checked in a fix for building 1.9.0.x with cygwin.
• Version Control and SeaMonkey Development:
  We had lots of discussions around the future of SeaMonkey's (and possibly Thunderbird's) source code management after Mozilla trunk has moved to mozilla-central, which is hosted in hg and will not only result in a Mozilla later on, but even in 1.9.1 - which is possibly what we want to base Thunderbird 3 and SeaMonkey 2 on.
  As a possible way to having our own code in hg as well and using this 1.9.1 mozilla-central repo for the platform code, Benjamin Smedberg proposed several repository options. We have no firm decision on what options we will go with, but there seems to be a strong tendency across multiple core developers of our teams to go with "option A", which would build the platform in a subdirectory of our own repository.
• German L10n:
  SeaMonkey L10n is complete again on trunk, though it stays orange due to the string from news header filters which I did leave in hoping that the original patch comes back.
• SeaMonkey Status Meetings:
  Unfortunately there has been no feedback to this, I still want to drive that forward though.
• Various Discussions:
  ChatZilla and chrome.manifest, session (re)store, Thunderbird/mailnews development, Firefox Plus Summit, etc.

Sorry for being quite late again this week with the status update, but this week I've been quite busy with getting working proof that we can build SeaMonkey with mozilla-central and finally have a hacky but working way for doing this. I have pushed back blogging this status update for that, I hope you don't mind.
I'll probably write up another blog post about this in the next few days, but it feels good to know how we can build SeaMonkey based on 1.9.1, so the option is open!