<< Weekly Status Report, W23/2008 | The roads I take... | Weekly Status Report, W24/2008 >>

A Possible Way For Hosting SeaMonkey And Thunderbird In Mercurial

It might seem that Mozilla people suddenly turned into chemists and are embracing the only liquid-under-normal-conditions metal, a toxic substance known as Hg or Mercury. Or they may just have learned that distributed version control systems (DVCS) have advantages over the old, file-based, centralized cvs system, and turn to a tool called "Mercurial", or "hg" for short, as a reasonable compromise between feature sets, well-working, well-maintained software and being able to run it decently on all our development platforms (including, even very prominently, a proprietary OS from a small company based in the town of Redmond, WA, USA).

The "trunk" of Mozilla development has moved to this tool at least, to a repository we call "mozilla-central" (people are working on getting this "hgweb" display nearer to feature-parity with the bonsai tool we used with cvs, including bug links and such stuff), the code in which is also web-browsable via MXR. With cvs HEAD now being basically the "1.9.0 branch" of Mozilla development and mozilla-central not including code outside of the Mozilla platform and Firefox, projects like SeaMonkey and Thunderbird need some changes if they still want to follow trunk development. As even Mozilla 1.9.1 will come off mozilla-central and the two mentioned projects may want to release their next big releases (Thunderbird 3 and SeaMonkey 2) off that (no firm decisions have been made yet), this topic gets even more pressing that originally imagined, when the move to hg was supposed to be for "Mozilla 2" only.

Over the last two weeks, I have intensely been investigating how we could do such a move, based on "option A" of a repository options doc that Benjamin Smedberg has thankfully written up for us, as most SeaMonkey and Thunderbird developers I talked to seemed to agree with Benjamin that this "option A" would be the best solution to go with, if it's feasible. Because of that, I worked on a proof-of-concept of this option - and amazingly, with a lot of help by Ted Mielczarek, I could figure that out more easily than I would have imagined.

And here's what I did:

Let's start with what this "option A" actually means. The basic building stone of this new build structure is that the mailnews/ directory with shared mail/news backend code as well as the Thunderbird-specific mail/ and SeaMonkey-specific suite/ directories can live in a shared hg repository, with the easy possibility of single changesets affecting all three (which we have frequently when a mailnews/ backend change needs e.g. locale string changes, which are done in both mail/ and suite/). While we can nest hg repos in a way that one subdirectory inside one repo is actually another repo that is being pulled in, we can't have a selection of subdirectories being in a different repo than another selection on the same level. What this means is that mail/, mailnews/ and suite/ can't live in the toplevel like in (the shared) cvs.

So, in this option, the new repository, which I call "mailsuite" for now, contains the said directories and pulls in mozilla-central into a mozilla/ subdirectory in parallel to them (we can automate that with a client.py script, of which I have a working version on my disk). As Ted proposed, we can still use the Mozilla build system to build our applications with one central tweak: In your mozconfig (or ./configure call), set --enable-application=../suite (or ../mail). This means that the top-level source directory for the Mozilla build system is actually the mozilla/ subdirectory of our mailsuite repository! Because of that, the specified objdir also needs to end in "/mozilla" - both those tweaks can be hidden with client.mk/configure scripts in the mailsuite repository in the future.

Starting with that, we need a list of rather boring changes to our "mailsuite" files to deal with the situation that they now live below $(topsrcdir)/../ instead of $(topsrcdir)/ from the view of the Mozilla build system, as well as some tweaks to set our own build variables ourselves, when they were set by e.g. Mozilla's configure script previously. Benjamin did a very fast patch to introduce app-config.mk/app-rules.mk which we can use to do the latter.

A full list of how one can get SeaMonkey or Thunderbird building with that structure is available in a wiki doc I wrote up, most of this will be done by e.g. client.* in our own mailsuite repository (including pulling in external extensions like ChatZilla or venkman) in the future or just checked into files in that repository.
Still, there are a few things left I still need to figure out:

The main thing here is though that I have working builds of SeaMonkey and Thunderbird built off that structure on my computer - I actually even could successfully run unit tests on SeaMonkey!

I personally think we can and should move to this structure (and to 1.9.1) for Thunderbird 3 and SeaMonkey 2, esp. as I'm told that there's the target to keep the 1.9.1 trunk in a shippable state at all times. We probably need to do some reconfiguration of our build and testing machines, but I think that's manageable.

The biggest open question (apart from the mostly technical issues listed above) is how we do the transition the this new repository. I could import a static snapshot of our cvs directories right now for testing and apply my local changes as a first changeset, but I'd rather have firm decisions if Thunderbird and SeaMonkey really want to go with that solution in the future and import the whole history into the final repo right away, as well as putting that transition changeset of mine right there. :)
For the time where we have both repositories (cvs and hg) in action, we still need to figure out if people should manually commit to both or if we use a script that merges in the changes from cvs (which was done for mozilla-central in for some time). And, of course, we need to figure out timeframes for all the needed steps.

So, all in all, we have still some lose ends to tie up, but we have a sketched out good way to go towards hosting SeaMonkey and Thunderbird in Mercurial - and I hope and am pretty sure it turns out less toxic than what the name "hg" suggests to me as an actual chemist. ;-)

Entry written by KaiRo and posted on June 10th, 2008 20:31 | Tags: hg, Mozilla, SeaMonkey, Thunderbird | 4 comments

TrackBack/Pingback

Comments

AuthorEntry

David Ascher

quote
OThanks!
Thanks for doing this digging. It's nice to know someone has actually figured it out, makes it less foreboding.
2008-06-11 00:29

Gordon P. Hemsley

from Long Beach, NY

quote
OBromine
Surely, as a chemist, you know that bromine (Br) is also a liquid at room temperature, and that the term "fluid" also encompasses phases other than liquid?
2008-06-11 01:01

Justin Wood (Callek)

quote
OHistory...
With regard to hg history, I think we should import history from the original mozilla-central Hg import date (there is a cvs tag for it) and use the importer script (which will need some changes to work for us).

Then when we are ready to switch primary dev to 1.9.1 we can "close" the CVS tree for whatever we import to Hg. And in between we can do periodic imports from CVS for those directories, (we can keep it all on a cvs-mailsuite-mirror repo or some such).

When we make the change we could/should tag the tip changeset as "Moved dev to 1.9.1"..., well a tag that could mean that. That will make history keeping quite well.

If Thunderbird and SeaMonkey wish to make the official switch seperately we can *require* CVS landings for that project and use the mirror-importer longer, merge conflicts should be easily rectified in such a case.

Re: extensions
It might make sense for an --enable-app-extensions=... flag that looks for a "app"/"extensions"/[extension] for each extension, and could have a "app"/"extensions"/list of some sort, with markings for "default" and for "all"

Might even make sense to have it as part of normal build system for said list and the real extensions dir could keep that (with IFDEFs maybe to do away with current configure rules)
2008-06-11 01:09

KaiRo

Webmaster

quote
OGordon: Er, sure... I replaced element with metal ;-)
I'm not too sure about English terminology though, so I was under the impression that liquids are more or less the same as fluids - but this may be wrong (I'm doing my best with translating what I know from German).

Last edited by KaiRo at 2008-06-11 04:21

2008-06-11 03:52

Add comment