The roads I take...

KaiRo's weBlog

June 2024

Displaying recent entries in English and tagged with "release". Back to all recent entries

Popular tags: Mozilla, SeaMonkey, L10n, Status, Firefox

Used languages: English, German


July 2023

February 2022

March 2021


November 26th, 2011

Firefox 7.0.1 updates to 8.0.1, but 8.0 doesn't?

I found the following question being raised on German newsgroups, but I guess there's more people interested in this:
I found out that on one of my computers Firefox 7.0.1 has been updated to 8.0.1, but on another one there's no update for 8.0, not even when I manually trigger it via "About Firefox". On the Mozilla website, 8.0.1 is also being offered as a download. What's going on there?

Short version:
If you're not using a Mac, switching from 8.0 to 8.0.1 doesn't really help, so we only offer that update for Macs, on all other, 8.0 is just as good, as long as it runs.

Long version:
8.0.1 fixes exactly two things over 8.0:
1) A crash with the newest version of Apple's Java plugin for Mac (the bug is in that new Java version from Apple, but we can work around it with a small patch on our side).
2) On Windows, we're blocking old versions of Roboform that cause 8.0 to crash on startup, we only allow newer versions that don't cause that crash. In case 8.0 is already running, i.e not crashing when starting Firefox, this update doesn't really help anything, and we're avoiding to send people an update when they have no benefit of it, so we don't disturb them. If 8.0 is installed but crashing on startup (because of old Roboform), we don't even get to where an update would be installed. The only thing that helps those people is to manually download 8.0.1 and install it freshly - and actually download a new Roboform version that works with it.
Because of that, 8.0.1 is being offered to all versions (starting with 4.0, incl: 8.0) on Mac OS, but on Windows or Linux only for 4.0-7.0.1, because it doesn't offer any benefit to all the others.

So, in the end, if you're installing a fresh version, 8.0.1 is the right choice, but you don't need to install an update where it's not being offered automatically.

By KaiRo, at 17:10 | Tags: CrashKill, Firefox, Mozilla, release | 5 comments | TrackBack: 0

August 27th, 2011

Why Rapid Releases Can Improve Stability

I have mentioned a few times, mostly in newsgroup discussions, that I strongly believe that the rapid release model Firefox is following now has a good chance to improve stability.

Some people without a deeper knowledge of how our new process works have at times implied that releasing way more often must make the product more unstable and worse quality than the one or two year cycles we had before. Given my multi-year experience in release management of a Mozilla product (SeaMonkey) and along with that insight into Firefox release management of the last few versions up to and including Firefox 4, my comparison of those experiences with the new model point into the exact opposite direction: Stability and quality should actually improve the more we get used to this "train" model and also the more we near the prospected user volume on the different "channels".

"Traditional" Process

Let's first look at how things worked with the old process that we used so far, including for Firefox 4: New work landed in the code for more than a year with first having only nightly testers run it every day, later alpha/beta testers running snapshots created along the way that included fixes found in some internal QA in addition to the nightly testing, but that was it for the alphas and betas - and at the point where those got shipped, we already had land the next set of feature changes on top of the code shipped there. From the view of crash analysis, this meant that we had a smaller audience of nightly testers sending crash reports we could analyze and from that see the larger and more obvious regressions from daily changes. And then there was a larger audience of beta testers that sent more data, which allowed a look at what happened with somewhat more real-world usage, but as soon as we got some good data in on those betas, the code on nightly on the way to the next beta might already have changed significantly again. With that, the most grave issues could be addressed, but sometimes it was hard to see how relevant the data from even the current beta still was. This game went on until the final betas, with increasing urgency of getting things in that should still make the release at the last second, and of course us as well as testers seeing new regressions that needed to be fixed. The criteria for accepting things into the code was being tightened up a lot towards final release, but some new feature work or invasive changes could even still be rushed into the code almost to the last minute. And the pressure was high to "get this in now or wait at least another year until users get it", so even with release drivers tightening possible changes up, some of those could still be argued for. When we shipped the final release to the really larger user audience with more than a year of piled up feature work and fixes, we very soon, usually even directly on release day or the next day, already have a list of quite visible stability problems we needed to get fixed a couple of weeks out in a stability update.

I hope you can see from this description that while we managed to control stability reasonably, the process was far from ideal for providing a product with which we could be happy in terms of stability. So when planning went into improving the processes and becoming more agile and fit for delivering features more quickly than before, a lot of thinking also went into how to make the new process give us a better story of stabilization - and I think the solution holds up pretty well.

"Rapid" Process

So, what we're doing now is getting in feature work and invasive changes into the base code and to Nightly testers almost as before, with only the difference that every such change must have an easy off-switch or be easy enough to reverse the change ("back it out") otherwise. We also still analyze crash data for this and spot major regressions there.
But with going to a next level, there comes the first major change: Every six weeks we're taking a snapshot of this Nightly code and put it on what we now call "Aurora", test it internally, disable things that are absolutely broken (as we have the off-switch/backout possibility) as found by internal QA and send it out to a somewhat larger testing audience. In the next six weeks, we are collecting data from that, reacting to user feedback and crash analysis and bringing in rather small fixes to those problems only or disable further broken features when a fix would be too invasive. We deliver the result daily to that Aurora audience in updates, getting more testing and crash data to analyze, based on the very same snapshot of code, without any more new feature or invasive work to go into it - that continues only on Nightly, no place for that in Aurora.
After those six weeks, this already fixed and stabilized snapshot is going to yet another level, which we call "Beta", and which has even more testers it's being delivered to (while Aurora picks up a new snapshot from Nightly). When the snapshot comes into the Beta phase, we have already put in six weeks of exclusively stabilization and fixing, so it is good enough for what we in earlier times probably would have called a "release candidate". It is as ready as we know at this stage as it can be - but exposing it to an even wider audience, now going into the millions, and which uses it for more normal day-to-day production work, usually turns up another class of potential problems. To deal with those, we could go and disable even more code if needed, and can apply some more small fixes, including of course crash fixes, and we deliver those to Beta testers with roughly weekly updates. Due to this being the first time this code snapshot is being exposed to a public of millions, it's usually the first time we get enough data to see some crash patterns more clearly and can get those fixed. Once again, no new feature or invasive work going into those six weeks of Beta, only disabling of problematic changes, fixing problems found in feedback and of course stability/crashes.
Having spent another six weeks in Beta, twelve weeks or three months of only fixing and stabilizing after taking the snapshot from development, and being OKed by a go/no-go meeting of release drivers, we ship this code to hundreds of millions of users as our next Firefox release (while the other snapshot moves from Aurora to Beta and yet another one is taken from Nightly into Aurora). Of course, we keep analyzing crash reports even from the release users and are able to react to large issues we haven't found before to do a fast fixup release (which we shouldn't need after looking at all the Aurora and Beta data from essentially the same code) and to smaller issues in the next round of Beta etc. before they go to being the next release.

In all this, we always have only six weeks of new development work isolated in every such snapshot (or "version") and not more than a year like previously, so pinpointing a cause gets easier. Then, we less of a rush to get a feature into a specific version as there's another one coming just six weeks earlier, so things will only go into the code in a better thought-out state. Even more, we have switches of some way we can throw to disable problematic code and give developers six more weeks to get it into shape if needed. And over all that, we have roughly three months (twelve weeks) of pure fixing and stabilization period on every snapshot/version to get problems worked out, with different sizes of testing audiences.

Of course, there are still some kinks to be worked out and the transition is not easy for everyone. Next to other concerns we've heard of some people and which belong in different forums than this particular blog entry, we have not scaled up the audiences esp. on Aurora but also on Beta up to what we want yet and therefore are not seeing as much data on them yet as we'd like to (the top crash/hang issue on Beta is typically seen by less than one in every 1000 daily users). So, there are still ways we can and need to improve things here to make it work for stability even better.

Still, having smaller sets of changes per release, no rushed landings of features and built-in calm stabilization periods of that length are all working together to improve stability, in my eyes - as long as people send in their crash reports and we continue to analyze them, of course. ;-)

By KaiRo, at 04:03 | Tags: CrashKill, Firefox, Mozilla, release | 13 comments | TrackBack: 1

September 15th, 2010

Our First "Oilspill"

In previous years, when we had a public security or grave stability problem in a "stable" release and needed to go into fast-pace mode to get update out as fast as possible to fix that, we called that a "firedrill". Still, that name is not very accurate, as it's a real problem case, not just a drill for one.

So, recently, Mozilla land adopted the name "chemspill" for those - needing to clean up after spilling chemicals represent the actual case better for sure.

But then, after recent events, I realized that for maritime life-forms like Sea-Monkeys, it's really more fitting to call it an "oilspill" in our project. Thank goodness, we didn't need that expression for quite some time after I came up with it - until now.

With SeaMonkey 2.0.7, we saw a sudden rise of a previously-rare crash signature in our topcrasher statistics, along with comments from users that it was on launch after updating that this happened, and the affected users were not able to run SeaMonkey at all any more in this version. Now, that's unacceptable, of course, so we stopped issuing updates from 2.0.6 on the release channel and went into investigating the problem.

It turned out that we had both a problem on our side with cleanup after updates as well as a platform problem that affected Firefox and Thunderbird as well, and it looks like both together were even worse for us. We found fixes for both now and decided to take a fix for font face in HTML email signatures along for the ride on creating a fast update that has nothing but those changes in comparison to the 2.0.7 release.

We now have candidate builds of this SeaMonkey 2.0.8 version up for testing and will ship it to the public between today and Friday of this week if everything looks as good as expected with it.

So, after all, we have our first "oilspill" release situation on our hands and I hope we are dealing with it in a satisfactory way. I just wish that real oil spills would be as easy to deal with as those in our software.

By KaiRo, at 16:08 | Tags: chemspill, firedrill, Mozilla, oilspill, release, SeaMonkey, SeaMonkey 2 | 11 comments | TrackBack: 1

June 17th, 2010

2.1a2 Code Freeze On June 22!

In this Tuesday's SeaMonkey Meeting, we discussed our next milestone, SeaMonkey 2.1 Alpha 2, and figured we should create that one sooner rather then later.

Because of that, we decided on the following date:

SeaMonkey 2.1 Alpha 2 code freeze: June 22nd, 2010, 23:59 PDT

Builds will be created as soon after that as no blockers remain, release once we have enough testing on the candidate builds - we hope that will be soon after we freeze and build, hopefully even still in June.

This release will once again be a testing preview of the next upcoming stable SeaMonkey series, and not feature localized builds, as usual in alpha milestones in Mozilla projects.

By KaiRo, at 23:59 | Tags: Mozilla, release, SeaMonkey, SeaMonkey 2.1 | 2 comments | TrackBack: 0

April 20th, 2010

May The 4th Be With You!

The releases wiki page is still stating May 4th as release date for Firefox and SeaMonkey security updates, though those may shift a bit. Also, Thunderbird 3.1 Beta 2 is planned to go out that day - and our next SeaMonkey Status Meeting should also happen that day. But all that is not what this post is about.

In today's SeaMonkey Status Meeting, we discussed if and when we should freeze for and ship SeaMonkey 2.1 Alpha 1 to show people things are moving in our project and to get code out to testers. We collected what changes we already have in trunk since the SeaMonkey 2.0 were cut, and the Mozilla platform has given us a ton of new things alone, ranging from speed improvements (in startup, Js, etc.) via privacy improvements (good-bye :visited sniffing!) to new features on the web content (WOFF, SMIL, and more) and platform/development side (js-ctypes, pymake, etc.) - and then there's mailnews work and already some SeaMonkey-specific improvements as well. In short, a lot of things to show to our testers!

Because of that, we decided to put an important date on our schedules:

SeaMonkey 2.1 Alpha 1 code freeze: May 4th, 2010, 23:59 PDT

We'll start builds as soon as possible after that, reopen the tree for 2.1a2pre development and release 2.1 Alpha 1 once we have enough testing (tentatively May 18th, possibly earlier).

So, if you have any code you want to get out to testers in SeaMonkey 2.1 Alpha 1, make sure it lands within the next two weeks!

(And thanks to InvisibleSmiley for the tagline of this post!)

By KaiRo, at 21:48 | Tags: Mozilla, release, SeaMonkey, SeaMonkey 2.1 | 2 comments | TrackBack: 2

November 4th, 2009

SeaMonkey 2.0 in the press

Here's a bit of an overview of press articles we got for the SeaMonkey 2.0 release. The English and German ones were collected by myself (we were pretty well-featured in German media), the French and Czech ones come from our respective localizers, Cédric and Pavel.
I've hidden two "easter eggs" in the English list that could help some of our users, by the way - a forum thread for ubuntu users (things are easier for some other Linux distros - openSUSE offers it in the build service and in upcoming openSUSE 11.2, and upcoming Fedora 12 also has SeaMonkey 2.0) and a link to the portable version that is available now. ;-)





Oh, and here's a scanned image from the German IT magazine - c't (Edition of October 26th, page 52):
Image No. 22083
It's about RC2, but still cool to be present in that magazine... ;-)

By KaiRo, at 15:47 | Tags: Mozilla, press, release, SeaMonkey, SeaMonkey 2 | 5 comments | TrackBack: 0

October 27th, 2009

SeaMonkey 2.0 - The Modern Internet Suite is Here!

The SeaMonkey project at Mozilla is excited to release its completely refurbished next generation of the all-in one Internet suite today: SeaMonkey 2.0, now available for free download, melds the ideas behind Netscape Communicator with the modern platform of Firefox 3.5 to create one of the most compelling open source products for advanced Internet users.

The combination of an Internet browser, email & newsgroup client, HTML editor, IRC chat and web development tools, that has already established a wide user base in its previous incarnations, has been rebuilt on top of the modern Mozilla platform, featuring world-class add-on management among other things. In addition, it has been improved with feed support (including an RSS and Atom feed reader in the mail component), a modern look, restoration of browser tabs and windows after crashes or restarts, tabbed mail, automated updates, smart history search from the location bar, faster JavaScript, HTML5 features (for example video and downloadable fonts), and even support for the Lightning calendar add-on (which will issue a beta for installation on SeaMonkey 2.0 in the next few weeks).

The Release Notes feature more in-depth lists of the improvements and known issues with the new version as well as installation requirements and instructions. Find even more information on SeaMonkey 2.0 and the SeaMonkey project at!

By KaiRo, at 11:37 | Tags: Mozilla, release, SeaMonkey, SeaMonkey 2 | 5 comments | TrackBack: 0

October 24th, 2009

SeaMonkey 2 Party and Release Event - In Vienna!

Some of you might remember that I was planning for an OpenWebCamp in Vienna - that was planned for October 27 but it was downgraded seriously due to not having a real Mozilla community around here yet and me not being able to organize the event myself when driving a major release at the same time. Still, the guys from quintessenz planned to have us on board for a track of the Cyber Liberties Conference, and so I did sit down with them for a while this week to work out what we actually can get into this track (speakers still wanted, if you can come around to talk on open web topics in Vienna on short notice, please tell me, travel costs can probably be reimbursed).

In any case, I told them that we're planning to release SeaMonkey 2.0 final right on that day, and the guys were suddenly cheering! We could do an official release event right there, they claimed, we could add this into press releases - and, above all, this could finally the topic they have been looking for as a label for the party on Monday!

In the end, what we decided there is the following:
  • October 26, 20:00: q/Treff Spezial @ q/uintessenz - Museumsquartier Wien: Seamonkey 2.0 Release Party
  • October 27, Open Web Camp/Track @ Cyber Liberties Conference, with a number of talks on Mozilla and the open web - and with an official SeaMonkey 2.0 Release Event

If anyone reading this is from or near Vienna, I'd be very happy to meet you at any of those events next week!

By KaiRo, at 02:21 | Tags: Mozilla, OpenWebCamp, party, release, SeaMonkey, SeaMonkey 2 | no comments | TrackBack: 0

July 22nd, 2009

The Beta Has Landed

Community - SeaMonkey Base here. The Beta has landed.

After somewhat more time than we had expected earlier, we finally have a great SeaMonkey 2.0 Beta 1 release available for you. This release really brings the suite up to modern standards, supporting feeds as well as HTML5 audio/video, managing add-ons in a comfortable way (as known from Firefox 3.5), having customizable toolbars in both browser and mail windows, and much more - and all that in more than a dozen languages right from the beginning.

In many ways this beta is much better than our stable 1.1.x releases, and it's probably not even less stable from most points of view, but it still has some corners that need smoothing and some features we want to add before calling it a final 2.0 release.

Pushing it out to the public today was one small step for a release engineer, but one giant leap for the project. One after after celebrating the 40th anniversary of the first human setting foot on a different celestial body, i.e. the Apollo 11 mission to the moon, this is our mission of exploration, boldly moving the well-known Internet suite where it has never been before - into a state where it fully fit for the challenges the Internet and web sets for us today and laying the base of where it can be tomorrow.

To echo what Buzz Aldrin said two days ago when asked if we can manage the challenges to go the way even further and step further out - in the case of space exploration, that's bringing humans to the Mars and beyond, in the case of SeaMonkey it's a 2.0 release and beyond - I fully agree with him paraphrasing a nowadays almost overused sentence:

Yes, we can.

(And you can join us in that task by testing this beta, reporting bugs, and if you're really bold, exploring possible improvements and writing patches. Yes, you can.)

By KaiRo, at 03:31 | Tags: Mozilla, release, SeaMonkey, SeaMonkey 2 | 3 comments | TrackBack: 1

July 1st, 2009

Schedule for SeaMonkey 2.0 Beta 1

Thunderbird came around with setting up a schedule just in time, so at yesterday's SeaMonkey Status Meeting, we agreed on adopting Thunderbird 3.0b3 dates as the SeaMonkey 2.0 Beta 1 schedule as well (times of 23:59 US/Pacific, as usual):
  • Slushy String freeze date: 2009-07-02 (Thur)
  • Slushy Code Freeze date: 2009-07-07 (Tues)
  • Firm String / Code freeze date: 2009-07-14 (Tues)
  • l10n-mozilla-1.9.1 freeze date: 2009-07-16 (Thur)?*
  • Target Ship date: 2009-07-21 (Tues)
This is the first time we have string freezes for SeaMonkey, so let me give you some explanation:

After the slushy string freeze, string changes should be avoided where possible, and those needed must get approval-seamonkey2.0b1+ before being checked in.

After the slushy code freeze, the same is true for any code changes, though blocking-seamonkey2.b1+ bugs without string changes can go in without further approval (blocking+ serves as approval).

The firm freeze should be the cutoff for any changes at all, unless there are blockers we still need to fix.

* I'm not completely sure about the freeze for L10n, we might not need that at all, as we'll likely do the same opt-in process as Firefox did recently and so might be able to just take any L10n changes up to the time when we start the builds. This is the first time we're doing that, so please excuse roughness in the process.

By KaiRo, at 13:52 | Tags: Mozilla, release, SeaMonkey, SeaMonkey 2 | no comments | TrackBack: 1

Feeds: RSS/Atom