The roads I take...

I've always been a huge fan of the idea of rendering (browser) UI with the web rendering engine and write even the UI itself in a "webby" language. This idea AFAIK came up at Netscape somewhere in 1998 when the new Gecko engine for Mozilla was built. Unfortunately, HTML wasn't in a shape back then to be used for writing complex UI, and actually was quite far away from doing that 15 years ago. The basic concept looked good though, and so an intermittent technology was created to build a UI language on similar concepts, using as much of the JavaScript and CSS from HTML as possible and a markup document written in something like "HTML for UI", which ultimately became XUL ("XML UI Language").

Firefox on desktop, as well as Thunderbird, SeaMonkey, BlueGriffon and others are written using XUL for all their UI, and it's working great there. Actually, this is what even lured me into contributing to Mozilla in the first place - I saw that the UI was written in a way that I could understand back then, as someone who had played around with writing HTML, including some CSS and tiny bits of JS. I felt right at home when I saw that a button in the UI was a <button> in the markup, and that markup followed basically the same, albeit stricter (due to being XML), rules as the HTML I knew. Sure, the tag names were a bit different, but it's UI, and they were easy to understand.
Of course, another import aspect was that this UI would work on any platform you could build Gecko for, be it Windows, Mac and Linux, or even OS/2, Solaris, BeOS and others (including exotics such as AmigaOS). Of course, to fit really in with the host OSes, a number of specific tweaks were added esp. for the mainstream ones, but it works without them, just doesn't integrate as well. As soon as you can compile the Mozilla engine (which takes enough effort anyhow), you also get the full UI, which is a nice deal.
And, of course, the extension system we built over the years has largely been based upon the concepts of XUL, but I won't go into depth on this right here.

Even on mobile devices, Mozilla used that concept for a while. That was great for portability, as you had a working browser with all UI once you could get it to compile somewhere. Hell, I even ran SeaMonkey on a Nokia N810 (a 4.1" Internet tablet) - with the full UI! Of course, that UI was way too small and too overloaded for use on a touch screen, but, being XUL, it loaded and could be used if you managed to tap the correct points with the stylus.
So, for portability, including getting it to run on new devices, this XUL UI was great - and of course, some XUL UI was created that would work decently on those small touch-screen devices. I like the concepts and UI of our modern mobile Firefox browsers (both look similar, see below) better, but we had a usable, easily hackable, nicely extendable UI built with XUL. When Android became a larger deal on mobile, we could just use that and make it work on this system as well, and had something usable pretty fast.



All would have been perfect if it wasn't for one problem: If you render all your UI with the web renderer, it means that you need to load up this rendering engine in its full glory before you can paint any UI at all. And with the web becoming more powerful, what needs to be loaded for that became pretty large, and with loading from permanent storage into memory being pretty slow on those weakly powered mobile devices, it meant waiting times of multiple seconds (on some devices 15 seconds and more) - while the same "smartphone" devices were propagating more and more the concept of instantly launching apps.
Back in 1998, waiting several seconds for an application to start was common and it was OK for Netscape or Mozilla browsers to do the same, maybe display a "splash screen" while the user was waiting for that. Now, in the fast-living world of smart phones, the usage patterns as well as the expectations have changed enough that this waiting time is what instantly (no pun intended) your app is being shot down by users for that, and nobody but your strong supporters (which Mozilla fortunately has) will use it. Things needed to change. And they did.

Basically, there's two ways Mozilla had to make the browser UI start fast: Either write it in a way that it used the "native" toolkit of the hosting OS and not depend on the rendering engine to launch (so that can be loaded lazily in the background), or to have that "native" toolkit of the system be our rendering engine already!
Well, as funny as it sounds, we ended up doing both!

On Android, we did the former, write a "native" (i.e. Java) Android app, which is very fast to be displayed as all the Java/Dalvik framework is already loaded, and load Gecko in the background once the UI is up - while the user interacts with the UI to e.g. enter a site to call up, we have enough time for that so Gecko is ready to display the actual websites once it's needed.
Unfortunately, that "native" UI is not "webby" any more, Java is very different from HTML/XUL/CSS - though AFAIK we are using quite a bit of JS to driver things, so there's at least some pieces left that someone like me would still understand. Oh, and as my job nowadays is stability, Java exceptions crash the browser, while we don't have that problem with JavaScript, and the amount of crash reports rose up significantly with that UI rewrite (the team is doing a great job on fixing them, though, and we've become actually pretty good in stability there nowadays). To be fair, we also added support for the Flash plugin, which seems to be causing by far the most stability problems with this version. All that said, this new Firefox for Android is really fast, esp. on startup, and works incredibly well, it's getting cheered as the best browser for Android by many - compare that with the quite bad reputation that the XUL-based version had and you need to admit this was a good change - even if you happen to be me.

Well, and then there's this thing that I mentioned with the "native" toolkit of the system be our rendering engine, and then you can use web technologies for the mobile browser UI, right? That's what we're doing in Firefox OS!
On that system, the whole UI of everything you see on screen is rendered by Gecko. And not just that, Mozilla took the final step and didn't even do it with XUL, but used plain HTML this time, so that everything running on this system is made of pure web apps (sure, with some new WebAPIs thrown in). All that said, there's some things in UI design where HTML still needs to catch up to XUL, but those mobile apps are working really nicely already (still, mobile has less UI with less complexity and less need for cross-app consistency, so the particular weak spots I'm thinking of don't come to light that much).
What it comes down to for what I want to say here is that in Firefox OS, we indeed have a browser on mobile which has its UI completely rendered by the web rendering engine (again) - but this time, not done in XUL but in plain HTML!

And it looks pretty good, compare our "native" Android UI to our Firefox OS UI and you'll see that there are some similarities:



And, go ahead and try yourself, e.g. in the simulator, it even pretty much works the same!

Time to come to the reason I actually brought this up today: You might ask what happened with the XUL-based UI for mobile, which I said was nicely portable to different devices (and older builds of which I still have in use on my "real-Linux-powered" N900 and N9 phones)? Well, it's dead and gone. And even its source has been removed today from our mozilla-central repository. So, if you want a browser with a "webby" UI on mobile, your only chance from now on is Firefox OS. (That is, if nobody comes along and resurrects a XUL UI for alternative mobile platforms in some external project.)

All that said, I'm excited that the original idea of rendering the UI with the browser engine survives on mobile - and actually thrives and is being hugely extended to powering the whole system, in Firefox OS!

I have mentioned in recent weeks that the touchscreen on my N810 stopped working (I could track it down to a problem with the ribbon coming out of the screen, right adjacent to its plug to the main ribbon, but that's nothing I can really fix) and so I have been looking for getting some kind of replacement for the usual tasks I performed on that device - mainly using ssh to get into other machines, do GPS tracking/mapping and accessing the web.

Now, I value the openness of the Maemo OS2008 system on the N810, as it's a full Linux that easily lets you have root access and install whatever you want on the device - even alternate operating systems if you wish to do so. Compared to that, even Android is a closed system (not to mention iOS or WinMobile). The Nokia N900 already came out in 2009 with the next version of the OS, Maemo 5, but that was some time ago, and meanwhile MeeGo had been announced as Maemo's successor (with the same level of openness) and new devices with that were originally expected for late 2010, so I had been waiting for those to jump to the new generation immediately, being eager to test MeeGo and Firefox 4 Mobile (which isn't available for the N810 any more).

Well, with the N810 problems and the delays in getting MeeGo handset versions ready and therefore the delay in the new devices becoming available, I finally settled for trying to get a used N900 from eBay - and I finally did win an auction for one at just 177 Euros. Not fully trusting private-to-private remote transactions, I didn't really want to believe it though until I finally had it in my hands a few days ago.

At first, I was a bit weary to do too much on Maemo 5 (and even more to use it as a phone), as I'd really like to go testing MeeGo, which has images for N900 available, but I figured I first should see how well the device itself works with the system it comes with. I updated the system to its latest release (which made it go into some strange start-install-and-reboot loop and had me actually flash it with the new version) and look for the application I liked to install, Mappero and Firefox being among them. And I even tried to get over the data from both the N810 and my Symbian phone, which worked fairly well (though getting SMS messages over required some fiddling with community-provided tools).

Once I tried and put the SIM card from my phone over into the N900, I realized I really can use the new device for that purpose as well - even though a few smaller improvements of the contact app would be helpful. Still, it works better than I expected. Even Firefox 4 Mobile (beta) works well - it's not very fast at startup but I expected it to be worse, and it performs reasonably well even with multiple "tabs" for being a full browser with all web-facing features that desktop Firefox 4 has.

All in all, for being more than a year old technology, this device still holds up better in comparison to the current generation that I would have expected, and it completely grew on me even in the just two days since I put it into full use. OK, it has some slightly rough edges, but I'm both used to doing beta testing (which this doesn't feel like, actually) and using very open Linux-based systems.

Oh, and I figured that while playing (Open!) Dune II is not that comfortable with a touch screen (but fun on a mobile device) it's absolutely awesome to play tuxrace with the accelerometer controls!

Back in 2007, I was wondering if a mobile device that fits me will ever be built and saw the unveiling of the N810 as a step nearer to that. It was a really open, hackable mobile device, with a Mozilla-based browser, but it lacked phone capabilities, so even when I bought one a few months later, I still needed to carry a phone around with me in addition to this surely nice device.

What I really wanted after all, was a smartphone, albeit one that was truly open and hackable at least on the side of software, reasonably fast, with a Mozilla-based browser that shows the web as it should be and a touch screen in a handy size - oh, did I say it should be a phone at the same time? Would it ever exist? I trusted it would, I was just thinking ahead of the time.

The OpenMoko was a phone that sounded interesting, but it never came to a state where it would have few enough problems to make the mass-market. When Google came up with Linux-based Android, it sounded good, but the G1 and friends are nothing like an open, hackable device. It goes without saying the the iPhone isn't, even though it looks cool and seems to be pretty functional.

Meanwhile, having the N810, I grew quite fond of the nicely sized touch screen, but even more of the maemo distribution, and of many things like maemo-mapper, which is my trusted OpenStreetMap-based travel companion nowadays. And I closely am following the maemo community through blogs, etc.

So, it was no big surprise to me when Nokia unveiled the N900, powered by maemo 5 this week. I've read enough rumors about this "RX-51" or "Rover" prototype device, esp. in recent weeks but even vague comments before that.

And still, it's a bit overwhelming for an open source guy like me. This is actually one of the largest mobile phone producers worldwide shipping a smartphone (yes, a phone!) based on Linux, with a fully hackable software stack and, importantly of us, a Mozilla-based browser! Shipping, as in pushing it out as their top-of-the-line product for consumers! Not just a geek thing like the OpenMoko, a normal consumer device that at the same time speeds up the heartbeat of any open source geek!

This is a bold step for Nokia, and I really hope it will be successful - this could be an important point in getting open software out to consumers, possibly even as large an impact as Firefox on the desktop. Thanks to Nokia for taking this step. This week, I'm really proud to be a Nokia - and maemo - user.

I'm not sure if the N900 is fully the device that fits me in every regard, I fear the screen might come to be a bit small of the things I've grow accustomed to (maemo-mapper, browsing, showing my pictures to people, even doing some programming), and I also fear that the smaller keyboard compared to the N810 makes typing even harder, but in any case, the N900 is another large step to the device I want. I think the fact that it's less a niche product but a general consumer product (even as a high-end smartphone) is overall the more significant matter than if it fits my requirements perfectly.

This week, I found some more time to hack around on my fun project to do a Mandelbrot set fractal renderer in XULRunner - some of that fun hackery done on the N810, some on my desktop, finally some on my laptop, and now those sources create a really usable application.
The only thing holding me back from doing a downloadable "(pre)release" package of any kind is that it's still somewhat hard to do that for XUL/JS-only XULRunner apps. I'd need to compile my own XULRunner and package it with that or such to get a usable downloadable thing.



What I added this week to bring it over the top was a zoom function that works by dragging the mouse over any piece of a rendered image and a possibility to bookmark locations and call them up again at a later point. With that, one can easily navigate to different places and get pictures like the one above - my laptop calculated and rendered the original 1280x1024 picture in a matter of seconds, with a 1.9.1 XULRunner from some time this weekend, so the same TraceMonkey present in Firefox 3.5.2 to calculate all those iterations, and I did save the contents of the canvas used to paint it as a PNG with the functionality I have in the application. Any quality degradation comes from scaling and converting to JPEG on my web server.

Oh, and there's one more interesting picture I did get out of this work:


(larger sizes linked, original 1280x1024 desktop-wallpaper-ready version available on personal request)

All the fractal pixels in this image come from TraceMonkey's calculation again, of course, but the GIMP helped slightly to finish off this one.

Yes, after the Thunderbird story, I couldn't resist:




This is SeaMonkey running on my N810 - more pics in my N810 gallery.

Some notes:

• Build identifier: Mozilla/5.0 (X11; U; Linux armv6l; en-US; rv:1.9.1a2pre) Gecko/2008072523 SeaMonkey/2.0a1pre
• not the ideal UI in terms of OS integration, but seemingly fully working
• some prefs are cut off, etc. but I guess that's expected
• I call that SeaMaeMonkey
• scrollbars on XUL trees don't work as expected, which makes DOMi slightly unusable
• User:KaiRo:SeaMaeMonkey on wikimo has some good links for building if anyone else wants to try
• This is just for fun - no mobile releases of SeaMonkey are planned!

Nice to know that it's doable!

Justin Dolske just posted he got Thunderbird on maemo now!

I just love a mobile platform for which one can (almost) easily build a lot of apps you want. Even though it lacks a decent mail program and calendaring currently, we might just be able to get that going with our stack of Mozilla software. Isn't that nice? Try that with your iPhone!

Why do I suddenly feel a certain urge to set up this scratchbox thing myself and try to build SeaMonkey for maemo for trying on my N810?

This Monday, my new Nokia N810 arrived! I'm played a bit with its features now, installed some additional software, tried browsing a number of websites, and all in all, I really like it.



You (as a geek yourself, probably) might notice that on the picture, its UI has a lot of similarity to Star Trek TNG's mobile tablets (PADDs), that's not the default look - one of the first things I installed was the LCARS PADD theme package, when I tried how well free WLAN hotspots work in Vienna - and I was satisfied (both with freewave and the theme).



Of course, for people like us, the web browser is quite important - and the device running on Linux and having a Gecko-based browser were among my main reasons for buying the N810. And, despite not being SeaMonkey , I like the microB browser a lot. It works like a real browser and displays web pages like they should look, not garbling a number of them like the Opera mobile version on my Nokia 7710 phone (which performs well enough for the N810's GPRS internet connection where no WLAN is in reach). I even did write a paragraph on the last weekly status update on this blog from the microB browser, when my desktop locked up and I had to reboot it - something that I haven't seen with the N810 yet.
The system feels very stable and comfortable, and you wouldn't realize it's actually Linux unless you e.g. open up an X terminal on it or connect to the device via ssh (after installing OpenSSH, that is). Oh, and as a regular long-time Linux user, I just like being able to look into top or ps to see what the system's doing/running and just ssh'ing into all my computers from all my computers.
Now I finally got a device I can take with me to take notes, look into the web and such stuff without needing to bring with me and unfold a fully laptop computer. Yay!



One further reason why I finally decided to get me an N810 is its GPS and mapping capability. Especially when I was in the US in April, but eventually also when on my way somewhere here in Vienna, I often felt it would be nice to have a good map handy, ideally along with some indicator pointing out my position. The N810 promised to be able to do both, with software being able to use free OpenStreetMap data (maemo-mapper), and even GPS positioning and tracking capability. I already read in advance to buying the device that its internal GPS takes a very long time to find a satellite fix, but then work quite well even where signal quality is not that good - and I can confirm both, though esp. the former. When turning on display of GPS details in maemo-mapper, it's quite common for the device to have 2-5 satellites in view but being unable to establish a fix - at least in the central area of Vienna that is covered with 6-7 story buildings. Once the miracle happens and an initial fix can be established, the position isn't always accurate due to heavy signal reflection between those buildings, but it hardly loses the fix and can re-establish it fast when it gets lost, at least when not being in a building for a longer period of time.
I'm looking forward to trying this outside the city (in my home town) this weekend and see hoe well things work there. If anybody knows a good trick to ease the device getting an initial GPS fix, please tell me, I'll happily try out your suggestions.

All in all, I'm positively impressed by this device, it works very well and perfectly shows off how well Linux and Gecko 1.9 are suited for mobile use already - but then, I barely scratched the surface of what it can do in those two days I have it now. I'm looking forward to the N810 supporting my work and perhaps even some fun activities in the future. Well done, Nokia!