Flash on Android 4.4 KitKat

There has been some some talk recently about the Flash situation on Android 4.4. While it's no secret that Adobe discontinued support for Flash on Android a while back, there are still a lot of folks using it on a daily basis. The Firefox for Android team consistently gets feedback about it, so it didn't take long to find out that things were amiss on KitKat.

I looked into the problem a few weeks ago in bug 935676, and found that some reserved functions were made virtual, breaking binary compatibility. I initially wanted to find a workaround that involved injecting the missing symbols, but that seems to be a bit of a dead end. I ended up making things work by unpacking the Flash APK with apktool, and modifying libflashplayer.so with a hex editor to replace the references to the missing symbols with something else. The functions in question aren't actually being called, so changing them to anything that exists works (I think I used pipe). It was necessary to pad each field with null characters to keep the size of the symbol table unchanged. After that I just repacked with apktool, installed, and everything seemed to work.

There is apparently an APK floating around that makes Flash work in other browsers on KitKat, but not Firefox. The above solution should allow someone to make an APK that works everywhere, so give it a shot if you are so inclined. I am not going to publish my own APK because of reasons.


Using direct textures on Android

I've been working at Mozilla on Firefox Mobile for a few months now. One of the goals of the new native UI is to have liquid smooth scrolling and panning at all times. Unsurprisingly, we do this by drawing into an OpenGL texture and moving it around on the screen. This is pretty fast until you run out of content in the texture and need to update it. Gecko runs in a separate thread and can draw to a buffer there without blocking us, but uploading that data into the texture is where problems arise. Right now we use just one very large texture (usually 2048x2048), and glTexSubImage2D can take anywhere from 25ms to 60ms. Given that our target is 60fps, we have about 16ms to draw a frame. This means we're guaranteed to miss at least one frame every time we upload, but likely more than that. What we need is a way of uploading texture data asynchronously (and preferably quicker). This is where direct textures can help.

If you haven't read Dianne Hackborn's recent posts on the Android graphics stack, you're missing out (part 1, part 2). The window compositing system she describes (called SurfaceFlinger) is particularly interesting because it is close to the problem we have in Firefox. One of the pieces Android uses to to draw windows is the gralloc module. As you may have guessed, gralloc is short for 'graphics alloc'. You can see the short and simple API for it here. Android has a wrapper class that encapsulates access to this called GraphicBuffer. It has an even nicer API, found here. Usage is very straightforward. Simply create the GraphicBuffer with whatever size and pixel format you need, lock it, write your bits, and unlock. One of the major wins here is that you can use the GraphicBuffer instance from any thread. So not only does this reduce a copy of your image, but it also means you can upload it without blocking the rendering loop!

To get it on the screen using OpenGL, you can create an EGLImageKHR from the GraphicBuffer and bind it to a texture:

#define EGL_NATIVE_BUFFER_ANDROID 0x3140
#define EGL_IMAGE_PRESERVED_KHR   0x30D2

GraphicBuffer* buffer = new GraphicBuffer(1024, 1024, PIXEL_FORMAT_RGB_565,
                                          GraphicBuffer::USAGE_SW_WRITE_OFTEN |
                                          GraphicBuffer::USAGE_HW_TEXTURE);

unsigned char* bits = NULL;
buffer->lock(GraphicBuffer::USAGE_SW_WRITE_OFTEN, (void**)&bits);

// Write bitmap data into 'bits' here

buffer->unlock();

// Create the EGLImageKHR from the native buffer
EGLint eglImgAttrs[] = { EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE, EGL_NONE };
EGLImageKHR img = eglCreateImageKHR(eglGetDisplay(EGL_DEFAULT_DISPLAY), EGL_NO_CONTEXT,
                                    EGL_NATIVE_BUFFER_ANDROID,
                                    (EGLClientBuffer)buffer->getNativeBuffer(),
                                    eglImgAttrs);

// Create GL texture, bind to GL_TEXTURE_2D, etc.

// Attach the EGLImage to whatever texture is bound to GL_TEXTURE_2D
glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, img);

The resulting texture can be used as a regular one, with one caveat. Whenever you manipulate pixel data, the changes will be reflected on the screen immediately after unlock. You probably want to double buffer in order to avoid problems here.

If you've ever used the Android NDK, it won't be surprising that GraphicBuffer (or anything similar) doesn't exist there. In order to use any of this in your app you'll need to resort to dlopen hacks. It's a pretty depressing situation. Google uses this all over the OS, but doesn't seem to think that apps need a high performance API. But wait, it gets worse. Even after jumping through these hoops, some gralloc drivers don't allow regular apps to play ball. So far, testing indicates that this is the case on Adreno and Mali GPUs. Thankfully, PowerVR and Tegra allow it, which covers a fair number of devices.

With any luck, I'll land the patches that use this in Firefox Mobile today. The result should be a much smoother panning and zooming experience on devices where gralloc is allowed to work.


Android = Linux-on-anything-with-a-screen

Here we are in 2011. Obviously, the year of the Linux desktop, right? You'd hope so. It's not like there hasn't been plenty of time to make it happen. Red Hat, Canonical, Novell, and many other companies and individuals have been working hard on it for quite a while. And yet, very few inroads have been made. Sure, Dell has been selling Ubuntu on some netbooks with limited success, but that is hardly a victory.

In 2007, Google unveiled the Linux-based Android smartphone platform. The first phone wasn't released until a year later in 2008, along with the Android source code. Now, in Q4 2010, Android is the best-selling smartphone operating system in the world, surpassing Symbian, BlackBerry, Windows Mobile, and yes even iOS. In 2 years, it went from zero to total domination, with no signs of slowing down. Not everyone agrees that Android is the best smartphone OS (ahem, Gruber), but it's impossible to say that it isn't a success. And it uses Linux. In fact, I believe you could say that Android is the most successful and widely deployed Linux product ever. So why haven't we standardized on it for other non-smartphone uses? It would require some work, but at least there is some chance it could work out.

Initially I was going to try to call out all of the infuriating things about developing on desktop Linux, but I got tired. All of that has been said elsewhere by people smarter than me. The main reason I'm really behind Android is that it dramatically reduces the amount of fragmentation. It's just not possible for a hardware vendor like AMD or NVIDIA to support the multitude of distros out there. If you told those guys they only had to support a couple versions of Android, they would be able to deliver much higher quality of support.

The Eclipse IDE, Dalvik VM, and Android application framework are all great assets and deliver a far superior development experience over standard Linux. You get that for free. If Java isn't your bag, though, you can always use the Native Development Kit to write in C, C++, C#, or whatever you want. I would love to see Clutter on Android, for instance. And Node.js/gjs. Or any number of other great projects.

It's already possible to run Android on a PC via the Android-x86 project. They've added basic mouse support, as well as hardware accelerated OpenGL ES2 via Mesa (on Intel hardware). I have a Cr-48 running it here, and it works amazingly well. Even wifi works.


Yeah, I'm a Rails fanboy now

A lot of my day job now involves working with Ruby on Rails. At first I wasn’t sure how much I would like Rails or ruby, given that I had been doing a lot of C#/C/whatever desktop work before. Not surprisingly, though, I’ve become quite addicted. The test-driven nature of development is a welcome change — most desktop apps I worked on didn’t even have tests. The Rails community has done a great job of banging automated testing into people’s heads. Almost every tutorial, book, or random blog post I’ve seen emphasizes the importance of good automated tests. Hopefully it has helped decrease the instances of ‘snorpage’, but perhaps my co-workers would disagree :)

Anyway, I love Rails so much that I’ve converted my blog from Wordpress to Enki. Enki is more of a create-your-own-blog construction kit than a turn-key solution like Wordpress. That was one of the main reasons I chose it over Typo or Mephisto — I wanted to be able to easily hack on it.

I wrote a quick and dirty script to help me import the Wordpress posts into Enki. Any fellow Enki hackers can grab it here.


Fun with Studio

I’ve been working on a new project now for a while called SUSE Studio. Essentially it is a web interface which allows you to build your own customized version of SUSE. You can select packages, do some configuration, and even add your own branding.

I created a media center appliance to see how hard it would be. The appliance is based on openSUSE 11.1, and boots right into the excellent Elisa Media Center. You can download the image here. The tarball contains one file, which you can ‘dd’ to a USB storage device. We’re working on writing a small application to make this part easier.

On the first boot it will do some one-time setup like repartition and resize the disk, install NVIDIA or ATI video drivers (if appropriate), and setup X.


Download Elisa Media Center Appliance