Josh On Design

In today's post I'll dive into Amino's new buffering support. At then end we'll talk about new API docs for Amino, the roadmap, and request for help on a domain name.

A big part of making a scenegraph fast is using lots of little tricks to do as little work as possible. Most of these tricks are decades old, but they still work. What makes a scenegraph good is letting developers easily use these tricks without having to code up anything special.

Dirty rect tracking is one such trick, but I haven't implemented it in Amino yet so I'll cover that in a few weeks. Another common trick is using intermediate buffers to store effects that are expensive to compute, such as blurs, shadows, and Photoshop style adjustments. The beauty of buffers is that if you can pretty much do any crazy thing you can think of, as long as you can figure out how to draw it into a buffer first. A good third of Swing Hacks, the book Chris Adamson and I wrote, is just different clever ways of using buffers.

Given the importance of buffers I made this a central feature of Amino. But before I go any further, how about a demo!

Zoom Effect

First, an MP3 style Visualizer. I say MP3 style because it's not actually working with audio. I'm just generating random data then pushing it through a simple buffer effect: draw into buffer1, copy buffer1 into buffer2 with stretching, flip buffers and repeat. It's a simple technique but if you do it over and over the results are very cool.

MP3 Visualizer. Click to view

On a modern browser you should get a solid 60fps. BTW, a quick shout out to Internet Explorer 9. The guys at MS have done a top notch job. Amino runs beautifully there, no matter what I threw at it.

The Buffer API

Buffering is broken up into two parts. First is the actual Buffer object, which is a bitmap with a fixed width and height that you can draw into and can be drawn into other buffers or the screen. In the Java version of Amino this is a wrapper around BufferedImage. In the JavaScript version Amino creates an offscreen canvas object to use as a buffer.

The simplest use for a Buffer is in the BufferNode, which just draws its only child into the internal buffer for safe keeping. If the child hasn't changed on the next drawing pass then it will draw using the buffer instead. This is the simplest use case, but very important because you can draw a bunch of complex stuff and save it by simply wrapping it in a buffer. Here's a quick example:

//create a group with 100 child rects
var g = new Group();
for(var i=0; i<1000; i++) { g.add(new Rect().set(i,i,50,50));
} //wrap the group in a BufferNode
var b = new BufferNode(g);
runner.setRoot(b);

The code above creates a group with a thousand rectangles. This will probably be slow to draw, but by wrapping it in a buffer it's only drawn once and then saved for later. Now the rest of your scene can draw at full speed.

Real Time Photo Adjustments

The next big use of buffers is for special effects like Photoshop filters. As of today Amino now has effects for blur, shadow, and photo adjustment (saturation, brightness, contrast). Each of these effects uses one or more buffers internally to manipulate the pixels before drawing to the screen. Blurring is a big topic, so I'll cover that in it's own blog later. Today I'll cover the photo adjustment.

Adjusting the saturation, brightness, and contrast of an image is actually pretty simple. It's just basic math and a lot of copying:

• create two buffers.
• draw your photo into the buffer 1
• loop over every pixel in buffer 1
• pull out the red, green, and blue components of that pixel's color.
• calculate a new red, green, and blue using some math
• Set the same pixel in buffer 2 using the new calculated color

For brightness, saturation, and contrast the equations are:

new color = old color + brightness new color = (old color - 0.5)*contrast + 0.5 //desaturation new color = old.red*0.21 + old.green*0.71 + old.blue*0.07

I won't bore you with the details of actually extracting the components with hex math and stuffing it back into the new pixels (well, maybe I will in a later blog on canvas performance). The point is you can do lots of effects with simple math.

The challenge with code like above is that it still may be too slow for real time work. Remember, the goal of Amino is a rock solid 60fps on a desktop browser and 30fps on a mobile device. To keep our framerate promise we need a way to do some work without blocking the UI. That means background threads.

Background Threads, Sorta

On the Java side we can use real background threads to do compute intensive effects; though honestly Java is fast enough for most cases that it hasn't been necessary yet. Canvas is most browsers is slow enough that we can't process an entire large picture (say 512x512) in a single frame. Unfortunately, JavaScript doesn't really have threads, or at least not until the forthcoming Worker API is released. So on to our backup plan: cheat.

We are allowed to do some work on the GUI thread as long as we don't take too much time. The solution: break up the work into small chunks and distribute them across multiple frames. It won't be quite as 'realtime' but it still allows us to do these effects in browser without slowing down the UI.

Amino now has an internal class called WorkTile which defines a subset of the full image to be processed. Right now it's set to 32x32 pixels. Once the effect starts it will process one WorkTile at a time until it runs out of time for this frame (currently set to 1000/40 ms). When the next frame arrives it will draw a few more WorkTiles until it runs out of time. After enough frames the image will be completely processed and saved into the final buffer, and work is terminated. Volia, 'background' processing of images in a browser without blocking the UI.

Currently only BackgroundSaturationNode uses this new worker system but eventually all effects will use it. Here's a demo to change the saturation, brightness, and contrast of a satellite image from Venus. Click the image to go to the demo page.

Photo Adjustment. Click to run.

API Docs, Website, and Roadmap

Along with the buffering work I've recently written a new doc tool for Amino. I wanted something that would work on both Java and JavaScript code and didn't have the ugly legacy of classic javadoc. It's still a work in progress but I'm happy with the results so far. It's a new design where classes are grouped by category instead of package. Feedback is very welcome.

• Java API Docs
• JavaScript API Docs

I think Amino is getting close to a real beta release. It's pretty stable in the major browsers and on mobile devices that support canvas (I haven't tested Android yet, but TouchPad and iPad work great). I have a bit more work to do on events, fills, and animation but once those are done we'll be ready for a 1.0 release.

Now the big question: where to put all of this? I think Amino deserves it's own domain so I'd like your help picking one. Please tweet your suggestions to @joshmarinacci.

That's it for this week. Thanks guys. I think Amino is shaping up be a rockin' scenegraph library.