When redesigning Amino I had a few core goals. These goals are in place to guide the product and ensure we created something genuinely useful, and not become "yet another gfx lib".

Goals

Amino 2 must be:

  • Simple: the api must be simple and easy to understand
  • Responsive: the goal is to hit a consistent fps for all graphics on screen. 60fps on desktop and 30fps on mobile. This should be very doable with modern devices. The UI must always be responsive, even at the cost of accuracy or graphics complexity.
  • Rich: complex effects and animation should be possible(shadows, gradients, real world textures, animation) while still hitting a consistent FPS.
  • Speedable: we should be able to add speed improvements, including hardware acceleration, purely in the implementation. Speedups shouldn't affect the API. Existing apps just get faster over time.
  • Subsettable: You should be able to use just the parts you need.
  • Portable: Nothing of the underlying graphics implementation should be exposed. (we aren't there yet).
  • Flexible: for lots of tasks you can just compose nodes together, but sometimes you may want to dig down to the lower levels. This should be possible, as long as you realize your code might not be as portable anymore.

Structure

With these goals in mind, here is the basic structure:

A Node is an object in the tree with a draw function. Anything which draws or affects drawing is a subclass of Node. This means all shapes, groups, transforms, imageviews, etc. Nodes also track their dirty state, and if they contain a given point (used for input processing). All nodes have parents (except the top most node)

A Parent is simple a node that implements the Parent interface. Currently only Group and Transform are parents.

A Scene is a tree of Nodes (a non-cyclical directed graph).

Everything is done on a single GUI thread. Touching nodes or trying to draw off the GUI thread is an error.

Resources (images, gradients, colors, textures) are immutable, to enable transparent caching.

Amino's internal system handles repaints, animation, and input events for you. you just create the tree of nodes and you are off to the races. By letting Amino handle these things we can ensure a consistent framerate and the best performance possible.

All events are generated by the system (usually by wrapping native events) and passed to your handlers through the event bus. You can listen to either a particular kind of event on a particular object, or all of that kind of event throughout the system. For example: Give me all mouse press events or Tell me when this node is clicked. Events will be automatically transformed into local coordinates when you click on transformed objects, and they are passed along with the target node of the event.

A quick note on cross platform support. My goal is to make the Java and JavaScript APIs identical. Wherever possible this is true. However, due to the difference between the languages (namely that JavaScript uses prototype based inheritance) there will be minor differences. In general the same code should work under both with only minor syntactic changes.

Example

The following code creates two colored rectangles with method chaining. Then it creates a square that spins and slides back and forth across the screen. I've included both Java and Javascript versions so you can see the minor differences.

Javascript

var runner = new Runner();
runner.setCanvas(document.getElementById("canvas")); var g = new Group();
g.add(new Rect().setWidth(100).setHeight(50).setFill("green")) .add(new Rect().setWidth(50).setHeight(50).setY(100).setFill("yellow"))
; var r = new Rect().set(-25,-25,50,50).setFill("white"); var t = new Transform(r).setTranslateX(100).setTranslateY(100);
g.add(t);
runner.addAnim(new Anim(t,"rotation",0,90,1).setLoop(true).setAutoReverse(false));
runner.addAnim(new Anim(t,"translateX",100,500,4).setLoop(true).setAutoReverse(true));
runner.addAnim(new Anim(t,"translateY",100,150,0.5).setLoop(true).setAutoReverse(true)); runner.root = g;
runner.start();

Java

final Core runner = new Core();
runner.setSize(600,400);
runner.setBackground(Color.BLACK); Group g = new Group();
g.add(new Rect().setWidth(100).setHeight(50).setFill(Color.GREEN)) .add(new Rect().setWidth(50).setHeight(50).setFill(Color.YELLOW).setY(100))
; Rect r = new Rect().set(-25, -25, 50, 50);
r.setFill(Color.WHITE);
Transform t = new Transform(r).setTranslateX(100).setTranslateY(100);
g.add(t);
runner.addAnim(new PropAnim(t,"rotation",0,90,1).setLoop(true).setAutoReverse(false));
runner.addAnim(new PropAnim(t,"translateX",100,500,4).setLoop(true).setAutoReverse(true));
runner.addAnim(new PropAnim(t,"translateY",100,150,0.5).setLoop(true).setAutoReverse(true)); runner.root = g;
runner.start();

Download

Amino 2 is still very much a work in progress, but so far I'm happy with the design. I've made several TouchPad apps already with decent performance, and the API really simplifies Java2D coding. Next I'm working on more graphics primitives, buffered effects, and a simple path API.

You can download a daily build here. Please join the Amino Dev List to provide feedback or contribute to the project.

Thanks!

Below is a screenshot of a debugging app I've been working on called SideHatch. It essentially lets you open up your Amino app from the side and poke around at the innards.

org.joshy.gfx.test.control.TranslationTestScreenSnapz005.png

In the screenshot you can see the two main tools: the Translator and the Inspector. The Translator shows all strings in your application that have been translated, broken down by category and locale. From here you can update translations and switch locales on the fly. The app will automatically refresh itself to show the changes without a restart.

The Inspector puts an overlay into every window showing the bounds of every control in your scene. Then you can move the mouse around to get info on particular components, like it's dimensions and translation. In the screenshot the indicator is telling me that the button I've hovered over doesn't have a translation yet.

SideHatch is currently in the 'translation' branch of the repo if you are interested in playing with it.

While I have many projects in progress right now, including Amino, Leonardo, getting the TouchPad out the door, and having my first baby (only a few weeks left!); every now and then I just get something into my head and have to code it out. Last night it was noise functions.

Since I've been using little bits of noise in my designs I thought it was high time to really learn how it works. Following code & articles from here and here I've managed to create a simple noise and turbulence generator in Java. With a few different settings you can get textures that range from completely random to something that looks like marble.

com.joshondesign.texgen.Test1ScreenSnapz004.png
pure random grayscale noise

com.joshondesign.texgen.Test1ScreenSnapz002.png
high turblence

com.joshondesign.texgen.Test1ScreenSnapz001.png
medium turbulence + one sinewave

com.joshondesign.texgen.Test1ScreenSnapz003.png
medium turbulence + three sinewaves

It's actually simpler than I thought. Noise is just a bitmap filled with random values from 0 to 1 (or a grayscale value between black and white). Turbulence is created by stacking multiple layers of noise, each stretched to different amounts. Add in a bit of linear smoothing and you get some quality texture. The marble effect is just a turblence texture merged with a sine wave.

Next up, making the noise repeat so we can have tileable textures. Then I might make a simple app to generate noise with various colors. If it all turns out okay I might make a plugin for Leonardo to create your own textures.

Today I'm doing a three hour hands on tutorial at OSCON on HTML Canvas. All you need is a text editor, Chrome, and basic JavaScript knowledge. By the end of the session you'll know a ton about Canvas and have built your own little video game that can run almost anywhere. The full lecture notes and hands on lessons are here.

You may be a new programmer, or a web designer, or just someone who's heard the word 'RegEx', and asked: What is a Regex? How do I use it? And why does it hurt my brain? Well, relax. The doctor is in. Here's two aspirin and some water.

What is it doctor?

Oh, it's two parts hydrogen and one part oxygen; but that's not important right now. We're here to talk about RegEx. RegEx is short for Regular Expressions, but that's also not important right now. Regexs are just patterns for matching bits of text. Whenever you hear RegEx, just think: pattern. In fact, I'll stop using the word regex right now, since it mainly sounds like the kind of medicine you'll need after trying to write a complex regex.

What are these patterns they good for?

Patterns are mainly used for three things: to see if some text contains the pattern (matching), to replace part of the text with other text (replacement), and pulling out portions of the text for later use (extraction). Patterns contain a combination of regular letters and special symbols like ., *, ^, $, \w and \d. Most programming languages use pattern matching with a subset of the Perl syntax. My examples will use JavaScript, but the same pattern syntax should work in Java, Python, Ruby, Perl, and many other languages.

Matching

Suppose you want to know if the text "Sally has an apple and a banana." contains the word 'apple'. You would do it with the pattern 'apple'.

var text = "Sally has an apple and a banana.";
if(text.match(/apple/)) { console.log("It matches!");
}

Now suppose you want to know if the text begins with the word 'apple'. You'd change the pattern to '^apple'. The ^ is a special symbol meaning 'the start of the text'. So this will only match if the 'a' of apple is right after the start of the text. Call it the same way as before.

var text = "Sally has an apple and a banana.";
if(text.match(/^apple/)) { //this won't be called because the text doesn't start with apple console.log("It matches!");
}

Besides the ^ symbol for 'start of text', here's some other symbols are important to know (there are far more than this, but these are the most important).

$ = end of the text
\s = any whitespace (spaces, tabs, newlines)
\S = anything *but* whitespace
\d = any number (0-9)
\w = any word (upper & lower case letters, numbers, and the underscore _)
. = anything (letter, number, symbol, whitespace)

If you want to match the same letter multiple times you can do this with a quantifier. For example, to match the letter q one or more times put the '+' symbol after the letter.

var text = "ppqqp";
if(text.match(/q+/)) console.log("there's at least one q");

For zero or more times use the '*' symbol.

var text = "ppqqp";
if(text.match(/q*/)) console.log("there's zero or more q's");

You can also group letters with parenthesis:

var text = "ppqqp";
if(text.match(/(pq)+/)) console.log("found at least one 'pq' match");

So, to recap:

. = any x+ = match 'x' one or more times
x* = match 'x' zero or more times ex: match foo zero or more times, followed by bar one or more time = (foo)*(bar)+
x|y = match x or y

Replacing text

Now that you can match text, you can replace it. Replace every instance of 'ells' with 'ines'.

var text = "Sally sells seashells"
var text2 = text.replace(/Sally/,"Billy"); //turns into "Billy sells seashells"
var text2 = text.replace(/ells/,"ines"); //turns into "Sally sines seashines"

Modifiers

Most pattern apis have a few modifiers to change how the search is executed. Here's the important ones:

Make the search case insensitive: text.match(/pattern/i)

Normally the patterns are case sensitive, meaning the pattern 'apple' won't match the word 'Apple'. Add the i parameter to match() to make it case insensitive.

Make the search multiple lines: text.match(/pattern/m)

Normally a pattern will only match the first line of the text. It will stop at the newline character '\n'. With the m parameter it will treat newlines as whitespace and let you search the entire string.

Make a replace global: text.replace(/foo/bar/g)

Normally the replace() function will only replace the first match. If you want to replace every match in the string use the g parameter. This means you could replace every copy of 'he' to 'she' in an entire book with a single replace() call.

Substring Extraction

Another major use for patterns is string extraction. When you do a match, every group of parenthesis becomes a submatch, which you can use individually. Suppose you have a text string with a date in it and you want to get the year and month and day parts out of it. You could do it like this:</p

var text = "I was born on 08-31-1975 and I'm a Virgo."
var parts = text.match("(\d\d)-(\d\d)-(\d\d\d\d)");
//pull out the matched parts
var month = parts[1]; //08
var day = parts[2]; //31
var year = parts[3]; //1975
//parts[0] would give you the entire match, eg: 08-31-1975

The Cheet Sheet

The standard pattern syntax in most languages is expansive and complex, so I'll only list the ones that are actually useful. For a full list refer to the documentation for the programming language you are working with.

Match anywhere in the text: "Sally sells seashells".match(/ells/) (matches both sells and seashells)
Match beginning of text: "Sally sells seashells".match(/^Sally/)
Match end of text: "Sally sells seashells".match(/ells$/) (matches only the seashells at the end)

any word at least three letters long \w\w\w
anything .
anything followed by any letter .\w
the letter q followed by any letter q\w
the letter q followed by any white space q\s
the letter q one or more time q+
the letter q zero or more times q*

any number \d
any number with exactly two digits: \d\d
any number at least two digits long: \d+
any decimal number \d+\.\d+ //ex: 5.0
any number with an optional decimal \d+(\.\d+)* //ex: 5.0 or 5
match the numbers in this date string: 2011-04-08 (\d\d\d\d)-(\d\d)-(\d\d)
also be able to match this date string: 2001-4-8 (\d\d\d\d)-(\d+)-(\d+)

Conclusion

Patterns are a very complex subject so I've just tried to give you the basics. While complex, they are also incredibly powerful and useful. As you learn them you'll find you use them more and more for all sorts of cool things. For a more in-depth tutorial read the Mozilla JavaScript Regular Expression guide.

PS: Found a bug in the above code? Want to suggest more examples of useful regex's? Want to just shoot the breeze? I've turned off comments due to 99% of it being spam, so please tweet your suggestions to me instead: @joshmarinacci. Thanks for understanding!

Javascript has always had properties but recently support for property accessors and mutators was added, or rather it is finally supported in enough browsers that we can reliably use it. Property accessors and mutators are what we would call in the Java world "getters and setters". The cool thing about the new JavaScript version is that you don't call the getX method. Rather you can set a property the way you always have, as a variable assignment, but the accessor method is called underneath.

This is a very powerful feature, and yet my research indicates that they are rarely used in practice. In fact I can find almost no reference to them beyond how the syntax works. Where are the cool crazy things you can do with them? They seem incredibly useful.

Here's how the syntax works

var foo = { 
    x:0, 
    get value() { return this._x; }, 
    set value(x) { console.log("set to " + x); this.x = x; }
}
var foo = new Foo();
foo.x = 5; // prints 'setting x to 5'

The ability to invoke a method whenever a property is set seems very powerful. For example, you could bind the width of a rectangle to the value of a slider. Whenever the slider changes the rectangle would move, no extra update code required. I think the new property system would also let us have proxy methods, meaning we replace a setter or getter with a new implementation that does something special then calls the old implementation.

Then we could add after an object is created:

  • A listener to be called when a property changes.
  • A filter on a property to veto certain value changes. ex: restrict value to 0->1. veto anything else.
  • Bind any two variables together. If x changes then y is updated to reflect the new value
  • Bind the value of a slider to an object on screen, modified with an equation. ex: rect.width = slider.value * 3;

The new property syntax can also be used to create synthetic properties. A synthetic property is one that has no direct underlying storage. Rather the value is calculated on the fly, or delegated to alternate storage. For example, you could have a rectangle object with x and y as true properties. Then add a position property which is a short hand that maps to the x and y underneath:

rect.x = 10;
rect.y = 20;
// pos is a synthetic property. really sets x and y underneath
rect.pos = [10,20]; // pos is a synthetic property, really gets value from x and y
console.log("the rect is at " + rect.pos);

This could let us create more convenient APIs for certain domains like event handling.

onDrag(mouse) { 
    //set rectangle's position to the mouse's position 
    rect.pos = mouse.pos; 
    //set rect's position to the mouse's position plus an offset 
    rect.pos = mouse.pos.plus(30,40);
}

I don't think we can do operator overloading, or else this would be possible: rect.pos = mouse.pos + [30,40];

We could also do structure swizzling like this:

color.bgra = color.rgba;

Or modify colors stored as RGB using synthetic HSV properties, like this:

color.rgba = #ff00ff00;
color.hue += 30;
color.saturation = 0.5;

So my real question is: Why is no one using this stuff? Where are the cool demos by language gurus who know far more about JavaScript than me. Am I just missing the magic website that talks about it?

Please send feedback and comments to my twitter account @joshmarinacci instead of on the blog. Thanks!

With all of the hoopla last week about the innovative features in the new iBooks 2 I thought it would be instructive to see what could be done with pure HTML 5. I put together a little demo which adapts to screen sizes and has simple interactive content. Here's what it looks like:

ebook prototype

View the live demo here. I highly suggest you load it up on an iPad as well. Try rotating the screen (or make the browser window narrow if you are on a desktop.) Keep in mind this is just a prototype. It's not skinned or made to look pretty at all.

You can spin the little sphere tree by dragging with your mouse. If you view this on a tablet it will respond to touch events as well. You can click on the photo to see a large fullscreen version. The footer will stay fixed to the bottom. The inline photo and canvas will resize and move to the left edge if you switch to portrait mode.

My goal for this effort is use pure semantic HTML. If we want to scale this up to a full book then the markup needs to be as close to pure text as possible. I'm using only divs with paragraphs and headers for all textual content. The interactive stuff is canvas or img tags. All interaction is standardized and put into reusable javascript files except for the actual dot tree itself (though it does use reusable components). I had to add a little hack to make the canvas resize properly, but that is also reusable. The footer and general layout is pure CSS.

I think the prototype turned out pretty well. It would be easy scale up to a full book with one page per chapter. Table of contents, index, and glossary would have to be written by hand unless we have some automated tool to do it, though I think such tools already exist.

So far I haven't found much that the new iBooks 2 format does that couldn't be done with plain HTML. They did add nice columns and shaped floats, but that is part of the CSS 3 specs and could be implemented in the renderer quite easily. (And shaped floats have been hacked into CSS2 for over a decade!)

I do like the look of iBooks Author. It's a very nice visual tool that should let people create books without any programming experience. The new terms of service are a different issue, but after thinking about it I've decided I don't care. iBooks Author is a tool for formatting content to be sold in the iBookstore. It really has no purpose other than that so the terms of use don't make much difference. Amazon has their own tools for their own store as well, and I expect them to get better with the advent of the Kindle Fire.

While I would love to see a beautiful visual editor like iBooks Author that formats content for all possible bookstores I don't really expect Apple to build it for me. Such a solution will have to come from a third party. Fortunately third parties now know that Apple won't compete with them to build such a tool, so I expect we'll see something like it soon.

Anyway, back to my HTML 5 prototype. Anything else you'd like to see it do?

Please send feedback and comments to my twitter account @joshmarinacci instead of on the blog. Thanks!

 

 

 

 

 

For some reason the concept of Genetic Programming got stuck in my head the other evening. At midnight, after spending about four hours reading up on the topic around the web, I came away disappointed. The concept of evolving code the way genes do is fascinating but the results in the field seem to be very narrow and limiting. Thus began this rant.

This article called Genetic Programming: A Novel Failure probably sums up my thoughts best. Genetic programming is only a slight variation on solution searching algorithms. Based on my reading, most work in the field has focused on how to make systems converge on a solution more quickly, i.e.: improving efficiency. This seems wrong, or at best premature.

We live in the 21st century. We have more CPU cycles than we know what do with. Where are the systems that are wide but shallow? The ones that are really non-determinstic and will generate truly surprising results? We should be wasting cycles exploring new possibilities, not generating new solutions for known problems.

The free ebook, A Field Guide to Genetic Programming, is a great primer in the topic. I read through most of it the other night. My biggest frustration is that almost all genetic programing systems focus on evolving syntax trees, usually some form of Lisp or it's semantic equivalent. I see why people would do this. Lisp code is easy to manipulate programmatically, so evolving it should be simple as well. There are other kinds of systems using different gene encoding, such as image arrays and direct byte code. However, these appear to be far in the minority. The Field Guide has a chapter on the topic, listing several alternate systems. The fact that these systems receive a single slender chapter while the rest of the book covers syntax trees gives you an idea of how under-explored the topic is.

When I first heard of genetic programming I imagined having a sequence of simple instructions that could be mutated. The instructions would be extremely simple and limited, perhaps more simple than most assembly languages. Evolving syntax trees certainly does let you make progress quicker, but the generated solutions will be limited to the underlying tree language. Our genetic beasties will never evolve novel flow control systems, or invent a crazy kind of memory register. ASTs are great if we want to produce a human readable program as the result, but it still feels limiting.

I would like to see a system that is as open ended as possible. Create a system of small instructions of uniform length that can only manipulate basic storage and do simple math. Then give them as much freedom as possible. Let them live in a wide fitness landscape. A digital environment with a huge number of potential ecological niches. Ideally we could take this to the next step and give our digital creations physical bodies so that they may evolve targeting real world constraints. Evolving a robot's brain seems far more interesting than figuring out how to gain 10 milliseconds trading stocks. Of course we lose rapid iterations by running them in the real world, so it my be better to run them in a simulation of the real world at many times normal speed. I imagine we could build self driving cars this way, starting with bots that play racing games then upgrade them to working with real world footage from actual self driving cars.

Am I wrong? Is there cutting edge genetic programming that is truly open ended? What successes have they made?

Please send feedback and comments to my twitter account @joshmarinacci instead of on the blog. Thanks!

 

Jen was working today, so I spent the day fixing bugs and coding new features in Leonardo. Today's awesome feature: HTML Canvas Export. Yes, oh yes! You can draw anything you want in Leonardo, then export it to JavaScript code that draws into Canvas. Why would you want such a feature. Lots of reasons:

  • Maybe you need to draw vector graphics in a browser that supports Canvas, but not SVG.
  • Maybe you want to learn Canvas and have some code to start with.
  • Maybe you are just simply too lazy to draw your own damn pictures in code.

Whatever your reason, Leonardo will be there for you. Look for it in our next release!

Check out this demo output page.

Another month has gone by with no update to Leonardo, or a real release of Amino. It's interesting how life changes. When I started this projects last summer I had no idea Jen and I would be having a baby in a month, nor did I truly have any notion how much my life would change. Everyone always says having children will change your life, but you never really understand it until you do it yourself, and our journey has just begun.

So, the upshot of all this rambling is that kids take time, and when you have to distribute a finite resource between multiple buckets, something has to get less. Sadly this time the short straw goes to my open source projects. It doesn't mean I won't work on them anymore, just at a slower pace. However, in order to feel at peace with myself I need to leave them in a state where they can still progress without my large time commitment. That's what this post is about.

I've spent the last year working on two main open source projects called Leonardo and Amino. Quick recap: Amino is a scene graph library for Java and JavaScript. Leonardo is a vector drawing program built on top of Amino. I want to get them both to a state where they are stable, useful, and can live on their own. Hopefully more of my job will be driving the community and integrating patches rather than directly coding everything. Every project reaches a point where it should stop being driven by a singular vision, and instead be driven by needs of actual users (the good projects anyway). Now is the that time. Time to focus on gaining adoption, growing a community, and making these projects rock-freaking-solid.

Concrete Plans

Amino

Amino basically works. Shapes are animated and drawn on screen, input events are properly transformed, and it's got pixel effects on both the Java and JavaScript sides. Speed, efficiency and features driven by actual use.

Amino finally has it's own website at GoAmino.org and I've set up auto-builds for both the Java and JavaScript versions. They also have a redesigned API doc template that you can see here. Last steps before a 1.0 release: bug fixes for Mobile Safari and FireFox, more demos, and a tutorial for integrating with Swing apps. (Oh, and if someone has a nice spring easing function, that would be swell). Target: next weekend.

Leonardo

It's basically done. It lets you draw vector art of shapes, images, and paths; and also create attractive presentations (which is just multiple pages of vector art). Now comes polish and adoption and export features. I suspect the value will really be in the export features so I need to know from you guys: what do you want?

In concrete terms I have a bunch of old bugs to fix and will finish the redesigned fill picker (colors, swatches, gradients, patterns, etc.) I also need your help updating the translations. Once that's done I'll clean up the website and cut a 1.0 release. Target: end of April.

Next Steps

In short, a lot of work for the next few weeks, but with luck (and hopefully some great feedback from you) , both Amino and Leonardo will be just fine.