The Sea of Ideas

Why Canvas is not an obvious choice for web games

I initially wrote this post for our corporate Dextrose blog at http://blog.dextrose.com but due to its technical nature, I republished it over here in order to get a bit more attention. Enjoy!

There has been quite some controversy about the decision to use plain old HTML to render a scene with the Aves Engine versus the utilization of the new popular Canvas tag since my Google tech talk is up (http://www.youtube.com/watch?v=_RRnyChxijA). I wanted to elaborate a bit about the reasoning behind all this.

Canvas doesn’t play well with images

Unfortunately, we happen to use lots and lots of different images in a typical isometric scene with the Aves Engine. With Canvas, there’s a friction in the API when working with images: When trying to include an image into your canvas render, you need to first construct it via DOM methods (new Image()), set the source, wait until it is loaded and then render it to the canvas. A step that is painful, even if you’re later on caching the images, and most importantly, a step that is horribly slowing down the Canvas interface with whatever reasoning behind it.

In fact, even with precached DOM representations of images, our basic tests have shown that Canvas based rendering of an isometric scene is almost always 2-3x slower than just dumping out a big HTML string with some class names linked to external stylesheets. External stylesheets just seem to be a much better cache when connected to HTML output than DOM-Canvas.

Canvas-based redrawing is painful

There’s a lot of motion going on within games based on the Aves Engine. They’re mostly full screen (at least extended to the window size) and feature character sprite animations and transitions. Say a character moves by 20 pixels to the left. It’s easy to clear the whole canvas and rerender it, but also extremely slow. It’s however extremely difficult to implement an own partial redrawing system. Here’s roughly what the engine would need to do:

1. Get the outer boundaries of the whole part that was changed, i.e. the old character position and the new one combined in a rectangle
2. Redraw this part with all elements not in motion during this redraw – includes characters, objects etc. Engine needs to keep track of all positions of all elements on the viewport all time.
3. Now that you cleared the part, you can place the character on top again with the new changed position

This might roughly give you an idea of the steps involved here. I’m not saying it is impossible, as this is in fact how most efficient rendering engines do redraws. It’s just very very time consuming and difficult, and we haven’t had the chance to explore it yet. On the other hand, the browser rendering engines already do this work for us at the moment, which is really convenient for the time being.

Action surfaces

If you have seen one of our demonstrations or videos, you have also seen one of the most unique features of the Aves Engine, our action surfaces. It allows you to place generic HTML content onto any kind of surfaces, transformed to isometric projection. This is working due to the possibilities of applying 2D matrices to elements via CSS Transforms (WebKit, Gecko, Opera) or the matrix filter (Trident [IE]).

Action surfaces obviously also need to be considered in the layering of objects. If a character is walking in front of a wall, the action surface needs to stay behind the character. With HTML based rendering, this is pretty easy, as we’re just using z-index to control the layering, and the action surface lives in the actual container element.

Now with Canvas, this gets pretty tricky. There’s no way to render HTML content (or a snapshot of it) to Canvas, first of all. Doing screenshots of portions of the screen and rendering them to canvas was possible some time ago in Firefox, and I think still is for XUL, but is disabled in web context for security reasons (you could snap contents of file inputs etc.). Now this leaves you with only one apparent choice:

1. Collect how many action surfaces need to be displayed in the current render
2. Count, how many different layers you would need
3. Split the Canvas render into multiple <canvas> elements for each layer and slide in the DOM based action surfaces inbetween the partial renders

Thank god canvas supports see-through transparency, but this is still a pain nonetheless. I’ve never seen a system like this implemented and working. We’re happy to be the first, but again, haven’t tackled it for the alpha version of the engine.

Again, we’re still experimenting all the time with possible render improvements, and this post was mainly meant to give you some insights in how we do R&D to base our assumptions on actual tests. For us, it’s not so much about the technology – it’s about delivering a truly sophisticated gaming experience. We’ll do our best to squeeze the most out of the open web stack, and we look forward to jumping on Canvas when the time comes!

The issue with progressive enhancement

If you have been following my latest tweets, you’ll notice this is simply the execution of a longer explanation why I think concepts such as progressive enhancement are often very problematic. In the style of a ppk rant, and with the fear of making people angry, disappointed, mad, excited or unbelievably happy, here it goes.

What is Progressive Enhancement?

First, let’s discuss what I’m actually talking about. Progressive enhancement is a programming concept that simply says you should start building your product or website with the “Lowest common multiple approach”, so it runs in browsers like Internet Explorer 6 or with JavaScript disabled, and then continue to enhance functionality, interaction and UI for more capable browsers, devices and users. When well executed, this means that your site is targeted and usable by a maximum audience.

..but isn’t this awesome?

Indeed it is! That’s what’s great about progressive enhancement. It makes sure your website runs on every device or browser if executed well, which is definitely something admirable. It also has a multitude of positive side effects – such as possible accessibility benefits and it is really well suited for a junior programmer, as it teaches the layers of a web app in a direct way. … so what am I complaining about?

PE limits creativity

My first thesis is actually very easy to explain and to follow. So you’re a student of a famous painter, and you’re learning how to draw basic forms. After the art class, the teacher decides to draw something in front of the class to show off his skills. Unfortunately, all canvasses are used by the class, so he has to pick yours and paint on top of it. I’m pretty sure he succeeded in making it awesome based on your painting, but what if he had a clear white canvas to start with?

I’m positive that most people will come to my conclusion that the second painting would have been truly his, unleashing all his creativity. The situation is, as you might imagine, extremely similar with progressive enhancement. Since you always improve upon the lowest layer of your implementation, you’re not inventing new user interfaces, new metaphors or interactions – you’re merely improving the old ones. Even more unfortunate is the fact that we accepted to live with this fact. It is pretty likely you wouldn’t even have noticed this if you didn’t read about it here.

So we want to start drawing on a clear white canvas. Wouldn’t Graceful Degradation fit the bill?

Graceful degradation

Graceful degradation is basically the opposite of PE. It says “start on a free canvas and build crazy shit, then make sure it runs in less charming situations”. While in theory definitely a great improvement to PE, it doesn’t always work out, since there are many complex UX patterns and ideas that cannot be reproduced easily in a lower metaphor. In general you could therefore say that it also simply requires a lot more time than PE. That being said, I still use GD to a great extent in many of my projects, and I’m pretty happy with most outcomes.

You are the problem!

Uh well…not personally. Let me explain: I fully endorse the concepts of Progressive Enhancement and Graceful Degradation. In fact, PE for instance is a vital concept for jQuery UI to follow and implement. The real problem comes with the usage. Web developers misuse PE and GD to a great extent. And it is very easy to follow the wrong path if every “Best practise” powerpoint slide tells you to always follow PE. Let me make that crystal clear: “Always follow” = guaranteed misuse of PE.

So what? I should just screw IE6?

Actually yes, you should screw IE6. …errr, let’s start over: First off, in many instances, you will never need an alternative. If you’re targetting iPhone Safari as platform for your web app for instance, or if you’re working on an intranet application to be used with only a single browser. However, there is a great alternative to PE & GD: Instead of adding layers on top of your current site or app and let it become a big fat hard to maintain blurb, how about developing a second client? I will argue that it doesn’t take longer than the GD approach, and quite important players have done so: Gmail, anyone?

The bottom line: Start thinking!

As mentioned, it is really not a problem of the concepts themselves, but of how they’re used. If you’re reading this, give it some serious thought and start thinking yourself. Think back in time and look at the projects you’ve done so far. Is there a chance to improve your workflow by not following every best practice advise blindly? I bet!
shre forever after

The coverflickr

When I was refreshing my labs and updated them to fit the corporate Dextrose design (http://labs.dextrose.com), I quickly figured that I wrote quite a few plugins in the past that still deserve a lot more attention, because they are..well… kind of awesome.

In order to receive more attention, plugins like Coverflow need a kickass demo. Before, I only had small image demos in a small canvas, which weren’t quite as effective, so I decided it’s time for a killer mashup: Meet the Coverflickr.

The Coverflickr is a mashup experiment to show the beauty and flexibility of the Coverflow plugin while making use of a multitude of new HTML5 features. It’s an interface for browsing Flickr’s interesting images in a graphically sophisticated way. It is best experienced in Safari (even better : Latest webkit nightly!) or Chrome, although it will run smoothly in Firefox 3.5+ (minus preview animations and reflections). Here’s the techniques that are being used in the Coverflickr for those who care:

• CSS box reflections combined with CSS gradients for the reflections
• CSS Transforms for the preview mode and within the coverflow plugin
• CSS Transitions for the preview mode and the alternate mode switching animation
• YQL for the fetching of the images

There are a couple not immediately visible but pretty neat features going on here. For them, I had to significantly alter the coverflow plugin (soon to be released in a new version):

• Endless browsing. When coming to the right end, it fetches more images, appends them to the list and refreshes the coverflow with the new method “refresh”
• Alternate transformations.Click on the bottom link “Alternate version” and the transformation will animate live into another one now using rotations. The coverflow plugins transformations can now be exchanged on the fly through the “transformation” option.

This has been an incredibly fun demo to do, and I realized it wouldn’t be possible to do so without the help of the Webkit project, pushing the web like nobody else. I therefore dedicate this demo to the Webkit team, my personal heros that are not afraid to pioneer new features for the greater (web developer) good.

Enjoy, and let me know if you like what you see (also if you don’t, but please be kind) !

Unleashing your power with music

In one of my productivity articles I wrote about how music can help you gain and maintain productivity. Today, I’ll tell you about a special method how to find the right tunes.

First of all, forget about the crap clear your mind about what you know and learned about productivity music or sounds. Things like “listen to nature’s tunes” and the likes. This might be a good, but too general advice. Also don’t think about music that helps you relax or focus. What you essentially are looking for is music that let’s you *perform* well.

So how to find what you need? Listening to a thousand songs while working and then trying to guess what worked best usually doesn’t work: Simply because your job is probably too complex and not repetitive enough to give you a clear idea of which 3 minute song actually worked best.

All hope is not lost though – the answer is simple. Find something that you deeply want to achieve or perform well on, but is simple and repetitive enough to actually feel and catch the results immediately. Tetris, anyone?

Games

Yes, I’m talking about videogames. The sort that is highly addictive and entertaining and lets you run after a certain achievement for hours. The most famous candidate here is Tetris – if you’re looking for something “fresher” on the iPhone, try Doodlejump! My personal choice features a jumping character with only one target: Jumping higher and higher.

With the aid of Doodlejump, my iPhone and my portable music library, I immediately see results when I’m listening to a song that works. Yes, it’s dead simple. Yes, grab your phone and try now! Enjoy!

Array.toObject

This is another tiny helper I’ve just realized I need badly. I often do something like this to convert Arrays holding Objects into Objects holding Objects for easier lookups:

var newObject = {};
for (var i=0; i < someArray.length; i++) {
	newObject[someArray[i].id] = data[i];
};

I realized this can be beautified a bit by creating Array.toObject:

Array.prototype.toObject = function(f) {
	var o = {};
	for(var i=0; i < this.length; i++) {
		o[f.call(this[i])] = this[i];
	}
	return o;
};

This very simple function lets you transform the above code to

var newObject = someArray.toObject(function() {
	return this['id'];
});

Arguably it is not much shorter, but simpler. The anonymous function is needed because we need to tell the toObject function how to build up the Object key for each Array entry. Cool enough, this doesn't only work with Objects now, but with any kind of Array data. If in fact we know we're only dealing with Arrays holding offsets, we can optimize:

Array.prototype.toObject = function(key) {
	var o = {};
	for(var i=0; i < this.length; i++) {
		o[this[i][key]] = this[i];
	}
	return o;
};

..which then lets us do:

var newObject = someArray.toObject('id');

Now that's better, no? Enjoy

Why tabs are often a bad idea

This article was also released as guest blog on SixCrayons.

There was a time a couple years ago when the world needed browser pop-ups. It was awesome – it gave you the technology to launch a dedicated part of your website in a new window, and opened countless possibilities for the booming market of internet advertising. That certain factor of ‘coolness’, however, made people blind -they did not understand that popups had literally no user experience benefits at all (prove me wrong if you dare!).

Just at this very moment, history is repeating itself once again. Now, everyone completely agrees that popups are sooo 199-something and distracting, and bad UX alltogether. Everyone wants tabs now.

The problem of tabs

What’s wrong with those tabs then? Well, actually nothing! Tabs are awesome, and I couldn’t imagine going back to a browser that doesn’t feature them. The problem really isn’t about tabs (Klaus should be relieved now!), the problem is how people intend to use them.

What are tabs good for?

To start with an easy concept, tabs basically allow you to distribute content into several containers that can individually be activated one at a time, thus saving screen real estate. Tabs can be static or dynamic, meaning they can be opened and closed, and work excellent in situations where you’re opening instances of the same content format. Take a browser, for instance: You’re essentially opening websites all the time – content containers with the format “Website”.

What are tabs not good for?

The obvious issue people don’t realize is that only one tab can be visible at a time because they’re sharing the same visual position. This is the big issue : Tab do not work when the content is..

1. ..of a completely different (visual) layout
2. ..meant to be visible all the time
3. ..needs to be easy comparable against other content
4. ..important content

Of those three, the third and forth are probably the most important to realize. If you are currently utilizing tabs on your site, think for a bit if not seeing the complete content currently distributed is a disadvantage, or in other words: Does the UX become better if all content is visible at once? If your content is vitally important, do not hide it within secondary tabs! All tabs require user activation, and makes your content less visible.

Conclusion: be reasonable!

If your screen is getting crowded next time and you’re thinking about utilizing tabs, take a minute and think about the above mentioned issues. It might very well be that tabs work perfectly in your situation – it’s when you realize the tabs do not restrict the user experience.

Cheers!

Underestimated UI techniques: Morphing

When I design new applications, I realized I use a couple patterns that I haven’t seen in much use elsewhere, so I’ll start to blog about them. The first pattern in this series is called morphing.

What is morphing?

Bush-Obama morphing

Morphing essentially describes a state change on an element, transforming it into another. Many UI frameworks have helpers for morphing, usually in the form of class animations. jQuery UI has made it transparent to animate jQuery’s class manipulation functions like addClass, script.aculo.us has Effect.morph which essentially does the same.

Why is it useful?

From a higher level UI perspective, morphing really describes that one element becomes another based on the context. It is essentially very powerful because it gives your users a visual cue what is happening, how is happening and why it is happening. Instead of doing an instant change, the users eye can follow the motion and you therefore give them a better feeling of control over the whole situation.

Show me!

Let’s do an example. While working on smart.fm’s new item builder, I had to visualize the following steps: Clicking on a button to add text, then displaying a text input, and hiding the button since you cannot add text twice. Usually, it would roughly look like this:

However, with morphing, it not only looks hot but solves the issue in style:

This was a particularly lazy example, since it doesn’t even truly morph one item into another – it just animates the width of the button and fades in the input on top. But visually, the element clearly transforms.

Morphing in the wild: Inline editing

One technique that was is heavily inspired by morphing is inline editing. Essentially, you click on a paragraph or element, and it transforms into and editable element. Most solutions don’t actually morph, but you get the idea. There are thousands of other usecases out there waiting to be discovered, and if you have some, I’d love it of you share them with me!

If Walt Disney was running a web business

I’ve long wanted to write about the little known Disney method, also called Walt Disney strategy. In fact, it is so little known that there’s not even an english Wikipedia article or reference about it, so here’s the german article, auto-translated by Google.

Walt Disney was not only known as great entrepreneur, but also as an individual who could switch easily between many different roles. Quoted from Wikipedia, he was “film producer, director, screenwriter, voice actor, animator, entrepreneur, entertainer, international icon and philanthropist“, and that’s only his official roles. The Walt Disney method essentially strips it down to three, sometimes four roles: The dreamer, the realist, the spoiler and the neutral. It is a highly effective creative method inspired by Disney, and can handle almost any product creation process. Before I’m going to explain how it can be applied to the web development world, I’ll first explain the basics.

The Disney method explained

Essentially a role playing strategy, the Disney method is said to work best with four different roles (I usually skip the forth though..):

• Dreamer (visionary, delivering ideas)
• Realist (doer)
• Spoiler (critic)
• Neutral (consultant, reviewer)

Take four chairs and mark them with above roles. As single technique, it is recommended to start within the neutral position to analyze the problem. I, however, recommend to start as dreamer, since there’s in fact not always a problem you want to solve. When you’re sitting on the first chair, you start brainstorming until you have a beautiful complete vision. As soon as the dreamer in you is happy, you move on to the next chair, taking the role of the realist. You analyze the dreamer’s ideas and make sure they’re implementable, or strip them down accordingly. When you think it works out, you pass your product to the critic, who will then try to find anything possibly wrong with it. When the though review is done, take a deep breath as neutral consultant to review everything from a bird’s perspective, and explore if every chair role is happy. If they’re not, simply continue your circle: Pass the dreamer the finished implementation and critique, and let him explore different or additional directions. Do those steps, until everyone is considerably happy with the product.

The whole strategy can of course also be applied to groups of people, meaning you have a full discussion going on between the roles, and every couple minutes you move places. However, I highly suggest trying out the single variant, because it proved to be much more effective for me personally.

Applying it the web development

As already mentioned, the Disney method can be applied to almost any situation. Let’s apply our roles to the web development world and see how they would look like:

• Dreamer becomes Designer, Prototyper
• Realist becomes Engineer
• Spoilers become your users, investors

Note that I skipped the neutral role, because the new spoiler is essentially both, and it allows for faster iteration. Now that we have a fully working cycle for a web development situation, let’s narrow it down and focus on individual implementation aspecs:

Going into detail: Implementation roles

If it’s not working well enough on a high level like above, you can always narrow it down to more granular roles. As an example, I’m taking the engineer role, and splitting it up into three:

• Model
• View
• Controller

or how about this one:

• Rapid prototyper
• Frontend engineer
• Backend engineer
• Performance Guru

As you can see, there are endless possibilities.

Now why should I do it anyway?

The whole point about the Disney method is that it allows you to understand your co-workers, the flow of a product, and the most effective path to your end goal. It gives you the chance to have a look at each role’s unique perspective, how they see themselves and others around them, and when mastered, gives you a bird eye or global view that incredibly improves your leadership or individual role. If you do it a couple times, it’s very likely you start projecting your 4 chairs on your actual co-workers and naturally improve the workflow.

Switching between different roles is a unique and highly rewarding skill. I personally believe that this particular skill let Disney become the legend he is today. If Walt Disney was running a web business, he would simply add a couple more roles to his mind model and start to build up highly successful web products.

Now go ahead and grab some chairs!

Why I would hire game developers for my startup

This is one of the ideas I’ve only realized now in context but always knew deep in my mind. Whenever I talk publicly, I talk about pushing the limits of the web. My startup ideas are nothing different – every idea I have pushes the web to its greatest limit to deliver the most impressive result.

Perfectionism vs. trying to be awesome

Note that all ideas I have, and everything I’ve presented so far is not related to being perfect or delivering the perfect web product. Perfectionism is extremely dangerous, and chances are you never get the job done. In fact, many of my labs demos are far from perfect, even stable, but they push the limits. They deliver awesomeness. They provide great experiences.

Looking at games

Why games? It’s actually really simple. While web developers have been whining for years and have been stuck in their despair, game developers never stopped to amaze with the little tools and possibilities they’re given. 3D on a Super Nintendo? Easy. Full blown 3D RPG on PlayStation 12 years ago? You bet. Yes, game developers deliver.

Cheating and optimizing

Why are game developers so different, and what are they doing? Well, for once, they’re simply cheating. Remember Final Fantasy VII, the greatest selling role playing game of all time that lead Sony’s success of the PlayStation? Since the limited 3D capacities where not enough to deliver a great experience at times, they simply uses prerendered scenes and mixed them with live renderings. Yes, 1997. The even bigger point though is the optimization efforts that are made. Web developers only optimize when they see performance issues. For game devs, they know from the beginning on that the console they’re developing for cannot handle the raw 3D data, so they need to find optimization patterns right away!

They for instance need to control the number of polygons on the screen, memory consumption and frame rates. Many 3D techniques developed in the last 10 years are in fact optimization techniques, that either deliver a richer experience with the current hardware, or consume much more energy to be able to use much more of the same.

Delivering

Most importantly, game developers aren’t afraid of going different routes if something doesn’t work out. They try hard to deliver the experience they’ve planned, not the actual expact specification. Whenever I work on new projects, I ask myself “how would a game developer do it in a game?”. This usually gives me brilliant new ideas, and I highly suggest you try it out!

Now if only I could convert some game developers to start over in the web world…

A UX pattern: Reusable hints with jQuery

Update (1): The CSS source had wrong formatted comments and the markup was using class instead of id, please update your sample code when you grabbed it earlier!

Update (2): Please find a working demo here: Hints demo

When building nifty UI solutions, always keep in mind that in order to make your target audience use your cool features, you must teach them. The iPhone is to a certain extend only intuitive because they have the marketing power to teach people on TV how to do pinches and swipes. Anyway, since most budgets are smaller, let’s try to achieve the same with contextual hints.

Contextual hints are basically help bubbles that pop up as soon as you are in a certain context. They can also serve as an application walkthrough or tutorial. If you build a todo list, the first hint you would show your users would be sticking on top of the “Create new todo” button, and would tell you something like “Why don’t you click here and create your first task?”.

I’m sure you got the idea, let’s build a very simple solution without any graphics (The downside: Will not look too good in IE..). First, we need to create the markup for the hint:

<div id="bubble">
	<span class="content"></span>
	<div class="pointer"></div>
</div>

Sweet. We just have the outer bubble element which has the id #bubble, a content div that will hold our text and a pointer div which we’ll transform to a nice triangle. Next, let’s do the CSS:

#bubble {
	background: rgba(255,255,255,0.9); /* use rgba to let the bubble shine through a bit */
	font-size: 1.2em;
	padding: 1em;
	width: 22em; /* the bubble needs a certain width since it's absolutely positioned */
	-moz-border-radius: 1em; /* give the bubble nice rounded corners */
	-webkit-border-radius: 1em; /* ...in both gecko and webkit */
	position: absolute;
	z-index: 100; /* give it a high value here to lie on top of all other stuff */
	opacity: 0; /* opacity must be set to zero at beginning (use filter: alpha(opacity=0) for IE) */
}

#bubble .pointer {
	position: absolute;
	bottom: -1em;
	left: 2em;
	width: 0;
	height: 0;
	border-style: solid;
	border-color: rgba(255,255,255,0.9) transparent transparent transparent; /* make sure only one border side is shown */
	border-width: 1em 2em 0em 1em; /* this is called a border slant and creates the shape of the triangle */
}

Obviously feel free to customize everything related to styling, font size etc. Next, we want it to see come all together by doing the JavaScript/jQuery work:

$.fn.hint = function(msg) {

	var offset = this.offset(), // generate the offset position of the hinted element
		bubble = $('#bubble'), // cache the bubble as jQuery
		pointer = $('.pointer', bubble), // cache the pointer of the bubble
		fadeDistance = 50; // the distance from where the bubble will fade in

	// append the message to the bubble, position it and slowly fade it in
	bubble
		.find('span.content').html(msg).end() // insert the new message
		.css({
			top: offset.top - bubble.outerHeight() - pointer.outerHeight() + this.outerHeight()/4 - fadeDistance, // the element offset minus the height of the bubble, minus the height of the pointer, plus one quarter of the height of the element to be on top of it, minus the fading distance
			left: offset.left + this.outerWidth()*0.75 - 42 // the element offset + 3/4 of the element's width to position the bubble at the right side, minus the pixel width to the edge of the triangle
		})
		.animate({
			opacity: 1, // fades it in
			top: '+='+fadeDistance+'px' // moves it in from the fade distance that we substracted above
		}, 600);

	// make sure the bubble goes away when clicking on the hinted element
	return this.one('click', function() {
		bubble.animate({
			opacity: 0, // fades out
			top: '-='+fadeDistance+'px' // animate back the fade distance
		}, 300);
	});

};

That’s it. Now all you need to know is it’s usage, but you probably know it already:

$('#myButton').hint("Click here to make it explode.");

This will blend it your hint, and as soon as you click on the element, it will fade away again. Sweet, huh?