Inspiration

Inspiration

Real-world APIs0

My friend Nicolas is defending localStorage in his blog, and I want to show my full support. In particular, this following sentence caught my eye and should make you curious:

[It] absolutely stinks of browser developers creating an API that’s easy to implement rather than creating an API that’s easy to consume. This is the opposite of how good APIs are made.

I simply couldn’t agree more. Browser vendors and the W3C tend do de-prioritize the most important feature of any API: Its usability.

In April 2011, I proposed a new CSS feature I called “hitmapping” in the form of extending the pointer-events property to support a new opacity threshold. Everybody I talked to – including browser vendors – agreed on its importance and usefulness. However, the thread soon turned into a discussion of how difficult implementation would be in WebKit, and the proposal was blocked. How can this possibly happen? As a web developer, I couldn’t care less whether something is going to be harder than something else to be implemented because of a browser’s architectural design decision!

What do we get instead? In my case, nothing. I have to perform black magic voodoo (image tracing and canvas manipulation) to achieve the same feature. In most cases though, we get alternative verbose APIs that are too complicated, too slow or too verbose. But hey! At least they’re easy to implement by browser vendors.

As a library developer, I’ve been going to hell so other web developers don’t have to. Vendors seem to have settled on the idea that web library developers will create abstractions around their new browser APIs, so developers can use them.

To the vendors and spec editors: This is not ok! It’s part of your main responsibility to create usable, real-world APIs. Library developers are doing your job because you don’t. I know many of you personally, and I know you are smart enough to fix this. Do it!

Thank you,
Paul

The jQuery Foundation0

Today, I’m incredibly proud and happy to read about the launch of the new jQuery Foundation, and rather than doing a quick tweet, thought I’d write down a couple words to congratulate all members of the jQuery team, and all jQuery users.

I was one of the original six signers of the documents when we joined (guy signing the paper is me) the Software Freedom Conservancy as a project on September 11th, 2009. Back then, the jQuery organization was still a toddler, and we were in real need for legal and administrative support to be able to concentrate what matters most to the jQuery team – jQuery, its sub projects and its users.

I’ve been out of the main loop for some time now, but reading about the new foundation gave me a smile today. jQuery as a project has matured to the point where they can pull this off and succeed, and there’s few open source projects out there who can. The secret sauce to success is an extremely passionate, strong team of developers that make up the jQuery Core team.

It’s been a hell of a (fun) ride when I’m looking back at my personal relationship with jQuery. When John and the community in 2007 asked me to start jQuery UI, I didn’t know what was in front of me. It’s been an amazing time that taught me many things about software development and working with others. Working on the jQuery UI Team helped many to get their next jobs, including myself, and most importantly, I have made many great friendships during my time at the jQuery project.

Keep on rocking, jQuery!

Enabling the internal debug menu of Safari2

I promised my next blog post will be positive, so here we go. Few people know this, but while Safari debugging on iOS doesn’t exist, Safari debugging on PC is actually fairly useful. There will be a more elaborate blog post following soon, but this first little goodie will get you going first. Paste this into your Terminal and run it:


defaults write com.apple.Safari IncludeInternalDebugMenu 1

Next time you start WebKit nightly or Safari, you will find a new menu called “Debug” (nope, it’s not the Develop menu.). This is how it currently looks like on Snow Leopard:

To me, the by far most useful features are the “Show Compositing Borders” and “Show Caches Window” options. Compositing borders gives you insight into the different render layers and helps you identify render issues (more on this soon), and the cache window gives you lots of info on memory usage. Finally, please note that there’s a good reason why this is hidden from us in the first place: Much of it is not self-explanatory and some of it will not work at all on your machine. Handle with care!

Enjoy! Thanks goes to Alex Russell, who gave me this very helpful tip around a year ago.

On accurately measuring fps6

If you look for a quick solution, I have bad news: It’s not possible. Not even close. Well, in the browser at least.

For many subsequent years, smart web developers have tried and failed to produce accurate ways to measure something that comes close to displayed frames per second on websites. Something that’d been so vital to the gaming industry for decades can’t be that hard to measure, right?

In the web world, fps map closest to redraws. Redraws (or repaints) happen when anything on screen needs to visually change (if the layout changes as well, an additional reflow/relayout will be done). Many implementations tried to address the same problem:

  • running intervals at ~16ms (60fps) and check how long they actually run to produce a number
  • changing the width of a DOM element and read out the offsetWidth to see if the new width has been applied
  • Internal browser debug tools (namely Chrome and WebKit)
  • the Mozilla-only MozAfterPaint event
  • requestAnimationFrame

I have tracked all of them down for you so you don’t have to try them out only to find out later that you’ve been tricked. Yes, none of them actually work. Yes, it’s a bloody nightmare.

Running intervals is never reliable – it just tells you that the JS execution is taking time, or sometimes not even that – many browsers throttle intervals artificially in some cases. It gives you no information whatsoever about painting. Changing the width of an element and reading back the offset values is better, as the numbers are smaller (smaller = more realistic), but too small – you want redraws only, not expensive reflows. Internal debug tools, such us the WebKit Debug menu or Chrome’s flags will show an FPS counter but no FPS. Rather, the opposite – Chrome’s FPS counter goes up when lots of stuff on screen happens, and goes to zero when the browser idles. The MozAfterPaint event wasn’t reliable when I tested it many months ago. The same goes for requestAnimationFrame.

I honestly don’t know why requestAnimationFrame is such a let down. It was supposed to be the cure for all our fps needs. Browser vendors told me over and over that this is the number I’ll want. Well, it’s not. I trust my vision more than your API. And I know when the counters are lying. You’re telling me 60fps when I’m clearly seing 20.

Update: I get the feeling that it’s wrong to blame requestAnimationFrame – it’s not even meant to report the number of draws, but rather informs you when you theoretically might be able to draw. I need feedback instead.

Seriously now. Can somebody please fix this?? You’ll be my personal hero for a month and I’ll buy you a nice german beer.

On creating great products10

People who know me are aware that I have been, and still am a long time supporter of the open web stack, and I have talked a dozen times about how HTML5, CSS3 and JavaScript is the only way to produce apps, sites and games that run cross-platform, accessible to the widest audience possible. Now don’t worry – I’m not changing my opinion. Native is out, HTML5 is the future. Period. But there’s something I have to tell you. It’s about finding the right compromises.

I’m getting bored of people publicly yelling and complaining at people who have build something great that only works in a limited set of browsers.

“Uh noes, you can’t ignore [insert browser of your choice here, i.e. Opera]! You’re fighting the open web!”

There, I said it. A couple years ago, as a developer, I could see their argument. Open, interoperable, non-monopolistic and awesome and all. A couple months ago, I was angry. And now I’m just bored.

Imagine somebody grants you a budget of three months development time, and it’s up to you to make the right compromises. You can either create an ‘okay’ product that runs on 95% of today’s browsers. Or instead, you create something unbelievably great that runs on 70%. What would you pick? And no, you absolutely cannot have both. I spent too much time with outdated browsers to still believe there’s a way to achieve both.

I find it amazing that many people will pick the ‘okay’ product, just because the ‘open web ethics’ demand it. If you are creating a consumer point, there is absolutely no reason to go that route. None. Let me repeat that, as it is important. You should always – always focussing on creating a great product. Great in absolute terms, not relative to the browser. Achieving 15fps in a game on IE7 is not great. It’s great relative to what the browser can offer. In the case of a game, your baseline is 60fps.

You don’t want people to play your game at 15fps. “Their game is so great it only runs on latest browsers” almost definitely sound better than “Many people complain about the bad performance in their favorite browser.”. Hell, maybe your favorite reporter is using IE7. It unfortunately doesn’t stop here, as there is more harm to be done supporting outdated browsers. Progressive enhancement almost never works. I guarantee you that if IE7 is your baseline, you won’t be using all the cool features you get with, say, Firefox 7. And another project that focusses on the right compromises will always win.

So please, please, please! Focus on creating absolutely great products. Literally.

Scroller vs. Scrollability deathmatch!6

I recently had an interesting Twitter exchange with Joe Hewitt on our (as in Zynga) newly released Scroller that is somewhat in direct competition with his excellent Scrollability plugin. Both share at least one similar ambition: To give us native-feel iOS panning wherever we want it. Joe even recently blogged about how to do fast animations on iOS.

He was spot on during our Twitter exchange – his Scrollability demo performed amazingly well, while our own Scroller demo sucked hard on an iPhone 4 with iOS 4 (it was ok on iPad). Joe suggested that one has to use CSS Animations or transitions to improve framerates. He also suggested to use the “scale” transform to handle content scaling. Being an early adopter of CSS Transforms and all things shiny myself, I had my doubts about the JS animation vs. keyframe animation part. My own tests on sprite animations suggested that in fact, CSS keyframe animations were not at all faster than JS animations. In addition to that, even if there was a speed increase, there are multiple issues with implementing it that way:

  • Scroller is designed with portability in mind. It only handles logic, and does not apply values. It can therefore be used in any environment, including Canvas.
  • Zooming isn’t as easy as just putting a “scale” transform on the content and be happy. CSS Transforms do not modify the bounding box, so the content doesn’t actually become bigger. You won’t be able to pan to the edges anymore.

That being said, I was still curious on why our demo performed so bad in contrast to Scrollability’s shining examples. So I went ahead and created a couple quick mashup demos to be able to compare the context as well as functionality. First, I took Joe’s table demo, removed Scrollability and put our Scroller on it. No content or logic changes. Compare:

Holy crap! Now that both are running in the same demo and scenario, performance on both is amazingly similar on an iPhone 4 running iOS 4. Scrollability still feels a tick smoother, but both are very usable and nice. I wasn’t finished yet so I created another interesting test case, putting Scrollability into our own DOM demo. In order to be able to compare, I deactivated zooming and horizontal scrolling on both. Here are the results:

This is pretty amazing. On my iPhone 4, I found Scroller outperform Scrollability by a quite visible margin, and the margin is even greater on Chrome for Mac. At this point, I don’t have a clue as to why Scroller performs better – one possible reason might be the missing scroll indicator.

Update: I have heard from a couple people that they couldn’t confirm Scroller being faster than Scrollability on iPhone 4. After hours of debugging, I finally found out why they saw so different results. If you have the debug console in mobile Safari enabled, Scroller will always outperform Scrollability. If you have it disabled, Scrollability will outperform Scroller by a great margin. It’s completely mysterious, and it doesn’t help that Apple doesn’t give us debug info. I will continue to investigate what the issues are and update this blog post. Stay tuned.

Update 2: Obviously it had to be something silly. It wasn’t the viewport size. It wasn’t even the Scroller. It was the logging of debug values into input fields that caused a dramatic slowdown in Scroller when the debug console was not activated. I still have no clue why it was fast with debug on, but the riddle is solved. Scroller now always outperforms Scrollability.

In the end, I have learned that it’s not only the code itself that makes things fast, but the context. Only by comparing against the same context, the same markup in our demos, I was able to realistically learn about performance impacts. But as with all comparisons, take it with a good grain of salt. While Scroller and Scrollability have overlapping featuresets, there’s things that are different: Scrollability has a scroll indicator while Scroller doesn’t (it’s all logic, really), and Scroller additionally comes with zooming, both directions scrolling and some more.

And both have their respective use cases: I found it amazing how simple it was to setup Scrollability – it’s a great and easy plugin if you need simple scrolling content. For advanced scenarios such as games or more interactive content on Canvas or WebGL, I encourage you to try the Scroller. It requires more setup, but comes with greater flexiblity.

Until next time!

Music in games2

It is a little known secret outside the game industry that there’s a very good reason why you won’t hear popular music in most popular games. It’s not that game developers don’t want you to listen to great soundtracks – it’s the horrible situation with licensing. But why is that?

License holders such as the labels and associations like RIAA or GEMA think of games mostly as boxed units. This works fine for games like Singstar. You go to the store, spend cash on the game, and a (pretty big) portion of it goes back to the license holders and artists. That’s cool though, as it makes sense: After all, the game’s business value is dependent upon the music, so it is only fair a big portion is shared.

Welcome to 2011, where the largest percentage of gamers plays on Facebook, tablets and mobile phones. As much as you don’t “buy” Facebook or Google, you don’t “buy” a Facebook game. Games are becoming interesting consumer platforms more than ever. Some of them are always on, always engaging and a new layer of social interaction. The prior business model of adding music simply does not work any longer for these types of games.

Another proposal from the licensing industry has been pay as your customers listen, a.k.a the “radio model”. Yeah, pretty much what you think: Game developers are expected to pay as much for the usage of popular songs as if it would play on the radio. It’s viagra to forget that – again – the business model for radio entirely depends on the music – better music is directly proportional to more listeners! No, games are not radio channels.

Here’s a twist. Embedding popular music into games is a great privilege for the license holders and artists, not vice versa. It’s an unprecedented way to market music in highly interactive and engaging ways. They should be *dying* to get their newest music added to the most popular games on Facebook, and beg browser game developers to develop new interesting ways to integrate the music into their game worlds in smart ways. Hell, if I was running a label, I would not only give them my music for free, I would *pay* them.

Sprite Animations on CSS Transitions, revisited12

…and I’m back! Not content with the hacky previous solution of animating spritesheets in CSS and with the help of the fabulous Doug Neiner, I created four individual ways to animate sprites via CSS3 Animations, which each of them having their own advantages. There goes the list:

background, stepped

This is by far the cleanest solution, and the solution I recommend for desktop browsers. It animates the background position, doesn’t involve any hacks and uses the new step-start easing property to disable easing, which, I found out, is implement in latest WebKit and Chrome already. Woot!

background, eased

Unfortunately, not all browsers (i.e. Safari, on both iPhone and iPad at least) support the step easing functions at this point, so a workaround is still required here. However, Doug Neiner found a better hack than me, which goes like this: Instead of doing the crappy resizing, simply double the keyframes and have a very low percentage keyframe before every change, which tricks out the easing. I recommend reading the source of the demo to get a clear picture. It rocks, and works reliably!

translated, stepped

I really really wanted to see if it was possible to accelerate animations on browser that feature hardware acceleration on CSS Transforms – in particular, iPad and iPhone’s Safari. As it turns out, it is possible – simply add an inner <div> resize it to the spritesheet full dimensions and put the background image there, set the parent div to overflow hidden and animate using translate3d(x,y,0). This, as it turns out, gives roughly a 300-400% performance increase on iPad – not bad, huh?

translated, eased

Might be obvious already – the above is great, but still looks broken on iPad. So again, we are using the brilliant little hack Doug came up with.

That’s it (harhar!), enjoy: Demo (run this on any WebKit browser)

Finally: Sprite animations implemented via CSS3 Animations12

Update: A way more awesome update with more techniques: Click me!
Warning: This is hackier and more dirty than anything I have published in recent years, and only works in WebKit. Use with extreme caution.

I’m lazy, show me the end result first!

CSS Animations are great – they are a lot less pain for the normal web developer, are rendered more smoothly as the browser can control the framerate, and even become hardware accelerated. You can animate almost everything with it and be happy. Unfortunately, 90% of animations a classical 2D game engine requires cannot be implemented via CSS Animations: yes, I’m talking about the good old sprite animations.

Sprite animations require a big spritesheet that includes all frames (I use big png’s), and for the render, only the size of a frame is shown, while the animation is done by shifting the masked image. In web development, the easiest way to implement sprite animations is by utilizing a <div> with the spritesheet set as background-image, and then shifting its background-position via JavaScript. So since background-position really is just a CSS property, CSS Animations shouldn’t be an issue, no? You bet.

CSS animations are implemented around the concept of keyframing. You define a key rule, name it, and then define keyframe rules in percentages. Within these rules, almost every CSS property is allowed (no gradients, unfortunately!). As second step, the named keyframe animation is used like a variable and placed upon another css rule, using the -webkit-animation syntax. It is important what happens now: if you create two keyframe rules at 0% and 100%, changing the background-position-x from 0px to -100px, it will create frames inbetween for a smooth transition. This is usually very nice, but it destroys the idea of sprite animating: sprite frames need to be switched from one to another instantly, not scrolled!

Frustrated as I was, I gave up and let it go for now, implementing it in JavaScript instead. Today though, I revisited CSS Animations with new knowledge and new motivation, and yes, finally found out how to do it. Let me show you how.

The markup

This is the easiest part. Just a div with a class.

<div class='animated-sprite'></div>

The basic CSS rule

We now continue with the CSS, starting to style it as if we would animate it normally via JavaScript.

div.animated-sprite {
	width: 86px;
	height: 90px;
	background-image: url(animated-sprite.png);
}

Where do the values come from? A single frame has the size 86×90, so we resize the div to exactly the size of one frame. This should therefore display the first frame of the spritesheet that we set as background.

The magic sauce

This still looks to simple, right? Well, lets spice it up! We now change our markup to the following:

div.animated-sprite {
	width: 1px;
	height: 1px;
	background-image: url(animated-sprite.png);
	background-size: 7px 6px;
	-webkit-transform: scaleX(86) scaleY(90);
	-webkit-transform-origin: top left;
}

For those of you who end up thinking “Holy crap, what is he thinking?!”, let me detail what this is doing:

  1. scale the element down to 1px by 1px
  2. scale down the background in proportion using CSS3′s background-size
  3. scale it up again using 2D CSS transforms
  4. set the transform origin to the top left corner so the scaling doesn’t move the element

If you closely followed the instructions, you will notice that the element looks exactly the same than before, with the much simpler rule above. Usually, I was expecting to see a blurred pixel – after all, you are scaling up a one pixel element! Luckily, 2D CSS transforms are smart enough to remap the actual image resource after the scale, realizing that the original image is of a higher resolution, and then displaying it with the highest resolution possible. Now what is this good for, you might ask? Read on.

Tricking out the easing

The whole purpose of resizing the element to the absolute minimum the browser can handle, is that it’s also the absolute minimum CSS transforms’ easing can handle. This means if you animate something via keyframes by a single pixel, it cannot tween inbetween, meaning it cannot do a subpixel transition. This is good, as we just fooled the CSS transform engine: We can now animate the background position by single pixels, which the result ending up shifting the background spritesheet by a whole frame, without any transition inbetween. Keyframes become frames.

Creating the final keyframe animation

Now all that’s left for us is the actual CSS animation that we still need to create. This is painful by hand, so I created a little helper function that generates them on the fly. Enjoy:

function generateKeyframeAnimation(animationName, frameWidth, frameHeight, spriteWidth, spriteHeight, frames) {
	var css = '@-webkit-keyframes ''+animationName+'' {', step = 100 / (frames-1), horizontalFrames = spriteWidth/frameWidth;
	for (var i=0; i < frames; i++) {
		css += 'n'+((i) * step)+'% { background-position: '+(-i)+'px '+(-Math.floor(i / horizontalFrames))+'px; }';
	};
	return css+'n}';
}

But in the appropriate values and as animationName any name the animation should receive, and it will return something like this:

@-webkit-keyframes 'animated-sprite' {
	0% { background-position: 0px 0px; }
	14.285714285714286% { background-position: -1px 0px; }
	28.571428571428573% { background-position: -2px 0px; }
	42.85714285714286% { background-position: -3px 0px; }
	57.142857142857146% { background-position: -4px 0px; }
	71.42857142857143% { background-position: -5px 0px; }
	85.71428571428572% { background-position: -6px 0px; }
	100% { background-position: -7px 0px; }
}

Take this and put it in your CSS, then add a couple animation rules to the actual animated-sprite rule, so the end result will look like this:

div.animated-sprite {
	width: 1px;
	height: 1px;
	background-image: url(animated-sprite.png);
	background-size: 7px 6px;
	-webkit-transform: scaleX(86) scaleY(90);
	-webkit-transform-origin: top left;

	-webkit-animation-name: 'animated-sprite';
	-webkit-animation-duration: 1s;
	-webkit-animation-timing-function: linear;
	-webkit-animation-iteration-count: infinite;
}

This will tell CSS to map the animation we just created to this element, have it executed in one second (I’m doing 8 fps here), and playback should be linear and looping.

The final result, with some caveats

That’s it, really! We tricked out CSS transforms, had keyframes become frames, and have awesome sprite animations running with nothing but pure CSS. Neat, huh?

Final running demo

However, there are a couple of caveats, which is why this is very experimental, and not meant for any production use:

  • This will not be faster than the JavaScript animation method. I was thinking to speed it up through hardware accelerated 3D transforms, but scale3D unfortunately ignores the image resource, and actually produces a big blurred pixel when scaling the 1px image up again.
  • There’s a little bump at the end/start of every loop. I don’t know why, but I bet it’s related to how CSS transforms jump from 100% to 0%. Might be fixable.
  • This requires tons of generated CSS for larger animations, not really practical. A JavaScript engine can compute steps on the fly.
  • This is only a temporary workaround, until all browser implement the ‘step’ easing property. It will automatically disable the easing, with no crazy workarounds required.

Enjoy, until next time!

Why you should stop using dialogs12

I posted a tweet viagraday about why I think you should not use dialogs, and it seems people are still pretty skeptical, so I felt the urge to elaborate a bit.

So why do I dislike dialogs that much? After all, I have written countless JavaScript implementations of dialogs and overlays in the past, and we have one in jQuery UI for quite some time now (and while I’m starting with my rant, keep in mind this is purely my personal opinion, not the one of the jQuery UI team). Diving into the browser and social game world, it slowly became crystal clear to me what the issue really is about.

Dialogs invite you to be lazy.
Solving the issue of navigation and interaction within your information architecture of your website or game is a very hard challenge. “Uh, if I just had a good place to put that invitation feature somewhere in my UI..”. If you are a web designer or developer, chances are you shared a similar frustration before. But wait! Thankfully, there’s an easy way out. Just take the feature and pack it in a dialog! That way, we don’t have to clutter our UI and can hide certain features, bring them up when needed. …right ?

Let’s turn it the other way around to see what’s going on here. By using a dialog, lightbox or overlay, you are effectively cheating. The content in your dialog will not fit into your centralized user experience, and remains disconnected. Your users have a harder time to relate to features in lightboxes. Even worse, modal dialogs often break one of the most important rules of great user experience: Never block the interaction of your users an your product.

Dialogs are the new popups. So the next time you are setting up a dialog, take a break and think about a better integration.