Javascript memory optimization and texture loading

Over the past couple of weeks I've seen a few great resources about optimizing your javascript. One is a wonderful GCD presentation by Lilli Thompson about best practices when writing code for V8 (Chrome's Javascript engine.) Although the presentation uses Chrome for it's concrete examples, much of the information is very practical for any browser.

GDC 2012: From Console to Chrome

It's a long presentation, but well worth the time. If you simply can't bring yourself to sit and listen for a bit, though, at the very least take a look through her slide deck.

The second resource is a great blog post by Ashley Gullen of Scirra about reducing the amount of time your app spends in that dreaded garbage collecting coma.

How to write low garbage real-time Javascript

I highly encourage you to go through both of those if you are a web developer. Even as an expert dev with years of experience, I'm sure you'll learn something!

Anisotropic Filtering in WebGL

On Google+ this morning Ilmari Heikkinen pointed out that support for the EXT_texture_filter_anisotropic WebGL extension landed in Webkit a little over a week ago. That's awesome, as it's one of the features that I've been waiting for essentially since I first started playing with WebGL.

[Update: To give credit where credit is due I've had several people point out that Firefox had this feature first. Guess I need to follow their commits more closely! Also, while some of the appropriate code is in Chrome I guess the feature isn't officially part of the browser yet. Apparently the fact that it works at all is something of a happy fluke. So I guess that for the time being Firefox is the only browser that actually supports this!]

I put together a really simple demo this morning to show how it works, and if you've got a recent Chrome dev build or Firefox nightly you should give it a try!

Yeah, the demo is ugly but it shows off the effect nicely. For those of you not familiar with the concept, Anisotropic Filtering is an extension of standard mip-mapping techniques that improves the quality of textures when viewed at an angle. A good example of this is the floor and ceiling of the demo above. With standard trilinear filtering (gl.LINEAR_MIPMAP_LINEAR) as the floor gets further away from you it becomes blurry pretty quickly:

By using anisotropic filtering, we alter the way that the mipmaps are queried in scenarios like this one, which can give a much crisper, cleaner image:

There is a performance penalty associated with enabling this, and when you do enable it you can specify how many samples are taken for a performance/quality tradeoff. This has become a pretty common setting available for the user to toggle in PC games, and so it's great that we're now able to take advantage of it on the web too!

Implementation

Turning this feature on is pretty simple in a supported browser. As with compressed textures the first thing we do is query the appropriate extension to see if it's available:

var ext = gl.getExtension("MOZ_EXT_texture_filter_anisotropic");

Obviously with Chrome you would query with WEBKIT_ instead of MOZ_, and eventually the prefix should go away entirely. If it returns null the extension is not supported and we move on, doomed to a life of blurry walls. If it is supported, however, it will return an object containing the required enumerations for the extension.

Since this is a filter mode, we enable per-texture just like we do bi/trilinear filtering or texture wrap modes, but with symbols from the newly queried extension:

gl.texParameterf(gl.TEXTURE_2D, ext.TEXTURE_MAX_ANISOTROPY_EXT, 4);

Here we're enabling 4x filtering (the value used by the demo) on the bound texture. Setting it to 1 effectively turns off this filtering, and your graphics card determines how high you can set it. You can query the maximum supported value like so:

var max_anisotropy = gl.getParameter(ext.MAX_TEXTURE_MAX_ANISOTROPY_EXT);

This particular API seems a little bugged right now, as I was only able to get it to return 0, but I expect that will be ironed out soon.

So, like I said, a simple API but a great one for bumping up your rendering quality!

The Gamepad API and the case of the shifting buttons

Took the opportunity to look at the Gamepad API for the first time yesterday while at the Mozilla Games Work Week. It's a simple API at it's core, and one that should be very straightforward to use. I gave a stab at it by sneaking gamepad controls into my Quake 3 demo (Have you noticed that tends to be the testbed for all my API fiddlings?) and am semi-pleased with the result. If you plug in most controllers (Xbox, PS3, etc) you should get at least be able to move with one stick and look with the other, though depending on the OS/gamepad/browser some things may not work quite as expected.

And that brings me to my grip with the Gamepad API as it exists now. It IS OS/gamepad/browser specific. My current code looks roughly like this:

for (var i = 0; i < navigator.gamepads.length; ++i) {
    var pad = navigator.gamepads[i];
    if(pad) {
        walk(pad.axes[0], pad.axes[1]);
        look(pad.axes[2], pad.axes[3]);
            
        if(pad.buttons[0]) { playerMover.jump(); }
    }
}

The basic idea here is that we loop over all the available gamepads and when we find one pull from axis 0 and 1 (hopefully left stick on your basic 360/PS3 style pad) for movement, axis 2 and 3 (right stick?) for looking around, and button 0 (???) to jump.

That's the theory, anyway. Here's what actually happens: In Chrome on my Mac the DirectInput gamepad I have returns the sticks as axis 0/1 for the left stick and 2/5 for the right. This means that looking up and down is broken. With my Gamepad-enabled Firefox build, though, the axes are actually 0/1 and 2/3. If you use the PS3 controller, the axes match up to what you would expect on both browsers, so that's nice, but the button mapping is COMPLETELY different between browsers. Firefox considers the "X" button to be button 0, while Chrome things button 0 is D-Pad up. (Firefox thinks the D-Pad is an axis, actually.) So if you're going to use gamepads on your page, um... you're kinda screwed. It's like if typing a "W" on one browser gave you a "W" and on another it gave you a "U". Madness!

There are projects out there already that try to normalize this situation. Gamepad.js and input.js are the big ones right now. Both do some checking against the browser/os/gamepad model and map them into a more sensible gamepad structure that looks a lot like XInput. (That's a good thing, by the way!) The biggest problem with these libraries is that if the library author hasn't gotten their hands on your exact controller you're out of luck. Also if you have a controller that happens to not look at home on a modern console, like an actual joystick, these libraries will do nothing for you.

I'm not trying to be down on the gamepad abstraction libs, because I actually think they're great and would encourage people to use them. What I feel is problematic is that the browsers aren't returning consistent data for the same device. If we could remove the question of OS and Browser from the equation and just focus on the device that's connected developers would be able to cope with the vast array of input devices out there with the minimal amount of pain required by dealing with such a varied field. If things stay the way they are now, we can still get everything to work, eventually, but we'll be working three times as hard for it.

So please, browser peoples, get everyone together on a conference call and all agree that you'll poll the hardware the same way. It will make our lives as developers much easier!

...in which I give updates on everything

Also known as: Why I've been so lazy lately.

So I know that theres more than a couple of you out there that have been very patiently awaiting the next post in my Building the Game series for the last couple of months. It's probably been a little frustrating to have the first 6 posts come out fairly quickly and then just abruptly cut off. For that I apologize.

The reality is that I've been buys with plenty of other, non-blog-related things lately and simply haven't been able to spend the time working on it. During the holidays I made a conscious decision to focus on family rather than code, and immediately after that I've been pretty swamped with a project from work (you know, the guys that actually pay me!) that has left me with little desire to do even more code when I get home. On top of that, I quickly realized that the where I wanted to take the bog series next would require some research into areas that I've never really poked into before (generating visible sets and such) and it's been difficult to find the time and resources to really dig into it.

In other words, I've been feeling a little burnt out. I think all programmers go through this from time to time, and I'm simply allowing myself to recover and gain back some of that all important enthusiasm.

Don't fret, this isn't my "well that was a failed experiment" post where I leave things forever unfinished. I'm going to continue the series soon, but I think I'll be back tracking a little bit and hitting some other aspects of it first before getting back into the whole level loading thing. Better animation management and player input feels like a good place to hit next, along with some retrospective tweaks to the model formats. I don't think I can give dates on when we'll see the next post, but rest assured it will be sooner than later.

Beyond the blog, however, there's been a couple of other developments worth mentioning lately. Since the contract is now signed I think it's safe to announce that I'm currently co-authoring a WebGL book! I was asked to help Diego Cantor complete a WebGL Beginners Guide for Packt Publishing, which has proven to be no small feat. I don't know anything really about potential release dates or anything like that, but rest assured that I'll be tweeting and blogging about it once I know more.

Also, I've been invited to attend Mozilla's HTML5 Games Work Week this February. Essentially, Mozilla is reaching out to people who have been active in game-related HTML5 tech (like WebGL) and is bringing them together for a week to ask "How can we make the web an awesome game platform?" This is a goal I think we can all support!

For my part, I want to try and address several concerns from the development community that I've run into either through this blog, via email, or that I've seen talked about on various other online resources. These include:

• Synchronizing game logic and rendering
• Considerations for reducing power consumption of games on mobile devices
• Content protection
• Content delivery
• Robust input handling
• High-performance audio

But I'd also love to hear what the rest of the HTML5 game dev community is concerned about! I can't promise that everything that I hear about will be addressed but if I'm hearing similar concerns from multiple people you can be sure I'll do my best to bring it up.

That's about it for now. There's a couple of conferences on the horizon that I may have a chance to speak at but nothing is confirmed right now. I'll let you know if things solidify. There's also some exciting new and upcoming bits of web tech that I'm itching to play with and do some little demos for, so expect to see some quickie proof-of-concept stuff in the coming months as well. In the meantime, I'm always available for questions by email, on twitter, or through this blog, and I've made a habit of trolling StackOverflow for WebGL questions. 

WebGL JS1K entry

As a sort of follow up to last years Itty Bitty WebGL post I decided to take a crack at actually entering the js1k competition this time around. Since the theme of the competition is "Love" I thought it would be fitting to do a minimalistic version of this awesome little "beating heart" shader.

You can see my under 1k version here.

And the not-quite-so-compressed-but-still-really-hard-to-read source code here:

c=document.getElementById('c');
c.height=300;
g=c.getContext('experimental-webgl');

b=g.createBuffer();
a=34962;
g.bindBuffer(a,b); 
g.bufferData(a,new Float32Array([0,0,2,0,0,2,0,2,2,0,2,2]),35044); 
g.vertexAttribPointer(0,2,5126,false,0,0);

p=g.createProgram();
function l(x,y){
    s=g.createShader(35633-x);
    g.shaderSource(s,y.replace(/#/g,'float ').replace(/\^/g,'vec'));
    g.compileShader(s);
    g.attachShader(p,s);
}

// See the uncompressed shader code at http://glsl.heroku.com/e#1599.2

l(0,'attribute ^2 p;void main(){gl_Position=^4(p.x-1.0,p.y-1.0,1,1);}');
l(1,'precision highp #; uniform #t;^2 r=^2(300,300);^3 h(#x,#y){#s=mod(t,1.0);s=1.0+(1.0-exp(-5.0*s)*sin(9.0*s));x*=s;y*=s;#h=abs(atan(x,y)/3.14);#d=(13.0*h-22.0*h*h+10.0*h*h*h)/(6.0-5.0*h);return mix(vec3(1.0,0,0),vec3(1.0),smoothstep(d-0.05,d,sqrt(x*x+y*y)));}void main(){^2 p=^2(-1.0+2.0*gl_FragCoord.xy/r);gl_FragColor=^4(h(p.x,p.y),1.0);}');

g.linkProgram(p);
g.useProgram(p);
g.enableVertexAttribArray(0);

u=0;
setInterval(function(){
    g.uniform1f(g.getUniformLocation(p,'t'),u+=0.02);
    g.drawArrays(4,0,6);
},16);

To be perfectly honest, I really don't care to explain the details of what's going on here. The basic setup is the same as my previous post on the subject, with the core difference being that we're now rendering a fullscreen quad. (That effectively makes this a tiny version of mr doobs GLSL Sandbox) The shader code is really just a minimal implementation of the original shader, and the only really interesting trick there is that I'm doing a string replace to simplify commonly repeated symbols like "float" and "vec".

I'm not certain that this even qualifies for the contest (since WebGL isn't enabled by default on some of the required browsers) but I had fun doing it and it was a good break from everything else going on in my life right now. (I'll be posting more on THAT shortly.)

OVERNIGHT UPDATE: So apparently I missed the line on the rules page that actually excludes WebGL from the competition (Second to last bullet under "Other things you should know"). That's a shame, but hopefully WebGL will be allowed for the next round! Doesn't matter much to me, I was just doing it for kicks anyway.

In other news, @p01 managed to cull 303 bytes out of my demo while I was sleeping which I think is deserving of a "I'm not worthy!" bow. There's some seriously clever little tricks in his version that I'm most certainly going to have to utilize the next time I try one of these! :)

Protecting WebGL content (and why you probably shouldn't)

I received an email this morning from a developer that is in a predicament that I think many WebGL developers will find themselves in pretty soon:

Do you have any experience with protecting assets in WebGL? I wrote a quick solution to protect textures during transfer - base64 encoding + concat server side and splitting + creating imgs client side. Still, using WebGL Inspector I can get all textures transferred to the graphics card. I wonder if you have any ideas on how to prevent that. I don't mind open sourcing my apps, but my clients apparently do :)

Ah, that last line is really the crux of the matter, isn't it? Most developers I know don't mind sharing their creations with the world at large, as we tend to understand that there's rarely anything sacred and secret in the lines of code we crank out, but that's not always an easy thing to convince management of. So: What's an enterprising WebGL developer to do when the boss decrees from on high "Protect our content?"

There's a couple ways to approach such a request, in my opinion. First: Do whatever you can to convince said management that you don't need to protect said content. Second: Convince them harder. Third: If all else fails, obscure the content as much as possible. I'll be talking about both approaches in todays post.

Compressed Textures in WebGL

[UPDATE: The compressed textures spec has been changing, and so the original code posted with this entry stopped running. I've since fixed the code and updated the information below. Be aware, though, that the spec may STILL be tweaked at some point!]

I gave a presentation this last Friday at WebGL Camp 4, the slides of which are online now. I had a great time, met some awesome developers, and saw a lot of things that got me really excited about the future of WebGL. I highly encourage anyone that is interested in WebGL to try and make it to WebGL Camp 6!

During my talk I was able to show what I think may be the first public demo of compressed textures in WebGL! The demo isn't terribly impressive, it simply displays a DXT5 texture loaded from a DDS file, but it shows off the required code effectively enough.

http://media.tojicode.com/webgl-samples/dds.html

(Warning: That demo will only work on machines that support DXT5 compression. That should be most desktops, but the majority of mobile or tablet devices will be out of luck! You'll also need to be running a fairly new Chrome dev channel build)

Yay! I got a textured cube on screen! Surely I'm the first person ever to do this!

Okay, yeah... it's not all that impressive. The key here is the potential that it provides. Compressed textures have been an integral part of 3D games and many other 3D applications on the desktop, console, and mobile platforms that they've become something of an invisible, pervasive optimization that everyone tends to take for granted. Up until now, however, they've been something that's been left out of WebGL (not without reason, they're tricky to get right). The fact that we're gaining the ability to use them now is, in my view, something of a benchmark of the maturity of the standard.

So what exactly do we mean by compressed texture? If you're not already familiar with the concept from a prior life as a game developer it can be a bit confusing to really grok what we're referring to and why it matters. After all, JPEGs and PNGs are compressed images, right? What's different here?