The new era of motion controlled gaming ushers in a new set of possibilities for gaming, and a new set of challenges for developers. The arrival of Tony Hawk: Ride, and its subsequent panning by the press (except the first couple of reviews, but that’s a rant for another day), tell us a lot about the problems that it can bring. I should preface this by saying that it’s not my intention to criticise the developers of Tony Hawk: Ride. In fact, I have a lot of sympathy for them, given the nature of the challenge they were up against.

From an end-user perspective, the main challenge is that skateboarding is hard.

Damn hard. Just ask anyone who has tried it. As such, simulating it was always going to be a complicated endeavour. Unlike playing the guitar, which was able to be successfully reduced to a simplified representation of a single ‘strum bar’ and five buttons, simplifying the act of skateboarding was never going to be so easy. While the basic concepts of guitar playing (i.e. holding the right note while strumming at the right time) are present in Guitar Hero games, almost every single element of skateboarding had to be reduced in Tony Hawk: Ride to the point of barely resembling its real life action. It would seem that manuals are about the only thing to survive intact, though obviously still much easier than on a real board.

Steering is also similar, yet apparently unresponsive enough that it barely counts. Ollie’s don’t leave the ground, grabs don’t have to touch the board, tilt and flick tricks don’t require you to actually leave the board and land back on it, etc. What you’re left with then, is a game where you stand on a plastic controller, tilt it up and flick it a bit at the right times, combined with a bit of hand waving. Perhaps they should have started with snowboarding.

Delving a little deeper into why the game turned out this way offers us a better look at both why it turned out this way and what this means for other games that attempt similar ‘motion sensing’ controls.

Problem #1: Acceleration isn’t motion

Systems like the board controller, wiimote and iphone aren’t based on ‘motion sensors’ in the dictionary definition of ‘motion’. What they actually measure is acceleration. As a result, constant velocity looks the same as being stationary, and orientation (which is only measurable due to the downward acceleration of gravity) can only be reliably detected while the device isn’t moving. Attempts to measure acceleration and orientation simultaneously will lead to all kinds of errors. An observant physics student might tell you that velocity and position can be determined from acceleration and a known starting point, but the accuracy of these devices is far too low for that to work in practice.

This problem has at least a partial solution: Adding a gyrometer like in the wii motion plus helps remove the burden of using the accelerometer for two things at once, but it doesn’t solve some of the other problems that arise.

Problem #2: Player one’s home run attempt is player two’s bunt

Two players pressing the X button on a gamepad will do it in fairly identical ways. Two players pressing down an analog trigger might do it at different speeds, but it’s still a one dimensional value with well-defined limits. From a code point of view, it’s still easy to deal with. But take that over to a three dimensional motion-based system and it’s a whole different ball game.

Tell the two players to perform some kind of complex physical action and the results can be dramatically different. They’ll move at different speeds (sometimes beyond the limits of what your hardware can detect), for different amounts of time, and with somewhat different orientations. A few years ago, no one ever had to wonder “Am I holding the controller level while playing this game?” Now a vital section of code might not work reliably if you’re not. Welcome to the future.

The obvious solution to the above problem is a fairly simple one; you simply make it so that the activation of any particular action is extremely tolerant. That is to say, you make it so that almost anything works. If anything which involves twisting a skateboard to the left counts as a flick trick, then most of the time you’ll get what the player is trying to do. Unfortunately, you’ve essentially turned a series of analog values into a single digital value: either you did the trick or you didn’t. In which case, you may as well have just pressed a button, which would have the added benefit that it works all the time. And so, ironically, where EA’s Skate gave the player more control over their tricks with a standard game controller, the new Tony Hawk game asks you to buy an expensive peripheral in order to have even less control than the original Tony Hawk's Pro Skater. By now, you may be thinking that this might have been a bad idea from the beginning. But there’s more...

Problem #3: Gesture overlap

Not only does that solution merely hide the problem, it creates another one. When your tolerances for what triggers a given action get too high, they start to limit the amount of actions you can have, and in the worst case can even start to overlap, causing an attempt at one action to accidentally trigger another. In theory, differentiating between two tricks that rotate the board around different axes should be fairly simple. In practice, nothing ever happens in only one direction or axis. Unless you hold a motion controller flat and move it in a perfectly straight line, the data being fed into the system will be some combination of X, Y and Z, and never just one. The software then has to take those numbers and try and deduce the actual motion that triggered them. Considering the fact that rotation is being inferred from acceleration, gravity and centrifugal forces retrieved from imprecise accelerometers, it’s not surprising that there’s a lot of guesswork involved and that occasionally the guess is wrong.

One of the ways this guess can be made more accurate is to delay activating the action until a few more frames of data have come in. What seemed like a small rotation around one axis might have just been leading into a big rotation around another, and if we wait until the controller seems to have slowed below a certain speed, it’s possible to get a better picture of what actually happened. But, as you may have guessed, this leads to our next problem…

Problem #4: Gesture triggering

Dedicated PC gamers are notorious for disliking wireless mice, due to a perceived lag between their movement and the actual input registering in the game. While I can’t claim to have experienced this myself, it says a lot about how important it is that a game responds to the user as quickly as possible.

So what do you think happens if a game based on motion controls were to take an extra few frames to try and make a more educated guess as to what action you just performed? If you answered “a game where actions seem to take too long to register”, you’ve hit the nail on the head.

This difference between player input and onscreen action is even worse in a motion controlled game due to the extra time it takes a player to perform the movement. Ideally, the frame you decide to take action is the one in which your ingame character starts moving. In the case of something like performing an ollie on a skateboard, this means you want the tail of the board to start moving towards the ground as soon as you think about doing it. Even in a traditional button-based skateboarding game, this is unlikely to be the case, because it’ll be a few frames before you actually hit the button, the game’s input system will be a couple of frames behind the graphics and the game can’t even start the animation required to do the ollie until you press the button, so the actual jump probably won’t happen for at least 1/3 of a second. If you then replace the button press, which most gamers could do very quickly, with a physical action that takes a decent portion of a second, then the actual triggering of the action is even further behind. If you were to wait until the players movement is finishing before even starting the ingame animation, then what’s happening onscreen will seem so far behind what the player is doing that they might think the game is broken.

I’d wager there’s even a psychological element to it that makes it seem even worse, since we expect the game to respond as soon as we start a physical action more than we expect it to respond to the first touch of a button, since there’s a closer physical connection between the actions.

Put it all together:

Combine imprecise controls, large tolerances for differences in player motion, action triggers that can overlap and large delays in processing input, and what’s the end result? You’d probably end up with a game where the user doesn’t feel in control, where user actions are limited and sometimes result in the wrong ingame action, and where everything feels unresponsive. In short, you’d get what critics are saying about Tony Hawk: Ride. Contrary to what many of them are saying though, it’d largely be the fault of the hardware, and hence the original concept, rather than just poor software. It has been suggested that Tony Hawk himself tried to convince Activision that the best way to reinvigorate their formerly great series was to go for a skateboard peripheral and that the technology was finally ready for such a thing. Turns out it wasn’t, at least not the way they did it.

What should they have done?

Firstly, they probably shouldn’t have made the game at all. Given the over-simplification of skateboarding mentioned at the start, it should have made the publisher a bit hesitant to put so much money into the project to begin with. Even if everything in the game worked exactly as the promo videos suggested it would, they were never going to get anything with enough depth that it would have been fun for more than a couple of hours. Just how long do you expect people to play a skateboarding game in this day and age where you can’t even choose which trick you’re doing?

But let’s assume they wanted to make such a game anyway. The most obvious improvement would have been to not rely on accelerometers for a game based primarily on rotation. Steering, ollieing, manuals, flick tricks and tilt tricks are all about rotating the board, not moving it. Putting in gyro sensors, as in the wii motion plus, would have been a good place to start. They would have needed a high degree of precision of course, and may not have come cheap, but when you’re already asking people to pay double the price of a standard game, a little more to make it work properly isn’t asking that much. Ultimately though, most of the problems would still be insurmountable.

What does this mean for the future?

In the short term, ‘motion-sensing’ games will have to be based around these limitations. They will need to tolerate imprecision, have over-simplified inputs (Wii Sports anyone?) and either have a limited range of actions and/or rely on other inputs (buttons, triggers and analog sticks) for most of their actions.

The motion plus attachment helps, as it enables more actions to be read directly rather than relying on a coder having to ‘figure out’ what you just did. But it can’t solve most of the problems inherent in motion controlled games. 1:1 gameplay will continue to be something that either isn’t feasible or isn’t desirable in many cases. Translation-based (as opposed to rotation-based) actions will still have to be limited to a small subset of easily-distinguishable motions. And there will always be some degree of lag between performing the motion and seeing the result in game.

Sony’s upcoming motion controller (still without a name as I write this) looks to be another step up. In addition to the acceleration and rotation currently measured on the Wii, it also uses a camera to get a 3D position of the controller in space. Like the motion plus, this doesn’t just add precision, it also takes work away from accelerometers. Why try and derive motion from acceleration when your camera is tracking exactly where the controller is moving?

Microsoft’s Project Natal takes this concept to its extreme. Why use a controller at all if you can make a camera that can track your entire body position in 3D? While this is probably the most exciting development, it doesn’t solve the lag problem, or the limited range of actions problem, as hinted at by Traveller’s Tales company director Jon Burton: “[Natal is] exceedingly clever, but the lag on the input and lack of physical buttons is really going to restrict the kind of games that can be done with it”

In my opinion, the best option for the near future (especially when you consider the limited penetration Natal will have, given that there are over 35 million Xbox 360’s out there that don’t have one yet) will be for games to use a standard 360 gamepad and utilise Natal for a very limited range of non-precise actions. To give some examples, in an FPS a punching motion could be a melee strike, a physical lean could look around corners and helping a wounded teammate back to their feet could be done by a reaching down/pulling up motion. Simple on/off actions that probably won’t ruin the experience if they take an extra few frames to register, but might help the game feel a little more immersive. But not the kind of complex, all-encompassing virtual reality gaming we all imagine playing some day. That day is still a long way off.

---

Robert Green (aka 'martian') is an ex-Brit who's been playing games since they came on tapes. By day, he makes games at Wellington-based development studio Sidhe Interactive. (Please note however that the views expressed in this article are his personal opinions and do not necessarily represent those of his employer.)