Every game player is familiar with the usual input devices that the games industry throws out no matter how revolutionary the game console or computer being launched is; we’ve all used keyboard, mouse, joystick, or joy pad. A few buttons moved around, dual analog sticks, shoulder buttons, poorly implemented vibration “feedback”–not much new ever gets traction.

But every once in a while a truly unique input device, either co-opted from another industry or a completely new invention, comes along that makes people sit up and take notice. Even with the computer industry in the doldrums, the past year has seen the launch of a half-dozen different input devices, and if the devices I saw in Kentia Hall at this year’s Electronic Entertainment Expo was any indication, it looks like the next year or two in games is going to be very interesting.

The game controllers there sparked some ideas of how they could be used for creating innovative games, and bringing back a few classic toys of our youth. Some of these concepts may be little more than interesting distractions, but I hope this article makes a few game designers–as well as the controller manufacturers–sit up and start thinking about designing some cool games to go along with them.

Tilt and Nudge

Digital accelerometers that can detect angle and acceleration have been around for decades but it is only in the past three or four years that small, cheap, and accurate models have become available, due mainly to the automotive industry. The Nintendo Gameboy (Color and Advance) is the only game platform that has ever made use of a tilt sensor in commercial games–Kirby’s Tilt & Tumble and Diddy Kong Pilot–that offered some unique game play experiences for the small screen. Recently IBM has been heavily promoting their “hard drive” parking technology which makes use of an accelerometer to determine if the laptop is being violently shaken. Toshiba has similar technology in the Portege m200/m205 line of laptops that can sense tilt in two dimensions.

It’s a shame that Toshiba and IBM aren’t really pushing the accelerometer technology in their laptops as a possible game input device. Toshiba’s laptop ships with a small utility that lets you activate the Windows Start Menu when the machine is in tablet mode, activated by violently shaking the laptop like an Etch-A-Sketch. It’s also possible to move the mouse in an application by tilting the machine, a rather useless application of some very neat technology.

Since reverse engineering the undocumented DLLs that communicate with the hardware I’ve contemplated how to use the input devices in my own small game demos. I’ll write more about this in the future, but let me share a few of the ideas I’ve had with you. Toshiba picked the most obvious game play mechanic, which is a marble on a flat plane. The Toshiba accelerometer utility software, when started with no parameters will attempt to launch Labyrinth, a marble rolling game found in the Microsoft Plus Pack for Windows XP. The implementation falls flat because the mouse controls the angle of the board under the marble, which responds cumulatively to the linear movement of the mouse, rather than the actual angle of the laptop.

Had Toshiba spent a little longer creating a stand-alone product, such as a clone of Atari’s Marble Madness using the laptop angle to control the acceleration of the ball, it would have made for a much more interesting and playable game.

Figure 1: The Marble Madness arcade game of yesteryear.

I also recall a children’s toy that consists of a flat, enclosed space, often with an interesting picture or simple maze as a background. Around this area, strategically placed to be frustrating, are a number of shallow holes, and it is the player’s task to maneuver many small ball bearings into each of those holes so that the holes are all simultaneously occupied. With Toshiba’s tablet PC accelerometers this could be achieved by leaning, jostling, and jiggering the machine. (I’m not sure how good this would be for the hard drive though.)

Virtual Arm Wrestling. I’m contemplating a two-player Pong game, either played on a single tablet PC with each player “fighting” the other for control of the horizontal accelerometer by attempting to tilt the laptop, and hence the vertical accelerometer, in their own direction. Pong expressed as a game of virtual arm wrestling, as it were.

The only problem I see with this game is that a boisterous or over-zealous player, using excessive brute force, could either cause the laptop to be dropped or quite literally snapped in two halves. The stresses on the LCD screen might be too great.

Perhaps a gentler, more pedestrian networked version of Pong played over Bluetooth while in a boring Monday morning meeting would be cool. Both opponents could casually tilt their Tablet PC back and forth as they surreptitiously play a quiet game of Pong. (Perhaps even an ad-hoc pair of mixed doubles with the cutie from accounting?)

The Eagle Has Landed. Marble Madness and Pong introduce basic physics but there was once a game called Thrust.

Figure 2: Thrust’s use of physics in the gameplay would naturally lend itself to being controlled by physics–through the tilt sensors in some laptops.

Thrust was incredibly popular in Europe during the 8-bit and 16-bit era. It involved the player navigating a spaceship through narrow caverns, avoiding gun emplacements, and picking up weighted cargo that swings like a pendulum at the end of a towline. All this while fighting a rapidly dwindling fuel supply and the merciless pull of gravity–it was Atari’s Lunar Lander on steroids.

The absolute inclination and rate of change could let the player fly, not through simple on/off digital switches, but with fine-grained analog movements. It would certainly make for an interesting game with the player needing to twist the Tablet PC in their hands to make the thrusters work. The screen might not even have a true “up” direction as far as the player is concerned.

Newton’s Apple. Indie Game Jam #2, the fruits of which were on display at the 2004 Game Developer Conference in San Jose, CA presented quite a few interesting physics “games”, one of which was Balance Board by Doug Church. A human figure attempts to balance on a beam that rests on a circular roller. To get the balance just right emoted a Zen-like experience. Each adjustment of the analog control stick makes the humanoid figure lean one way or the other. This experimental game, reworked to utilize accelerometers, would evoke similar mechanics as exhibited by the kinetic, balancing executive desk toys popular in the ’80s.

Water, water, everywhere. Accelerometers and the physics aspect bring to mind executive toys and puzzles, not games in the literal sense, just simple, interesting software toys that waste time and battery life. Newton’s Cradle, wire tight-rope walkers, sand pictures, and plastic fish aquariums are the ’80s executive desk tchotchke. And as geeks we’ve all bought them, lusted after them, or just been plain mesmerized by them. With a reasonably robust fluid dynamics simulation coupled to a rigid body system it would be possible to create an interesting executive toy consisting of water flowing around objects, creating never-repeating patterns similar to the weird seesaw water toys that rock back and forth.

The fluid dynamics simulation, controlled by the accelerometers, could also be used to create sand pictures–fine-grained sand suspended in a viscous fluid pressed between two sheets of glass–that the user tilts back and forth to create interesting art.

Buzzwords, Fish and Ouija! OK, I know most of these concepts are not of full-fledged games, but the point is to draw inspiration from them, and apply them in similar ways to the kind of games you’re used to making money from. Here are a few more quick ideas that I came up with when pondering fun demos to write that make use of angle sensors and accelerometers.

Can’t decide who to outsource or what to have for lunch? Magic 8-ball can help! A digital version, using the laptop motion sensors could provide the shake necessary to initiate the 8-ball’s answer. This is a software toy I’ve already implemented on the Toshiba, along with customized lists of “power buzzwords” for using in meetings that float to the surface with each shake. And consider, for a moment, the possibility of using the same tilt sensors as a seed for random numbers…

A virtual fish tank similar to the popular 3D screen saver, but with the ability to torture the fish by shaking the tank and having water slosh around inside. An excellent physics demo! A bit sadistic, like a virtual ant farm that can be shook up. Or a Mod for SimCity, where the quakes are real.

A Tablet PC Etch-a-sketch is a simple idea. Draw straight black lines on a silver background with the pen, introducing slightly random perturbations at each corner so that it looks like an etch-a-sktech. Similarly, shaking the Tablet PC upside down would clear the screen.

With the earlier multiplayer Pong idea I briefly explored a variation with two people holding on to a single Tablet PC. It could be made even weirder by having multiple people hold the tablet PC and the accelerometers could be used to control a virtual Ouija board. Speak to the dead (i.e. the marketing drone boring you to death in the Monday morning meeting) through your Toshiba tablet PC.

Fight The Flab

The Kentia Hall at E3 is always worth a visit, referred to by some as the “cheap seats”, many people never give it a look, but within this underground cavern lurks many an interesting bauble. Along with the usual vendors promoting their CD repair device or service, CD anti-piracy technology, CD duplication, and various international game developers and publishers you can find a variety of strange hardware and software some of which defies categorization. I believe that this year I spent more time than usual in Kentia Hall because of the Classic Gaming Expo exhibition–As a side note, the Classic Gaming Expo (www.cgexpo.com), formerly known as “World of Atari”, is well worth going to if you can make it to San Jose in August and have an interest in the history of video games–so I spent more time playing with the new hardware than I normally would.

Figure 3: The Exertris exercise/game controller.

A number of companies, Exertris and Electricspin being among them, were promoting exercise equipment wired up as alternate input devices. Electricspin manufactures the “Golf Launchpad” that lets golfers practice while playing Microsoft Golf or Electronic Arts’ Tiger Woods PGA Tour. Hooked up to a computer via USB, the device features a tethered golf ball, placed either on a tee or on the fake grass, replacing the mouse as the main control. Sensors track the direction & speed of the ball when hit. The exciting part for golfers is the ability to use their own set of clubs, and while I no longer posses my own set, from my limited experience of playing golf five days a week during my misspent teenage years, this is about as close as any computer golf game has gotten to the real thing, without all the tedious walking and hunting for the ball that got hooked in to the rough.

Across the way from Electricspin was Exertris, promoting its elliptical cross-trainer and stationary bikes, both of which have been tried at various times in the past and usually result in failure. My local gym has a number of reclined bicycle exercise machines with Wintel computers running a web browser and audio player on a touch-screen interface. The exercise equipment in that case was just a “passive” part of the system, where Exertris hopes to differ is that their games require you to peddle faster or harder at certain points of the game, making the equipment itself an engaging experience. Currently the Exertris system ships with four games, Solitaire and Space Tripper among them, with the exercise portion integrated in different forms for each game, with bursts of speed required to fire weapons or move cards, for instance.

I believe the failure of most exercise equipment integration is mainly due to two circumstances, division of activity–when I exercise, I’m exercising, when I’m gaming, I’m gaming–and non-compelling software. Exertris and Electricspin are the first two companies to actually start getting close to the ideal of marrying the two.

Everquest On Steroids. An idea I’ve always toyed with is marrying exercise equipment, e.g. stationary bike or treadmill with an MMORPG, e.g. EverQuest. Whilst exercising the player’s in-game avatar would see a boost in their statistics, bestowing physical or experiential benefits on to the avatar. Given the addictive nature of both EverQuest (EverCrack) and the compulsive makeup of many of the players, it might just turn out that any MMORPG that was far-sighted enough to implement this as a feature would have the fittest player-base in existence. I wonder how long it would be before someone filed a lawsuit claiming that their heartache was brought on by exercising too much.

Swinging Swords and Punches

As a geek I’ve played with the odd plastic sword or Nerf sword in my time. As a practitioner of various martial arts with a little kendo blended in I’ve swung a few real swords too, which is why I’m pretty excited by an emerging trend that’s occurring right now. Two distinct companies, Hori Japan and Gametrak, have begun manufacturing martial arts weapons that plug in to the USB port on a Sony PlayStation 2.

The Katana controller from Hori Japan is an official tie-in with the third installment of the Onimusha games, approximately 40″ long and making use of weight sensors in the hilt, similar to baseball bat controllers. Because the sword is battery powered with a range of around 15′, it allows players to make some fancy moves. Combined with Sony’s EyeToy I could imagine both of these gadgets being used for practicing kata (standard sets of movements found in martial arts that mimic fighting with one or more opponents) and other large area fighting games. It is only a matter of time.

Figure 4: The Katana sword controller.

The same technology in the Katana controller would also work in a tennis racket, benefiting such games as Sega’s Virtua Tennis. It might need to come with a warning to move all furniture out of the way so that an enthusiastic player doesn’t injure themselves against the bookcase while diving for their opponent’s high speed service. Swinging a tennis racket at high speed indoors probably isn’t such a good idea around small pets either.

Gametrak, a UK company exhibiting their wares at 2004’s Game Developer Conference in San Jose, CA, has chosen a different solution to detecting movement and positions than the Hori Katana controller. Gametrak’s controller features a weighted base that sits on the floor, and has two retractable strings attached to it, with sensors for the amount of string pulled out and the x/y angle. They demonstrated several controllers that connected to the strings, including a pair of gloves, similar to the fingerless type worn during weight lifting, a sword controller, and a golf club. The gloves allow the player to box with their virtual opponent in a first-person fighting game or swing a sword against an opponent. I’m not sure of the wisdom of first-person fighting games, other than first-person shooters and a few online role-playing games, first-person doesn’t work too well when you have limited peripheral vision. I see this input hardware failing on so many levels unfortunately.

Figure 5: The base of the Gametrak controller, to which various accessories, such as punching gloves and swords, can be attached.

Similar input devices utilizing infrared that offer both hand gloves and foot gloves were demonstrated at this year’s E3. Plastic swords and ping-pong paddles were also available as variations on a theme. The infrared versions exhibited slow response times with angles and positioning not entirely accurate, some of the blows, if executed too fast or in rapid succession would not be registered at all and a fast ping-pong serve could be completely ignored.

Combining these technologies I can see an interactive martial arts training program, such as for Tai Chi or kendo, that showed on screen where to place your limbs, and then critiqued you on your form is a possibility. This would be quite useful for teaching kata without an instructor being constantly present.

The major issue that any alternative controller faces, especially one that requires large body movements, is one of game play tuning. Rapidly stabbing at buttons to execute a flawless combo to beat an end boss is one thing, but the human body is just not fast enough to do the same thing whilst holding a large weapon. The games really would need balancing to the controller. Much of this is first-generation technology and will take several iterations, should any of the controllers survive, to become an enjoyable experience.

Apart from the minor issues that these input peripherals face and the lackluster games demonstrating the technology the possibilities for game developers to explore new games un-tethered from the keyboard and mouse are dazzling. It’s possible to create interesting, live-action role-playing games that allow the player to really swing their sword or fists and achieve an effective workout too.

Use the Force. I’ve discussed the idea of a sword controller with a number of game playing geeks, and almost to a person they deny any interest in possessing the controller, the accompanying game, or even a Sony PlayStation 2 (or any other console for that matter in the case of PC-based gamers). The reasons are many but as always it comes down to a “killer app”. In the small hours of the morning sitting at a coffee shop in Santa Monica, California, I was kicking around game ideas with some friends and colleagues and came up with a the “killer app” that made everybody’s eyes light up. Should Lucas Arts ever put out a Star Wars game with a light saber controller that plugs in to the game console, I’m convinced that it would sell like hotcakes. The game and controller might even convince a few people to purchase a console just to play this one game.

The Eyes Have It

Francois-Yves Bertrand, a good friend of mine and ex-Activision cohort went to work for Sega in the early nineties, having designed a full-body optical motion tracking system using nothing more than a few cameras. Technically the system was brilliant but was never very popular commercially–I’ve only ever seen one in existence, a later model at the Sega GameWorks in Las Vegas, NV.

With the increase in computing power, and low cost CMOS CCD cameras Sony has re-animated this dead technology, actively promoting their EyeToy, not just with a few lackluster curio games (remember the Nintendo robot for the NES and it’s dearth of “entertainment” titles?) but with high quality, big-budget games such as snowboarding. With Sony bravely stepping in to the market a number of follow-the-leader companies are trailing along behind, bringing out identical peripherals for other consoles. I believe these other companies are really missing the point. To quote an old adage “It’s the software, stupid!” Sony has got it right; they realize that the EyeToy is just a vehicle to support their games, not the other way around.

The cheapness of Sony’s EyeToy means that it’s not unimaginable to think of games that require multiple cameras for full body motion tracking–currently the EyeToy only performs a simple 2D edge detection algorithm–or even multi-player elements. A multi-player game of Gauntlet using several EyeToys all focused on a group of people sounds appealing, a sort of multi-player cross between Twister and Gauntlet Dark Legacy. Multiple cameras also open up the idea of large play area games that map to a virtual space inside the computer.

Auxiliary Programmable Screens

It was only a couple of years ago at E3 that Massworks showed off their ID-75 LCD panel. This gadget, a solution looking for a problem, and strangely, marketed towards gamers, provided a 320×240 resolution, 4″ diagonal, programmable, touch-screen LCD panel that plugged in to an available USB port. The LCD would display custom bitmaps and act like a programmable keyboard which evoked memories of a Bosch graphic workstation from the ’80s. The Massworks ID-75 failed as a product for a number of reasons, not least of which was the nearly $300 price tag, which is a shame as it held a lot of potential for outside-of-the-box game designs. A programmable touch-screen hooked up to a computer or console provides opportunities for unique game play mechanics that a small, remote screen, exclusive to each player can be used for; two I can think of are hidden plays and fog of war. Both of these mechanics have been briefly explored previously by developers. Sega made an attempt with their Visual Memory Unit, a memory card containing a small LCD that plugged in to the regular memory card slot on a Dreamcast gamepad that allowed hidden plays during multiplayer sessions in NCAA Football. Other than this one game, the VMU was mostly used to display a logo for the game currently being played or showing “Paused” while the player went and made a sandwich. Nintendo’s Miyamoto has explored the fog of war in the new multiplayer Pacman game for the Gamecube that lets human players control individual ghosts via Gameboy Advance slave units. Each ghost can see only a partial area of the entire playfield.

This very same idea could be applied to face-to-face RTS and war games where the individual units or small groups are only able to see a small fraction of the world and must rely on information fed to them by a player at HQ to find out what’s over the next ridge. Similarly HQ has only an abstract, incomplete view of the world that is fed by information from the player-controlled units in the field. This idea of individual screens used for hidden plays/fog-of-war could be explored in collaborative game play where several players must work together, with each having access to different information on their personal screens. Competing groups would see the results of their efforts, along with the results of their competitors. Collaborators would not be able to openly discuss their information and strategy without it becoming apparent to competitors.

Putting It All Together

With so many differing low-cost input controllers available to game developers it is only a matter of time before I can enjoy a full-body motion tracking, sword-swinging, VR goggle-wearing EverQuest game?

Seriously, though, I hope this fun little stream of consciousness has provoked you to take a second look at some of the alternative controllers that are around, and consider how they might improve interaction with the games you are working on, or perhaps even design an entirely new game around one of them.

Subscribe to the RSS feed or follow me on Twitter to stay up to date!