A virtual gamepad or on-screen gamepad is an input design pattern for video games for platforms with touch screens.
Background
The most common handheld platform for independent video game developers is a pocket-size tablet or smartphone running Apple iOS or Android. Tablets that run Apple iOS include the iPod touch, iPhone (an iPod touch that makes cellular calls), and iPad (larger versions of the iPod touch). Tablets that run Android include Nexus, Droid, Galaxy, Transformer, and several other brands. Unlike handheld devices made by Sony and Nintendo, these devices are open to small independent studios founded by people who haven't moved to Austin, Boston, or Seattle to work for several years in the mainstream video game industry. But also unlike handheld devices made by Sony and Nintendo, these usually have no physical buttons for the application's use; games instead have to rely on touch sensors behind a flat sheet of glass.
A touch screen is a pointing device that reads the positions of one or more fingers in contact with the screen. The earliest pocket-size tablets, called PDAs, had a resistive sensor that detected pressure but could detect only one point at once, and they were designed for use with a stylus. Newer devices, especially since 2007, have a capacitive sensor, which can sense multiple fingers but is less precise because a finger has a much larger contact area than a stylus. Several successful video games for touch-screen devices are designed around selecting objects on the screen or pointing at places where an action is to be performed. These include "casual" games such as Cut the Rope and Words With Friends and puzzle games such as Bejeweled and Planet Puzzle League, as well as any of several strategy games involving selecting units and giving commands to them.
But not all video game genres are as suited to pointing, especially platformers, shooters, and the like, where the player controls the movement of a character on the screen. For these genres, which are traditionally associated with gamepads or joysticks, emulating a gamepad effectively is a challenge. If a game puts action buttons at fixed positions on the touch screen, the player is going to have a hard time pressing them reliably, resulting in what fuzzyfuzzyfungus has called "a frankly lousy feedbackless attempt an overlaying a touch-sensitive picture of a Nintendo control pad".[1] The player can't see the buttons while looking at the action in the center, nor can the player feel the edges of the buttons to correct for hand drift. The touch screen feels the same everywhere, so actions have to be designed to behave the same everywhere.
Sliding
An anonymous Slashdot commenter described a solution that bases everything on something the player can feel: relative motion in sliding gestures of the thumbs. Placing a thumb near the lower left or lower right corner establishes an initial point of contact (IPOC) in that corner, and movement is measured relative to that. Lifting the thumb and replacing it sets a new IPOC. A resistive touch sensor, such as that in the Nintendo DS or Archos 43, supports only one point of contact; a capacitive sensor supports separate points of contact for both thumbs.
There are two ways to interpret slides. Centered produces a current displacement from the IPOC, or zero if the displacement is closer than the radius of the dead zone. Delta produces a stream of displacements from the previously seen position in units of "mickeys", much like the trackpad of a laptop computer. Normally, something simulating a joystick, a directional pad, or a set of buttons would use centered, while something simulating a mouse or trackball would use delta. Optionally, delta can use a "kinetic" interpretation that keeps returning mickeys after the thumb is lifted, based on the last seen velocity, until the velocity decays away. Kinetic mode closely matches the heavy trackball of arcade machines such as Centipede and Marble Madness, and the major mobile operating systems already use it for scrolling.
Seven distinct actions can be bound to gestures at each corner:
- Tap
- Thumb is placed and released quickly
- Hold
- Thumb is held in the center dead zone for a longer period
- Slide up, slide down, slide left, slide right
- Thumb is at least a certain distance from the IPOC in the given direction.
- Slide out of bounds
- Thumb is a larger distance from the IPOC (past the outer ring in the diagram)
Most tablets also include an accelerometer, which measures the net force acting on the device. Games use this to sense tilt. This opens up five additional actions: slight tilt left or right, large tilt left or right, and shake (rapid fluctuation of force). A few games that use the accelerometer as the only input device, such as certain marble racing games, may use pitch (tilt around the X axis) up or down.
Actions used less often may be assigned to on-screen buttons at bottom center or a top corner. These may correspond to items in a previous/next ring or pause menu on a traditional gamepad, on the touch screen of a Nintendo DS or Wii U, or on the screen connected to a Wii console. Common actions to put here might include selecting items from inventory or pausing the game.
In December 2010, Microsoft submitted United States Patent Application #20120169610 for thumb gestures, despite Nintendo having previously used the method of claim 1 in two games first published in 2004: Metroid Prime Hunters First Hunt (delta) and Super Mario 64 DS (centered, but sliding out of bounds moves the IPOC).
Examples
In first- or third-person shooters:
- Left slide
- Move in direction, centered
- Left slide out
- Sprint
- Right tap
- Fire
- Right slide
- Rotate aim in direction, delta
In side-scrolling platformers:
- Left slide
- Move, centered
- Left slide out
- Sprint
- Right tap
- Attack
- Right slide up
- Jump
- Right hold
- Sprint (for consistency with overloading of B in Super Mario Bros. and other third-generation platformers)
Categories: Human-computer interaction