AirConsole Dev Diary: Smartphones as Controllers

Screen and Controllers of Brick Wars

Note: this blog post was also published on Gamasutra and the Making Games blog section.

When introducing AirConsole to people who’ve not heard about it, reactions are usually positive. But the one point of skepticism that we’ve read repeatedly is along these lines: “But why would I want to use a phone as a controller? Touchscreen gamepads suck!”

Now, we don’t know if the people that make these remarks have tried our games and didn’t like them or assumed the controls were subpar based on their previous experiences with touchscreen controllers, but the criticism is valid to a certain extent: When designing a control scheme for an AirConsole game, there’s a couple of important things to remember to make sure the controller feels good in the end.

Action vs Round Based

In order to find controls that make sense for AirConsole, there is one very important distinction to be made early on: Does the player have time to look at their controller or not?

In-game Controller for Tower Of Babel
One large button
Depending on the answer to that question, the approach to designing the controller changes drastically.

The difference is this: In a round based game where timing is not an issue, the player can take their time to look down at the controller screen and place their fingers where they see the buttons. That means that the buttons or interactive areas on the screen can be relatively small. A non-action game, such as Cards and Humanity, or the hidden role game Simoria, can thus easily have ten or more buttons on the controller.

Action games on the other hand, require an entirely different approach. If the player’s success in the game is based on doing things at the right time, they will want to be able to give inputs without having to look at their smartphone. For action games such as Silly Run Valley, Brick Wars or Battle Snakes, where half a second of distraction can mean “Game Over”, it is crucial that the player doesn’t need to look away from the big screen, but is able to give the right inputs blindly.

On a regular gamepad with actual buttons, this is not a problem because the player’s fingers feel where the buttons are and when they are pressed - the player receives haptic feedback. Since feeling the buttons isn’t an option on a touchscreen, the controller for an AirConsole action game needs to be kept relatively simple: buttons need to be big enough for the player to find them without looking. In our experience it works to have up to four buttons - one in each corner of the controller.

If using one to four buttons doesn’t fit the game mechanic we have in mind, there is another option: not using buttons at all. And that’s where we can use the smartphone controller to its full potential - by making use of the gyroscope and swiping.

Controls in Silly Run Valley

With the latest game I worked on, an endless runner for 1-8 people called Silly Run Valley, we tried to use these inputs as well as possible: the primary mechanic of the game is to swipe up and down as your character approaches obstacles. A secondary mechanic involves a bomb, which players can pass between each other by swiping left and right, or defuse by shaking the phone. During the game, players have no need to look at their phone, since the instructions are also displayed on-screen. The only time Silly Run Valley uses the controller to communicate is during the character selection and on the end screen, where timing is not important.

Ingame Controller for Silly Run Valley
No buttons, just space to swipe
Character Selection in Silly Run Valley
Shows character image
'Bomb view' in Silly Run Valley

The controls make the game

I don’t know how other game designers approach this, but usually, when I brainstorm game ideas, I start with a basic mechanic and then think about how to control it secondly.

When coming up with a concept for an AirConsole game however, the controls are so central to the idea that they’re usually the first thing we have to consider. If we’ve got a cool mechanic in mind, but controlling it would require a d-pad or joystick then said mechanic is probably not the best fit for AirConsole, period.

What not to do

AirConsole is not a ‘normal’ gaming console, and smartphone controllers are not normal gamepads. It is important that we as game developers remember this and don’t try to emulate a traditional gamepad with the touchscreen. We’ve put a lot of time and effort into the d-pad for our game Tic Tac Boom in order to find the most comfortable way for the player to move four ways without looking at their smartphone. Unfortunately, even with the best digital swipe-pad and joystick we were able to create, this type of movement is not ideal for AirConsole.

Quite a few games have tried to use the movement that we’re so used to from ‘regular’ games, be it with a four-way d-pad or with a joystick that allows ‘free’ movement. Sadly, it just never feels quite as awesome as the games that use more intuitive controls.

We still offer the d-pad variation and a joystick in our controls library however - and devs are very welcome to prove us wrong and find a way to really make controls like that work.


With the 24 games we currently have in our store, there are already a lot of different control schemes that use the smartphone controller in unique and creative ways - Balloon Party made a D-Pad work by splitting it up into two two-button pads controlled by two thumbs, for example, while the wonderful Grannies and Planes and Tiles of Doom use nothing but the Gyroscope.

We are looking forward to creating new awesome ways to use our smartphone controllers to their full potential - and to other developers doing the same.

• Additional Info for Developers
• AirConsole Game Developer Competition
Next PostNewer Post Previous PostOlder Post Home


  1. Tiles of doom uses accelerometer as well something that is rarely used in the console realm but suits air console.

    1. Good point! I had written the first draft of this before the Global Game Jam, when Grannies and Planes was the only gyroscope-only game we had. :)

  2. That is the article I was waiting for. =)

    I knew a conventional controller won't do it. It just don't feel the same, obviously. I tried something a little different but I was thinking about the gyroscope and I guess I was waiting for you guys to say that it is ok. =P

    1. Of course that's ok! :D
      Yeah, it took us a moment to fully acknowledge these facts ourselves - The first games on AirConsole were also D-Pad based.

  3. Great post!

    Designing for the smartphone as a controller turned out to be one of the biggest challenges we had for our game. We had a concept for robot battles, but the controller design process was iterate->iterate->iterate->iterate until we had playable game. We tried the following:

    1. D-pad with proportional control, sending D-pad position to the screen every 0.1 seconds. This turned out to be completely unplayable due to lag and the fact that your thumb was never where you thought it was.
    2. D-pad with binary control (up-down-left-right-diagonal), sending data every 0.1 seconds. This was playable, but suffered from a ton of lag.
    3. D-pad with binary control, sending data only when the buttons changed. This was WAY more playable, but the buttons were too small, and it was hard to tell where your thumb was.
    4. Left Thumb: up/down. Right Thumb: left/right. This worked really well, but we lost the ability to control primary/secondary weapons. We then had to redesign the game mechanics so that the primary weapon was always on. We still had no way to do a secondary weapon, and we needed a way for people to self-destruct if they had lost all mobility.
    5. Left thumb: forward/self-destruct. Right thumb: left/right. In order to get this to work, we had to totally change the mechanics to eliminate the ability to drive backward, and add an "automatic flipping" feature to the physics engine so that players could always get out of a bad situation.

    We went with iteration number 5. Self-destructing is so much more fun than driving backward. If we had physical console controllers, we would not have needed to go through this process, but because we were forced to iterate through the controller design, I think that we had a game that was a whole lot more fun in the end.