Screen and Controllers of Brick Wars

Note: this blog post was also published on Gamasutra and the Making Games blog section.

When introducing AirConsole to people who’ve not heard about it, reactions are usually positive. But the one point of skepticism that we’ve read repeatedly is along these lines: “But why would I want to use a phone as a controller? Touchscreen gamepads suck!”

Now, we don’t know if the people that make these remarks have tried our games and didn’t like them or assumed the controls were subpar based on their previous experiences with touchscreen controllers, but the criticism is valid to a certain extent: When designing a control scheme for an AirConsole game, there’s a couple of important things to remember to make sure the controller feels good in the end.

Action vs Round Based

In order to find controls that make sense for AirConsole, there is one very important distinction to be made early on: Does the player have time to look at their controller or not?

In-game Controller for Tower Of Babel
One large button
Depending on the answer to that question, the approach to designing the controller changes drastically.

The difference is this: In a round based game where timing is not an issue, the player can take their time to look down at the controller screen and place their fingers where they see the buttons. That means that the buttons or interactive areas on the screen can be relatively small. A non-action game, such as Cards and Humanity, or the hidden role game Simoria, can thus easily have ten or more buttons on the controller.

Action games on the other hand, require an entirely different approach. If the player’s success in the game is based on doing things at the right time, they will want to be able to give inputs without having to look at their smartphone. For action games such as Silly Run Valley, Brick Wars or Battle Snakes, where half a second of distraction can mean “Game Over”, it is crucial that the player doesn’t need to look away from the big screen, but is able to give the right inputs blindly.

On a regular gamepad with actual buttons, this is not a problem because the player’s fingers feel where the buttons are and when they are pressed - the player receives haptic feedback. Since feeling the buttons isn’t an option on a touchscreen, the controller for an AirConsole action game needs to be kept relatively simple: buttons need to be big enough for the player to find them without looking. In our experience it works to have up to four buttons - one in each corner of the controller.

If using one to four buttons doesn’t fit the game mechanic we have in mind, there is another option: not using buttons at all. And that’s where we can use the smartphone controller to its full potential - by making use of the gyroscope and swiping.

Controls in Silly Run Valley

With the latest game I worked on, an endless runner for 1-8 people called Silly Run Valley, we tried to use these inputs as well as possible: the primary mechanic of the game is to swipe up and down as your character approaches obstacles. A secondary mechanic involves a bomb, which players can pass between each other by swiping left and right, or defuse by shaking the phone. During the game, players have no need to look at their phone, since the instructions are also displayed on-screen. The only time Silly Run Valley uses the controller to communicate is during the character selection and on the end screen, where timing is not important.

Ingame Controller for Silly Run Valley
No buttons, just space to swipe
Character Selection in Silly Run Valley
Shows character image
'Bomb view' in Silly Run Valley

The controls make the game

I don’t know how other game designers approach this, but usually, when I brainstorm game ideas, I start with a basic mechanic and then think about how to control it secondly.

When coming up with a concept for an AirConsole game however, the controls are so central to the idea that they’re usually the first thing we have to consider. If we’ve got a cool mechanic in mind, but controlling it would require a d-pad or joystick then said mechanic is probably not the best fit for AirConsole, period.

What not to do

AirConsole is not a ‘normal’ gaming console, and smartphone controllers are not normal gamepads. It is important that we as game developers remember this and don’t try to emulate a traditional gamepad with the touchscreen. We’ve put a lot of time and effort into the d-pad for our game Tic Tac Boom in order to find the most comfortable way for the player to move four ways without looking at their smartphone. Unfortunately, even with the best digital swipe-pad and joystick we were able to create, this type of movement is not ideal for AirConsole.

Quite a few games have tried to use the movement that we’re so used to from ‘regular’ games, be it with a four-way d-pad or with a joystick that allows ‘free’ movement. Sadly, it just never feels quite as awesome as the games that use more intuitive controls.

We still offer the d-pad variation and a joystick in our controls library however - and devs are very welcome to prove us wrong and find a way to really make controls like that work.


With the 24 games we currently have in our store, there are already a lot of different control schemes that use the smartphone controller in unique and creative ways - Balloon Party made a D-Pad work by splitting it up into two two-button pads controlled by two thumbs, for example, while the wonderful Grannies and Planes and Tiles of Doom use nothing but the Gyroscope.

We are looking forward to creating new awesome ways to use our smartphone controllers to their full potential - and to other developers doing the same.

• Additional Info for Developers
• AirConsole Game Developer Competition
Today we launched our online karaoke machine. Our HTML5 app transforms your big screen into the karaoke display and your smartphone becomes the remote control. You can connect any number of smartphones and everyone can queue up songs.

The smartphones are connected to the big screen using the AirConsole API. In this blog post I'm going to dive into the karaoke tech we have built and not so much into how the AirConsole platform works.

One of the coolest features of our web app is that the artist can help you sing. If you forget the lyrics, the artist will take over and will help you to get back into the song. Let's look at how we do this.

Singing together with the artist

We're getting our songs from YouTube. We automatically crawl the most popular karaoke YouTube channels to get the latest songs and also search for the original music video. So far we have about 7500 karaoke songs in our library. A few hundred have the "singing together with the artist feature" enabled. In the screenshot above, you can see that the original video clip including the artist audio is running on the left side of the screen, while the karaoke version without the artist audio is running on the right side of the screen. When you sing, we mute the original video clip, when you don't sing we mute the karaoke version and un-mute the original clip so you can hear the artist.

Syncing the songs

In order for this to work well, the two video clips need to be perfectly in sync. Most music and karaoke videos have some kind of intro of different length, so you can't just start playing both youtube videos and expect that they are in sync, you need to time shift them. Finding the correct timeshift manually would be very time consuming, so we developed an algorithm that looks at the audio tracks of both videos and tries to minimize the difference by trying different time shifts. Once the difference is minimal, the songs are in sync. Well, most of the time. We have a hitrate of about 98% and we review the results manually.

Detecting if the user is singing

We have different mechanisms to decide if the artist audio should be played or if the user wants to sing himself. The easiest is the microphone button on the smartphone. If the user presses it, we know he wants to sing and we should mute the artist. But we also have a more advanced way. You can hookup a real microphone to your laptop and we use the WebRTC getUserMedia and AudioContext HTML5 functionality to get the input volume of the microphone in javascript. When we see that the volume is above a threshold, we know the user is singing and doesn't need any help so we mute the artist. We could also use the smartphones microphone, but because iOS does not support WebRTC yet we decided to use the laptops microphone instead.

Recording yourself

Our karaoke machine can also record a video of you singing your favorite song. We only wanted to use HTML5 technology without plugins. This turned out to be trickier than we first thought. WebRTC getUserMedia has no implemented support for recording yet. However there are some great javascript libraries like RecordRTC that help you to record your videos locally. However different browsers use different video codecs and some can only record raw WAV audio. These files are huge and can't really be easily shared by the user. Another problem is that Chrome, the most popular browser on AirConsole, records audio and video into two separate files and they need to be merged. So we decided to add an autoscaling server side, that merges these files, transforms them into a compact mp4 video and emails them to the users. Only one problem left: How do you transfer huge WAV files to the server? 40MB for a 3 minute video? No thank you. It's amazing what you can do these days in javascript, for example transcode a WAV file into an MP3 inside the browser using multithreading with the help of webworkers. Problem solved.

Now give it a try!

Now it's time to try our AirConsole Karaoke Machine. We had tons of fun building the karaoke machine and sang way too much during development. Apologies to the rest of the people here in the office.

AirConsole Karaoke

Welcome to a new Karaoke world! 

Starting today the AirConsole Online Karaoke Machine is available in our store. The Karaoke app brings thousands of free songs for you to sing. Tap your smartphone to switch between the original artist singing and the karaoke version. You can also connect a real microphone and control the machine with it, just switch on to sing the karaoke version and switch off for a duet alongside your favorite celebrity!

If your big screen has a camera, you will be able to record your performance which will be sent to your email after. Organize a party, have a meet up! When you’re ready, you can use your smartphone to queue up your song. A playlist is automatically generated. Other people can also join the party and add new songs to the playlist. While you perform your favorite song, the crowd can send emojis to the big screen to support you!

Imagine the party you can organize using the AirConsole Karaoke!

The AirConsole team had a blast last weekend during the Global Game Jam. Not only did our team take the chance to create a new AirConsole game prototype but we also saw quite a few developers across the world making games using our technology. If you made a game using AirConsole during the GGJ and haven’t yet uploaded, please do so, so we can update this list.

Here is a list of all games already uploaded on our GGJ store: (Click on the titles to play the games).

Tiles of Doom:

A fantastic Global Game Jam game made in the UK by Aniode. In Tiles of Doom, you control an orb and must roll over floor tiles marked by symbols to complete the correct sequence before other players. It sounds simple but gets challenging quickly. Players must be quick in finding the right symbol among three dozen similar ones. In later levels the floor tiles will sometimes flip over while others will disappear entirely, making certain tiles very tricky to reach.

Questionable Cult Summoning Party

QCSP is a cooperative game made by Goodwolf Studio. Players must coordinate efforts to complete rune sequences to summon demons. After every completed sequence, you will face a greater memory challenge. To succeed on QCSP you will have to communicate with your team members to remember every piece of the puzzle.

Summoner Sacrilege:

Your goal in Summoner Sacrilege is to fight off the heavenly forces of angels and unicorns who want to keep you from completing your satanic summoning ritual. To do so, 1-10 players must tilt their phones to face enemies and then enter increasingly complicated combinations of swipes on their smartphone controllers.


Valhalla is a fast-paced couch multiplayer game. You control a wizard, zooming around an arena using a joystick on your smartphone. While moving around, you will imprison other wizards and score points by bringing them to the center of the screen.

Blocky Assassin

Blocky Assassin is an original game by TJ and Niels. You will have to find your target and kill him without getting noticed by other players. You’ll have a selection of weapons to help you complete your task. Be sure not call any attention to yourself or you will quickly fall victim to your opponents.

Dogma Derby

Dogma Derby is a local multiplayer crowd game. The game supports up to 16 players. The goal is to convert as many people as possible before the time runs out to beat the other teams.

Ritual Trip

Let the Ritual begin! In Ritual Trip, you will open the space-time doors between universes. Will you succeeded? 

Greek for Speed

Greek for Speed is an endless runner where each player controls a specific action of a greek athlete carrying the Olympic torch. Jump, slide and hit as a team while enjoying the greek vase style visuals.
Next PostNewer Posts Previous PostOlder Posts Home