The smartphones are connected to the big screen using the AirConsole API. In this blog post I'm going to dive into the karaoke tech we have built and not so much into how the AirConsole platform works.
One of the coolest features of our web app is that the artist can help you sing. If you forget the lyrics, the artist will take over and will help you to get back into the song. Let's look at how we do this.
Singing together with the artist
We're getting our songs from YouTube. We automatically crawl the most popular karaoke YouTube channels to get the latest songs and also search for the original music video. So far we have about 7500 karaoke songs in our library. A few hundred have the "singing together with the artist feature" enabled. In the screenshot above, you can see that the original video clip including the artist audio is running on the left side of the screen, while the karaoke version without the artist audio is running on the right side of the screen. When you sing, we mute the original video clip, when you don't sing we mute the karaoke version and un-mute the original clip so you can hear the artist.
Syncing the songsIn order for this to work well, the two video clips need to be perfectly in sync. Most music and karaoke videos have some kind of intro of different length, so you can't just start playing both youtube videos and expect that they are in sync, you need to time shift them. Finding the correct timeshift manually would be very time consuming, so we developed an algorithm that looks at the audio tracks of both videos and tries to minimize the difference by trying different time shifts. Once the difference is minimal, the songs are in sync. Well, most of the time. We have a hitrate of about 98% and we review the results manually.
Detecting if the user is singing