Rhythm games like Guitar Hero, Rock Band, or Tap Tap Revenge have a common problem: Creating 'beat tracks' for users to play along to a song is a time-consuming, labor-intensive task. The problem is similar to music transcription, and today there isn't an automated system that performs better than humans at this task. Some music games like Rhythmatic automatically generate beat tracks so you can play along to any song you own. However, these automated algorithms aren't as good as manual transcription, and the logic is usually hard-coded rather than machine-learned.
This project uses convolutional and recurrent neural networks to automatically learn a beat detection algorithm from hundreds of human-created beat tracks from the popular iOS music game, Tap Tap Revenge.
After working through the lectures and assignments for Stanford's CS231n: Convolutional Neural Networks course, I wanted to learn more about Google's popular new Tensorflow framework. So, inspired by Andrej Karpathy's char-rnn blog post, I tried training a character-level language model using a recurrent neural network (LSTM) in Tensorflow on a few years' worth of Google Hangouts chat history with one of my friends. Using the resulting language model, I can create an "autocomplete" or "smart reply" for Google Hangouts based on my personal speech patterns. Equally fun, I can simulate entire conversations by sampling from the model, character by character.
I got super excited when I learned that all the lecture videos and course materials for Stanford’s CS231n course were freely available online. I had experience in machine learning through coursework at school, but I didn’t have much specific expertise in neural networks and deep learning. So I decided to work through the course on my own after work – much in the same way I worked through CS106A to learn how to program years ago.
The course was extremely well taught, and the techniques and concepts I learned motivated several other projects.
Rhythmatic is my iOS claim to fame. It’s a rhythm game akin to Guitar Hero or Tap Tap Revenge, but unique in that a self-developed beat detection algorithm generates tap tracks to any song in your music library. It was downloaded over 300,000 times and at one point was the #1 Overall Game in the Japanese App Store, thanks to a well-executed marketing campaign on Free App a Day.
Rhythmatic was a technical challenge for me: I had to learn a lot of new concepts from digital signal processing, like Fast Fourier Transforms and spectral flux-based onset detection. The end result was a popular game that validated my belief that the best products are those that you make for yourself.
A professional-grade investing app for iPhone and iPad, MarketShark helps you research stock fundamentals and track your portfolio on the go. I developed MarketShark with Bryson Alef after a summer internship in private equity, where I discovered that software for researching stock fundamentals was prohibitively expensive for all but large enterprises.
After Rhythmatic’s success, MarketShark was largely experimental:
Diaballic is an endless side-scrolling platformer game for iOS, written in Objective-C using the Cocos2D engine.
I developed it in high school as the culmination of my effort to teach myself how to program computer games (I had yet to take a Computer Science class in school). After working through Stanford’s introductory CS106A: Programming Methodology course on iTunesU while playing squash in Europe during the summer, I made my dream come true and released the first version of Diaballic on the App Store. It was downloaded a few thousand times and got me hooked on building iOS apps.
Tap Master is a reflex, timing, and dexterity game for iOS. For example, one of the game modes has you try to stop a timer at exactly 1.000000 seconds. Strangely addictive, it proved a huge time waster for me and my friends.
A brutally randomized way to jack up your workouts. Go until you can’t across a variety of exercises, with randomized intervals and rests to keep you pushing your limits.
Best done with friends. If you’re the competitive type, this app is for you.
Since I’m no longer a student-athlete, getting enough exercise can be a challenge. To fix that, I designed and prototyped an iOS app called Simply Fit. You set a goal for how many times a week you want to exercise, and then you gain or lose points each week if you meet or fail to meet that goal.