RollTab: Training a Real-Time Transformer for Piano Autocomplete

The author built RollTab, an iPhone app that uses a 125M-parameter transformer to autocomplete piano performances in real time. By refining MIDI tokenization and applying Direct Preference Optimization, the model achieves high-speed, musically coherent continuations. The project demonstrates that careful data curation and preference tuning are essential for creating effective on-device generative music AI.

Key Points

• Optimized MIDI tokenization by using a single multi-attribute note token instead of separate note-on/off events, which prevented drift and increased generation speed.
• Data quality proved more important than quantity; aggressive cleaning and deduplication of MIDI files yielded better results than a 5x larger but noisier dataset.
• Direct Preference Optimization (DPO) was the most impactful post-training step, using AI-driven pairwise comparisons to refine the model's musicality.
• The 125M-parameter model was successfully deployed on-device using Core ML and INT8 quantization, achieving real-time performance on modern iPhones.

Sentiment

Primarily positive and technically inquisitive, though marked by a polarized philosophical debate regarding the impact of AI on human creativity.

In Agreement

• The project is a great example of the Hacker News spirit of tinkering and learning.
• On-device performance of a 125M model at 108 notes/sec is technically impressive.
• Using compound note events is a highly effective way to optimize transformer inference for music.
• The tool is valuable for musical ideation and overcoming creative blocks.

Opposed

• AI-generated continuations of classic pieces like Für Elise can feel disconcerting or uncanny.
• AI application to art may disempower human artists and lead to lower-common-denominator creative output.
• The model currently lacks a strong sense of rhythm and long-term musical composition.
• Some users find the constant presence of AI in creative tools to be tiring or suspicious.