RollTab: Training a Real-Time Transformer for Piano Autocomplete

The author built RollTab, an iPhone app that uses a 125M-parameter transformer to autocomplete piano performances in real time. By refining MIDI tokenization and applying Direct Preference Optimization, the model achieves high-speed, musically coherent continuations. The project demonstrates that careful data curation and preference tuning are essential for creating effective on-device generative music AI.

Key Points

• Optimized MIDI tokenization by using a single multi-attribute note token instead of separate note-on/off events, which prevented drift and increased generation speed.
• Data quality proved more important than quantity; aggressive cleaning and deduplication of MIDI files yielded better results than a 5x larger but noisier dataset.
• Direct Preference Optimization (DPO) was the most impactful post-training step, using AI-driven pairwise comparisons to refine the model's musicality.
• The 125M-parameter model was successfully deployed on-device using Core ML and INT8 quantization, achieving real-time performance on modern iPhones.

Sentiment

The overall sentiment is positive and curious, with Hacker News mostly appreciating RollTab as a technically impressive, playful, and culturally interesting project. The main disagreement is not over whether the implementation is competent, but over how AI-assisted generation should fit into musical practice and whether it enriches or cheapens creative work.

In Agreement

• The project is a strong example of hands-on technical exploration, with value in the author's process of building and learning rather than only in the final app.
• MIDI is well suited for this kind of model because it gives a compact symbolic representation of musical structure, timing, and performance detail.
• On-device real-time inference makes the tool feel like a practical creative instrument rather than just an offline generation demo.
• Musical autocomplete has historical parallels in classical composition training, improvisation, formula learning, and composer continuation games, so the idea is not alien to music practice.
• The app could be useful as a creative partner for quickly exploring variations, dead ends, surprising continuations, or live jamming ideas.
• There is interest in broader availability through shared weights, Web MIDI, browser inference, VST plugins, Max for Live devices, and versions for other instruments.

Opposed

• Some commenters argue that letting a machine generate continuations removes part of the joy of improvisation and bypasses the embodied learning of harmony, timing, voicing, and instrument technique.
• A few readers find the generated continuations musically awkward, disconcerting, or simply poor when they diverge from familiar piano phrases.
• Skeptics connect the project to broader discomfort with AI in art, arguing that generative systems can lower creative standards or disempower human artists.
• Some comments question the purpose of the tool, asking whether it offers meaningful musical value beyond novelty or technical demonstration.
• There is concern that AI tools may encourage people to claim authorship over outputs they did not truly perform or compose, though other commenters dispute that framing.