Duetto is an open-source project that lets you play a MIDI keyboard while a pretrained transformer plays a second piano alongside you. It comps while you play and takes the melody when you rest. The developer, Riccardo Cataldi, shared the repository on Hacker News on 24 October 2026.
This Duetto music transformer demo is early and small, but it shows an idea worth understanding: an AI that follows you instead of replacing you.
Key takeaways
- Duetto listens to your key, chords, pulse and phrases, then answers on a second piano.
- It builds on the pretrained Anticipatory Music Transformer, which was trained on the Lakh MIDI dataset.
- The author’s setup runs the model on a Jetson Orin Nano. You can also run the model server locally.
- The license is PolyForm Noncommercial 1.0.0, so commercial use is off the table.
- The repository gives no latency figures, so treat “real time” as a claim to test yourself.
What does Duetto actually do?
You plug in a MIDI keyboard and play. Duetto tracks the key you are in, the chords you play, your pulse (the steady beat) and where your phrases start and stop. Then the model plays a second piano part back to you.
The behaviour changes with a preset. In the “it comps” mode, you improvise and the model plays the harmony. In “it solos” mode, you hold down the chords and the model improvises on top. The repository includes demo videos, including a blues duet, and sample files so you can hear the output before building anything.
Think of it as a bandmate who never talks over you. When you stop, it steps forward. When you start again, it steps back.
How does the model know what to play?
Duetto uses the pretrained Anticipatory Music Transformer, published in Transactions on Machine Learning Research in 2024 by Thickstun, Hall, Donahue and Liang. The paper proposes “anticipation”: interleaving one stream of musical events with a second, correlated stream of controls. That lets the model be steered, for example to fill in an accompaniment for a given melody.
The authors trained on the Lakh MIDI dataset, a large collection of MIDI files. Per the paper’s abstract, human evaluators judged its accompaniments over a 20-second clip to be about as musical as human-composed ones. That result was for offline generation, not live playing, so it does not prove Duetto sounds that good on stage.
The Duetto page does not describe the architecture in detail. The model folder name hints at a smaller decoder-only network, but the page does not confirm it.
What hardware do you need to run it?
The author’s setup splits the work in two. A Jetson Orin Nano running JetPack 6 hosts the model. Your laptop handles MIDI input, sound and the piano roll display.
| Part | Job |
|---|---|
| MIDI keyboard | Sends your notes to the laptop |
| Laptop | Runs MIDI handling, sound and the piano roll |
| FluidSynth plus a General MIDI soundfont | Turns the model’s notes into piano sound |
| Jetson Orin Nano (JetPack 6) | Hosts the transformer |
No Jetson? The page says you can run the model server locally instead. You will also need a Python virtual environment on the laptop.
Is it better than earlier piano AI?
That is not yet answerable. Earlier work such as Google Magenta’s Piano Transformer generated continuations and accompaniments from a prompt, usually offline in a notebook. Duetto’s pitch is different: a live back-and-forth with a human at a keyboard.
The repository, as read for this article, publishes no benchmarks, no latency numbers and no listening tests. The 43 commits and the “© 2026” footer suggest a young personal project, not a finished product. My opinion: the interesting part is the interaction design, the comp-or-solo switch, more than the model itself.
What does the license let you do?
Duetto is released under PolyForm Noncommercial 1.0.0. You may use, modify and share it for noncommercial purposes. Selling it, or building a paid product on it, is not covered. The underlying model carries its own license, so check that too before you plan anything beyond a hobby project.
What happens next
The Hacker News post had 1 point and no comments when the feed captured it, so reaction is still unknown. The useful signals to watch are measured latency, a recipe that runs on a plain laptop without a Jetson, and independent listening tests. If the author publishes those, Duetto stops being a demo and becomes a tool musicians can judge.
FAQ
Do I need a Jetson to use Duetto?
No. The author’s setup uses a Jetson Orin Nano, but the page says you can run the model server locally instead.
Can I use Duetto in a paid product?
Not under its PolyForm Noncommercial 1.0.0 license, which allows noncommercial use only. The underlying model has its own license as well.
What model powers it?
The pretrained Anticipatory Music Transformer, trained on the Lakh MIDI dataset and designed for controllable generation such as accompaniment.
Next step: watch the demo videos in the Duetto repository, then decide whether your MIDI keyboard deserves a bandmate.
2 comments