When a Text AI Decides to Compose Game Music

When we think about the current boom in AI-generated music, our minds usually go straight to specialized platforms designed specifically to output audio waves. But a recent experiment titled "Scrimshaw Jukebox" suggests that the AI writing your emails and debugging your code might also be harboring some hidden musical talent.

Curious about the creative boundaries of advanced language models, a developer decided to put Claude Opus 5.5 to the test. Rather than asking for a standard audio file, the prompt was uniquely tailored for a text-based AI: design a simple text-based format for music, build a web component (an artifact) that can actually play it out loud, and compose a few example tracks to demonstrate how it works.

To give the AI a creative target, the developer requested music that matched the quality and vibe of the classic 1990 adventure game, The Secret of Monkey Island. The AI took the prompt and ran with it, leaning heavily into the requested thematic style. It didn't just build a functional music player; it generated tracks that were surprisingly competent and deeply evocative of the retro game's iconic atmosphere.

This experiment is fascinating because it highlights a growing trend in the AI space: emergent capabilities. Language models are not explicitly trained to be audio engines. Yet, by digesting vast amounts of human knowledge—including music theory, programming languages, and structural logic—they are figuring out how to bridge the gap between text and other mediums. The author of the experiment noted a striking parallel to how text models recently began generating 3D graphics purely through code. They aren't manipulating pixels or soundwaves directly; they are manipulating the underlying mathematical and logical structures that create them.

As general-purpose AI models continue to evolve, the lines between specialized tools are blurring. We are entering an era where a single conversational interface might be all you need to write a story, code the video game it belongs in, and compose the soundtrack that plays in the background.

Key Points

• A developer successfully used a text-based LLM (Claude Opus 5.5) to design a music format and compose original tracks.
• The AI generated both the underlying code for a music player and the thematic melodies inspired by classic adventure games.
• This suggests that language models are developing emergent abilities to understand and create non-text media (like music and 3D graphics) through code.

Why It Matters

It demonstrates that general-purpose language models are increasingly capable of handling multimedia creative tasks by translating artistic concepts into programmable code.

---

Sources:

• Scrimshaw Jukebox — Simon Willison's Weblog