Bringing Ode Poetry to life with MAI’s audio models

Bringing expressive speech to conversational AI with MAI-Voice

Ode Poetry also uses MAI-Voice to re-create William's distinctive vocal style. What makes MAI-Voice special is its ability to reproduce William's speaking style with very high fidelity, using only a short sample of audio. Once you've heard William's voice, you'll understand why this is so important.

MAI-Voice's capabilities were achieved by building our own large-scale multilingual data pipeline, augmented with high-fidelity studio recordings of voice talent. This enabled researchers to train a model with both the breadth to sound natural across contexts and the depth to faithfully reproduce any specific speaker. Here's an example:

An audio sample showcasing the natural expressivness available with MAI-Voice.

Reasoning between conversation and poem recommendations

Ode Poetry is a unique conversational experience because it not only enables a natural conversation, but it also recommends a poem as an outcome, using the structured conditions William has provided for the poems in his "Poetry Pharmacy" books.

We developed a harness to distill William's years of experience with personal consultations into a system that can match the unstructured conversation with the user into a framework that enables the right poem to be recommended. This system included carefully chosen deterministic logic and tool-calling mechanisms.

We also focused on responsible AI for a safe user experience; for example, experimenting with jailbreaks that can trick a language model into attempting real psychiatric diagnoses. Through an iterative approach of safety evals, red teaming, and guardrail updates, we were able to prevent a range of misaligned model behaviors.

Hill-climbing was also a critical component to achieving a premium editorial product. High-quality production AI systems hinge on countless small improvements. Accuracy, consistency, latency, reasoning, and expressiveness all contribute to the final result, and a fix in one dimension can cause regressions in another. We evaluated Ode Poetry across a wide range of emotional scenarios and used the results to iteratively refine prompts, recommendation logic and speech output. Achieving a consistently excellent result took relentless iteration of both text and audio prompts, and the shared commitment from both Microsoft AI and William's team made this collaboration a joint act of creative care.

This is reflected in the app launching today. As William shared when he first began interacting with his digital extension: "One of [the researchers] recorded my introduction and the next thing I knew, he was playing my voice…and the intonation, the pauses…I just thought, wow, that's quite nice.

"It's really exciting for me," he added, "because what we're about to embark on together is in a way my lifetime's ambition, which is to bring poetry to everyone."

We invite you to try out the experience of a poetry session with William at Ode Poetry, and test the latest versions of our models at the MAI Playground.