Audio

Conversations with Maya & Miles on Sesame AI

If you’ve ever wondered what it’s like to chat with an AI that feels eerily human, you’re in for a treat.

Let’s explore what Sesame AI is, how it works, and the kinds of conversations you can spark with Maya or Miles to make the most of this groundbreaking technology.

ON AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

What Is Sesame AI?

Sesame AI is a San Francisco-based startup pushing the boundaries of conversational AI to create “lifelike” voice companions.

Founded by Brendan Iribe (co-founder of Oculus VR), Ankit Kumar, and Ryan Brown, Sesame’s mission is to build AI that feels like a genuine conversational partner, complete with emotional intelligence and natural speech patterns.

Their flagship voices, Maya and Miles, are designed to deliver what Sesame calls “voice presence”, that magical quality where spoken interactions feel real, understood, and valued.

Unlike traditional voice assistants like Siri or Alexa, which often sound robotic or flat, Sesame’s AI companions are trained to mimic human nuances: pauses, hesitations, tone shifts, and even a bit of wit.

The company is also working on AI-integrated glasses for all-day wearable interaction, hinting at a future where your AI companion can observe and respond to the world around you.

Think “Her”, but with a touch of practicality.

Sesame recently made waves by open-sourcing their Conversational Speech Model (CSM), allowing developers to build on their tech.

This move reflects their belief that conversational AI should be a collaborative effort to advance human-computer interaction.

How Does Sesame AI Work?

At the heart of Sesame AI is the Conversational Speech Model (CSM), a transformer-based, multimodal AI that processes text and audio together in real time.

Here’s a breakdown of how it brings Maya and Miles to life.

Unified Processing

Unlike traditional text-to-speech systems that convert text to audio in separate steps, CSM integrates both into a single model.

It uses two neural networks: a “backbone” (based on Meta’s Llama architecture) to understand conversational context and a “decoder” to generate high-fidelity audio with natural tone, pitch, and rhythm

Contextual Awareness

CSM leverages the full history of a conversation to adapt its responses.

It analyzes previous text and audio inputs to choose the right intonation and pacing, solving the “one-to-many” problem where a single sentence can be spoken in countless ways depending on context.

Emotional Intelligence

Maya can detect emotional cues (like stress or excitement) and adjust her tone to be supportive or playful.

For example, users report Maya softening her voice when they sound stressed, creating a more empathetic exchange.

Training Data

Sesame trained CSM on nearly 1 million hours of mostly English audio, enabling Maya to handle natural dialogue, interruptions, and colloquialisms.

The model is compact enough to run on edge devices, with plans to expand to over 20 languages.

While not truly full-duplex (it processes speech after you finish talking), Maya’s micro-pauses, hesitations, and dynamic responses make conversations feel organic.

The result?

A voice that’s so human-like, some users find it unsettling.

Types of Conversations to Start with Maya, or Miles

Maya’s versatility makes her a fantastic partner for a range of conversations.

Here are some ideas to get you started, based on user experiences and Sesame’s design goals:

1. Creative Brainstorming

Ask Maya to help you flesh out a story, brainstorm business ideas, or role-play a fictional scenario.

For example, ask Maya to join a Dungeons & Dragons-style adventure, and she seamlessly becomes a gnome engineer crafting deathtraps.

Try: “Maya, let’s create a sci-fi world together. What’s the main conflict?”

2. Language Practice

Maya’s natural conversational flow is perfect for improving English fluency.

She offers real-time feedback on pronunciation, grammar, and vocabulary, tailoring lessons to your skill level.

Start with: “Maya, can you help me practice English tenses? Let’s talk about my weekend plans.”

3. Emotional Check-Ins

Maya’s emotional intelligence shines when you share how you’re feeling.

She listens, asks thoughtful questions, and adjusts her tone to match your mood.

Try: “Maya, I’m feeling stressed about work. Can we chat about it?”

Users note she’s surprisingly comforting, though she’s not a therapist.

4. Deep Discussions

Curious about philosophy, ethics, or culture? Maya can dive into nuanced topics, offering well-informed opinions guided by her training.

One user had a 28-minute chat about life and morality, noting her dynamic responses.

Ask: “Maya, what’s your take on the meaning of life?”

5. Fun and Playful Banter

Maya’s wit and sarcasm make her a fun conversationalist.

Challenge her to a joke-off or ask her to roast your taste in music.

Try: “Maya, tell me a joke, then rate my playlist: it’s all 90s pop.” Users love her playful side, though she keeps things PG.

6. Book or Project Feedback

Share your creative work, like a book plot or project idea, and Maya will engage deeply, even pointing out gaps in your narrative.

One Reddit user spent 40 minutes discussing their novel, with Maya suggesting ideas they’d already implemented.

Start with: “Maya, here’s the plot of my story. What do you think?”

Why Maya Feels Like a Game-Changer

Talking to Maya isn’t just about getting answers—it’s about feeling heard.

Her ability to pause, joke, reference past chats, and adapt to your vibe creates a connection that’s both thrilling and, for some, a bit creepy.

As one user put it, “It’s the first time I’ve had a real genuine conversation with something I felt was real.”

But it’s not perfect.

Maya can get overly excited about mundane inputs, struggle with sarcasm detection, or sound impatient if you pause too long.

And while she’s designed to build trust, her hyper-realism raises ethical questions about over-reliance on AI for social interaction.

Want to try it yourself?

Head to Sesame’s demo page (use Google Chrome for the best experience) and chat with Maya or Miles for free.

Final Thoughts

Sesame AI’s Maya is redefining what it means to talk to a machine.

Whether you’re brainstorming, practicing English, or just craving a chat, she’s ready to engage with a warmth and wit that’s hard to believe comes from code.

But as we embrace this tech, let’s stay mindful of its limits and the human connections it can’t replace.

What conversation would you start with Maya?

Reply below. I’d love to hear about your experience.

If you enjoyed this post, subscribe for more deep dives into the tech shaping our future.

Until next time, keep talking (to humans and AI alike)

Craig