Uber Genie is the AI bot that answers Uber engineers’ questions inside Slack. The most useful thing about it isn’t the language model. It’s the plumbing around the model that decides what the bot reads, what it cites and when it stays quiet. Since its launch in September 2023, Genie has handled more than 70,000 questions, and a later rebuild cut its incorrect advice by a relative 60%.
A ByteByteGo breakdown published on 19 October 2026 brought the system back into the spotlight. Here is how it works and what you can borrow from it, based on Uber’s own engineering write-ups.
Key takeaways
- Uber Genie uses retrieval-augmented generation (RAG). It searches Uber’s internal docs first and then has a large language model (LLM) write an answer from what it found.
- According to Uber, it runs in 154 Slack channels, has a 48.9% helpfulness rate and has saved an estimated 13,000 engineering hours.
- An “enhanced agentic RAG” upgrade, described in May 2025, raised acceptable answers by a relative 27% and cut incorrect advice by a relative 60%.
- Most of those gains came from cleaner documents and smarter retrieval, not from a bigger model.
What is Uber Genie and why did Uber build it?
Before Genie, Uber’s Slack support channels received about 45,000 questions a month, according to Uber’s original Genie post from 10 October 2024. Engineers waited for whoever was on call, and the same questions came up again and again.
The answers already existed. They were scattered across Engwiki, Uber’s internal wiki, and an internal version of Stack Overflow. Genie was built to act as a librarian who knows which shelf to check before anyone has to page a human.
Uber chose RAG over fine-tuning, which means retraining a model on your own data. With RAG, Uber didn’t need a curated training set before launch, and when a wiki page changes, the next answer reflects the change without any retraining.
How does Uber Genie turn wiki pages into answers?
The work happens in two separate phases: preparing knowledge ahead of time, and answering a question on demand.
- Extract: An Apache Spark job pulls pages from Engwiki and the internal Stack Overflow, keeping each page’s URL and metadata.
- Transform: LangChain splits the text into chunks, and an OpenAI embedding model turns each chunk into an embedding, a list of numbers that captures its meaning.
- Load: The embeddings go into Sia, Uber’s in-house vector database. Search indexes are built and stored in Terrablob, Uber’s blob storage, and serving machines called “leaves” sync them from there.
- Answer: The Slack bot sends your question to a Knowledge Service. The service embeds the question, fetches the closest chunks and passes them to the LLM through Michelangelo Gateway, Uber’s route to its models.
Because the source URL travels with every chunk, Genie can cite exactly where an answer came from. A unique ID also follows each request to the gateway’s audit log, so Uber can track what the bot costs.
How does Genie avoid making things up?
Each retrieved passage is labelled as a separate “sub-context” with its URL. The model is told to answer only from those sub-contexts and to cite them. Uber also limited the knowledge base to sources most of its engineers can already see, so the bot doesn’t surface restricted material in Slack.
Every reply comes with buttons to ask a follow-up, mark the issue resolved or escalate to a human. Once a thread is escalated, Genie goes quiet. Users rate replies as Resolved, Helpful, Not Helpful or Not Relevant. Those ratings flow through Kafka into a Hive table and onto a dashboard.
Uber also uses an “LLM as a judge”: a second model that grades answers for hallucination and relevance. A similar judge scores the wiki pages themselves and suggests how to improve them.
Why wasn’t the first version good enough?
A 48.9% helpfulness rate is solid for general support, but it isn’t good enough for security and privacy questions, where a wrong answer can do real damage. Uber’s subject-matter experts built a “golden set” of more than 100 test questions and ran Genie against more than 40 policy PDFs.
Many answers came back incomplete or wrong. A big culprit was PDF tables: standard loaders split them into loose cells, so a rule lost the row and column that gave it meaning.
What changed with Uber’s agentic RAG rebuild?
Uber’s 29 May 2025 post on Enhanced Agentic RAG describes the fix. It adds small AI agents, each with one job, before and after retrieval.
| Stage | Original Genie (2023) | Agentic RAG (2025) |
|---|---|---|
| Source format | Wiki pages and PDFs | Google Docs read by a custom loader, with tables converted to Markdown |
| Metadata | Title, URL, IDs | Adds summaries, FAQs and keywords |
| Before search | Question used as written | Query Optimizer rewrites or splits the question; Source Identifier picks the likely documents |
| Search | Vector search only | Vector search plus BM25 keyword search |
| After search | Chunks passed straight to the LLM | Post-Processor removes duplicates and restores document order |
| Testing | User feedback and an LLM judge | LLM judge scores 0-5 against expert answers, cutting evaluation from weeks to minutes |
The agents are orchestrated with LangGraph. If you’re weighing that framework, our LangChain vs LangGraph guide explains the trade-offs. Uber reports a relative 27% rise in acceptable answers and a relative 60% drop in incorrect advice.
Opinion: the headline lesson is unglamorous. Uber got its biggest accuracy gains by fixing tables, adding summaries and narrowing the search, not by switching to a smarter model.
What happens next
Uber’s May 2025 roadmap lists five steps:
- Extend document enrichment to images.
- Replace one-shot query rewriting with an iterative “Chain-of-RAG” for questions that need several lookups.
- Add a self-critique agent that checks answers before they are posted.
- Expose features as tools that agents choose based on the question.
- Roll out to more internal help channels.
The earlier post also points beyond Slack, toward help built into tools such as Michelangelo and code editors. Uber has not given a timeline. For a way to judge how far along a system like this is, see our 7-level maturity model for production AI agents.
FAQ
Is Uber Genie available to the public?
No. Genie is an internal tool for Uber engineers, and it runs inside Uber’s own Slack workspace.
Why did Uber use RAG instead of fine-tuning?
RAG let Uber launch faster without building a training dataset first. Updated documents also improve answers without retraining the model.
What does the 60% improvement actually measure?
It is a relative 60% reduction in incorrect advice. Uber measured it on a golden set of more than 100 security and privacy questions, graded against answers written by experts.
If you’re building an internal help bot, start where Uber did: audit your documents, fix the tables and write a golden set of test questions before you touch the model.
1 comment