This blog post provides a summarized version of my conversation with Shanea Leven, with slight adjustments for clarity and readability. To listen to the full discussion, find the latest episode of She Builds with AI on Spotify, Apple Podcasts, Amazon Music or your preferred podcast platform.

In this episode of She Builds with AI, I sat down with Shanea Leven, Co-Founder and CEO of Empromptu AI, to explore one of the most important but often underestimated questions in enterprise AI:

What happens after the impressive demo?

Shanea has spent more than 15 years working at the intersection of AI, machine learning and developer tools. Her career includes roles at Google, eBay, Cloudflare and Docker. She later founded CodeSee, a platform designed to help developers understand complex codebases, which was acquired by GitKraken in 2024.

Today, she is building Empromptu AI, a company helping organizations move beyond prototypes and create AI systems that are accurate, reliable and ready for real-world use.

Our conversation covered the engineering reality of non-deterministic systems, the importance of evaluations and edge-case detection, the economics of model ownership, the value of operational training data and the lessons Shanea is carrying into her second company.

Key Takeaways

• AI must be treated as a complete system, not a feature added to an existing product.
• Non-deterministic systems require new approaches to testing, monitoring, governance and reliability.
• Every AI workflow needs its own definition of what a good output looks like.
• Edge cases should be considered from the beginning, not discovered only after failure.
• Subject-matter experts are essential to the design and evaluation of enterprise AI.
• Companies can begin with frontier models while capturing the data needed to create greater independence later.
• Model ownership is not only about data privacy. It is also about economics, continuity and strategic control.
• Real workflows, expert corrections and user interactions can become valuable proprietary training data.
• Raising less capital can create focus and protect strategic flexibility during the early stages of a company.
• Domain knowledge, trust, brand and audience may become more defensible than access to the models themselves.

Every Experience Contributes to Founder Intuition

Before founding Empromptu, Shanea built products and platforms across several influential technology companies.

At Google, she gained experience inside one of the world’s most significant technology ecosystems. At eBay, she developed a deeper understanding of machine learning and its commercial applications. Cloudflare encouraged her to think about disruption and the dynamics described in The Innovator’s Dilemma. Docker demonstrated how a developer product can become foundational infrastructure, even when many end users do not realize they are interacting with it.

Shanea described these experiences as something founders gradually collect.

The deeper someone becomes immersed in an industry, its users and its recurring problems, the more clearly they begin to see opportunities others might overlook.

This is less about receiving one perfect startup idea and more about building enough context to recognize a problem worth solving.

What Selling a Company Teaches You About Value

Building CodeSee taught Shanea how to solve a problem for customers. Selling the company taught her to view the business through an entirely different lens.

During an acquisition, a company is not evaluated only as a useful product. It is considered as an asset.

Acquirers and investors use a different language from customers. They examine the structure, intellectual property, market position, technology, team and strategic value of the entire company.

Understanding this distinction changed how Shanea thinks about businesses today.

A founder may spend every day serving users and improving the product. An acquirer, however, asks what is being purchased, how durable it is and what it could become inside a larger organization.

Learning to understand both perspectives can influence how a founder structures and grows the company long before any acquisition conversation begins.

Why AI Is Not Just Another SaaS Feature

One of the central ideas in our conversation was the distinction between deterministic and non-deterministic systems.

Traditional software is usually designed to behave predictably. A defined input follows a defined path and produces an expected output.

AI systems work differently.

An AI application can receive an enormous variety of inputs and produce many possible outputs. Some may be useful. Some may be surprising. Some may be completely inappropriate for the context.

This changes what it means to engineer, test and operate a reliable product.

A production-ready AI application needs more than a connection to a powerful model. It also requires:

• Clean and relevant data
• An interface through which people can interact with the system
• Backend infrastructure and databases
• Guardrails that limit unacceptable behaviour
• Monitoring designed for changing AI outputs
• Evaluations that measure whether the system performs its intended task
• Feedback loops that support continuous improvement

Many teams attempt to create this architecture by connecting a growing collection of separate tools.

The result can be a brittle system with unclear accountability, inconsistent integrations and no unified learning loop.

The model is therefore only one part of the product. Reliability emerges from the way every layer works together.

The Opportunities and Limits of Vibe Coding

Rapid development tools have dramatically reduced the barrier to creating software.

Shanea sees enormous potential in this shift. With the right tooling, many more people can translate their ideas into functioning applications.

At the same time, not every problem requires AI.

Some tasks can still be solved more reliably and efficiently through conventional software logic. Adding a model simply because it is available can introduce unnecessary variability, complexity and cost.

Shanea also highlighted a current limitation of generative AI. Models are highly capable of extending, recombining and reproducing patterns from existing information. They are less reliable when asked to create something genuinely new or capture the full depth of human experience.

This leaves an important role for human discernment and taste.

The tools may accelerate building, but people still need to determine what is worth building, why it matters and whether the result is genuinely valuable.

Defining What “Good” Means

Accuracy is not a single universal score.

A shopping assistant, an invoicing workflow, a financial application and a healthcare system each involve different consequences, acceptable margins of error and definitions of success.

Every AI agent and workflow therefore needs its own evaluation criteria.

Shanea emphasized that evaluations cannot be added as an afterthought. Teams should first define what a good output looks like and then work backwards to build the system capable of producing it.

This process requires human domain expertise.

The people who understand the workflow must help define:

• What a correct output looks like
• Which mistakes are tolerable
• Which mistakes create unacceptable risk
• When human review is required
• How ambiguous situations should be handled

Without this clarity, teams may continue adjusting prompts and models without knowing what they are actually optimizing.

Edge Cases Are Where Reliability Becomes Visible

One of Shanea’s most memorable examples involved two self-driving vehicles meeting in an unexpected position on a one-way street.

The scene represented an edge case, a situation the system either had not seen before or had not been equipped to handle appropriately.

Self-driving vehicles make these failures highly visible, but the same issue exists in every AI application.

What happens when the system receives information outside its expected distribution? What does it do during an unusual combination of events? How should it respond when it lacks enough context?

These are often the moments in which hallucinations, errors and broken user trust emerge.

Teams cannot predict every possible scenario in advance. But they can create a process for identifying, recording and responding to new edge cases.

Subject-matter experts play a crucial role here. They can explain what a human would do in an ambiguous situation and help convert that judgment into evaluations, policies and escalation paths.

The goal is not to create a flawless system on day one. It is to build an infrastructure capable of learning safely from the situations it encounters.

Renting Intelligence Versus Building an Asset

Many companies currently build their AI products on third-party foundation models.

This approach provides speed, capability and access to technology that would otherwise require significant resources to develop.

But it can also create dependency.

Even when a provider states that it does not use customer data for model training by default, the enterprise still depends on an external platform for a critical part of its product or workflow.

That dependency may involve:

• Pricing changes
• Service interruptions
• Product or policy changes
• Geographic or regulatory restrictions
• Changes in model behaviour
• Limited control over future capabilities

The deeper question is therefore not only whether a provider trains on a company’s data.

It is whether the company has enough control and optionality to protect its business continuity.

Shanea describes this as the difference between a tenant economy and an asset economy.

In a tenant economy, a company continually rents access to intelligence.

In an asset economy, the knowledge created through the company’s applications, experts, customers and workflows contributes to something the organization can increasingly own and improve.

When Should a Company Build Its Own Model?

Shanea does not suggest that every company should immediately abandon frontier models.

They remain highly valuable for experimentation, prototyping and early product development.

A company can use them to test an idea, gather feedback and learn whether the proposed application creates real value.

At the same time, the organization can begin capturing the information generated during normal operations.

This might include:

• Corrections made by domain experts
• User feedback and preferences
• Decisions taken in ambiguous cases
• Examples of successful and unsuccessful outputs
• Real workflows completed by employees
• Recurring edge cases

Together, these interactions can form a valuable proprietary dataset.

Traditionally, model fine-tuning has been associated with expensive external data-labeling projects and artificially constructed examples. But many established companies already employ the experts and operate the workflows required to create high-quality training data.

The opportunity is to capture that expertise as work happens.

Over time, this gives the company the option to fine-tune an open-weight model or develop a more specialized system. It can continue using external models where they provide the greatest value while reducing total dependency on any single provider.

Why Empromptu Raised a Lean Pre-Seed

The AI funding market is often defined by exceptionally large rounds.

Shanea deliberately chose a more focused approach for Empromptu.

One of the most important lessons from her first company was the need to prioritize go-to-market and build a genuine revenue-generating business.

Limited resources create constraints, but those constraints can function as a force multiplier. They require the team to understand its customers, make deliberate choices and prove the fundamentals of the business before scaling its spending.

Shanea also discussed the underlying economics of venture capital.

A venture fund must generate returns for its own investors. The larger the fund and the investment, the larger the outcome a startup may eventually need to produce.

This can narrow the range of strategic options available to a founder, particularly when a large amount is raised before the company has found a repeatable market. A lean round can preserve flexibility while sharpening the team’s focus on customers and revenue.

Leading With Conviction the Second Time Around

Building a second company has also changed the way Shanea leads.

As a first-time founder, she sometimes over-rotated on being liked and on finding external confirmation that a decision was correct. Experience taught her that leadership often means acting with incomplete information while remaining open to evidence and thoughtful challenge.

Her reminder that “nobody really knows what they’re doing” is not cynical. It is liberating. The people who appear most certain are still interpreting a fast-moving market through their own limited perspective.

For Shanea, confidence now comes from trusting her accumulated domain knowledge, choosing a small circle of people who challenge hype, and creating enough quiet to hear her own thinking. Sometimes that is as simple as leaving her phone behind and walking her dog.

Her experience as one of relatively few Black women building at the AI infrastructure layer adds another dimension to that conviction: expertise does not become less valuable because others fail to recognize it immediately.

Where Competitive Advantage May Remain

By 2030, access to capable models may be far less differentiated than it is today. Models, interfaces and common AI features are likely to become easier to obtain and more interchangeable.

The durable advantage may sit elsewhere: in trusted brands, proprietary operational data, deep domain knowledge, engaged audiences and the systems a company creates to turn feedback into learning.

That is why the decisions companies make now matter. A prototype can demonstrate possibility. A reliable system can earn trust. An intelligent asset can compound over time.

Listen to the Full Conversation

The full episode of She Builds with AI with Shanea Leven is available on Spotify, Apple Podcasts, Amazon Music and your preferred podcast platform.

Listen here

If this conversation raises new questions about your AI roadmap, share the episode with a builder on your team and leave a review so more people can find the show.

Stay connected

• Shanea Leven: https://www.linkedin.com/in/shaneak/
• Empromptu AI: https://empromptu.ai/