AI for SaaS and Software Companies
Shipping an AI feature is easy. Shipping one you can still change six months later, when the model provider has updated something and your quality moved, is the actual engineering problem.
The difference between an AI feature that compounds and one that quietly rots is whether you can measure it. Without an evaluation suite you cannot upgrade a model, change a prompt, or switch providers without hoping, and hoping does not scale.
AI for SaaS and software companies covers in-product AI feature development with evaluation harnesses and cost controls, provider classification under the AI Act, support and documentation automation, product analytics and churn modelling, and the operational practice that keeps shipped AI features stable.
Build the evaluation harness before the feature
This is the recommendation we make most often and the one most frequently skipped, because an evaluation suite is invisible to users and a feature is not. It is also the thing that determines whether you can change anything after launch.
- A representative test set with expected behaviour. Drawn from real usage, covering the ordinary cases and the ones that embarrass you, with agreed criteria for what good looks like.
- Run it on every change. Prompt edits, model upgrades, retrieval changes, provider switches. Without this each of those is a leap of faith. See LLM evaluation.
- Provider drift is real and silent. A hosted model updates, behaviour shifts, and nobody notices until support volume moves. Pin versions where you can and run evaluation continuously where you cannot.
- Track cost per request from day one. Retrofitting cost attribution is painful and the number determines whether the feature has a business.
- Capture user feedback as evaluation data. Thumbs, corrections and abandonment are labels, and most products discard them.
The question that exposes an unevaluated feature
Ask the team what would happen if the model provider shipped an update tomorrow. If the answer is that they would find out from support tickets, the feature is unmaintainable — not because it is bad, but because no one can safely change it. Every prompt tweak, every model upgrade, every cost optimisation becomes a gamble. Teams in that position usually freeze the feature, which means it degrades relative to what is possible while nobody feels able to touch it.
Unit economics, which is where in-product AI actually fails
Most AI features that get pulled were not withdrawn because they worked badly. They were withdrawn because gross margin moved in a direction nobody modelled, and by the time anyone noticed the feature was in the pricing.
Cost per request against revenue per account
A feature costing a few cents per invocation is fine at ten uses per account per month and a serious problem at a thousand. Model the distribution rather than the average, because heavy users are the ones who will find the feature and use it hardest.
Design the cost lever before you need it
Model tiering by task complexity, caching, shorter contexts, smaller models for classification steps. Building the ability to trade quality against cost is much easier before launch than during a margin crisis. See model optimisation.
Watch for the free-tier problem
AI features are expensive to serve and free tiers exist to be used. An unlimited AI capability on a free plan is a business decision that should be made explicitly rather than inherited from how the rest of the product is priced.
Usage-based pricing has its own failure mode
Charging for AI usage aligns cost and revenue and makes customers ration the feature, which suppresses the adoption you were trying to create. Neither answer is obviously right and the choice should be deliberate.
Provider or deployer, per product
Internal AI use makes you a deployer. Shipping AI capability to customers under your own name generally makes you a provider, with substantially heavier obligations if the system is high-risk. Most SaaS features are not high-risk, and establishing that is itself the work.
| Situation | Likely position | What follows |
|---|---|---|
| Third-party model in internal tooling | Deployer | Light. Ordinary governance applies. |
| Third-party model behind your product feature | Provider of that system | Transparency, documentation, and high-risk duties if applicable |
| Fine-tuned model shipped in your product | Provider | Fine-tuning on your own data can reclassify you |
| Feature that scores or ranks people | Check carefully | Employment, credit and education uses are Annex III |
| Chat interface with users | Article 50 applies | Disclosure that they are interacting with AI |
| Feature generating media | Article 50 applies | Machine-readable marking of synthetic content |
Your customers will ask before a regulator does
Enterprise procurement and security questionnaires now routinely ask what models you use, where data is processed, whether customer data trains anything, what your evaluation practice is and how you handle model changes. Having documented answers is a sales advantage well before it is a compliance requirement — and assembling them under deal pressure is how commitments get made that engineering then has to honour.
How an engagement runs
Evaluation and cost instrumentation come before the feature, because retrofitting both is expensive.
Classification and scope
Provider or deployer per product, whether any use is high-risk, and what Article 50 requires of the interface.
Evaluation harness
Representative test set from real usage, agreed quality criteria, and cost attribution instrumented.
Feature build
Built against the harness, with model tiering and cost levers designed in rather than added later.
Staged release
To a cohort, with quality, cost per request and support volume monitored together.
Operation
Continuous evaluation against provider drift, cost tracked per account, user feedback fed back as labels.
What you receive
A feature you can keep changing, and a cost line you can defend.
Provider classification
Per product, documented, with what follows from it stated.
Evaluation harness
Representative cases, agreed criteria, run on every change including provider updates.
Cost attribution
Per request and per account, with the distribution modelled rather than the average.
The feature itself
Built against the harness, with quality-cost levers available from launch.
Article 50 compliance
Interaction disclosure and machine-readable marking where content is generated.
Customer-facing documentation
The answers procurement and security reviews will ask for, written before they ask.
Is this the right starting point?
Worth being direct. There are situations in SaaS and software where custom AI work is the wrong spend, and those are listed rather than buried.
Worth doing if
- You have shipped AI features and cannot tell whether a model update changed their quality.
- Cost per request is unknown or growing faster than the revenue attached to it.
- You need the provider-versus-deployer question settled before a procurement review.
- Support volume moved after a model change and nobody can explain the direction.
- You are about to ship an AI feature and want the evaluation practice in place first.
Do something else if
- You want the feature shipped without evaluation. It will be unmaintainable within a quarter.
- The underlying product problem is not one AI addresses and the feature is a positioning exercise.
- There is no route to instrument cost. The economics cannot be managed blind.
- You want customer data used for training without settling the contractual position.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
What should we build first — the feature or the evaluation?
The evaluation, and this is the recommendation we make most often and see skipped most often. Without a representative test set and agreed quality criteria, you cannot upgrade a model, change a prompt or switch providers without hoping. Ask your team what happens if the provider ships an update tomorrow: if the answer is that support tickets would tell them, the feature is already unmaintainable.
How do we stop AI features destroying gross margin?
Instrument cost per request from day one and model the distribution rather than the average, because heavy users will find the feature and use it hardest. Then build the levers before you need them — model tiering by task complexity, caching, shorter contexts, smaller models for classification steps. Most withdrawn AI features were not withdrawn for quality; they were withdrawn because margin moved and nobody had a way to trade quality against cost.
Does shipping an AI feature make us a provider under the AI Act?
Generally yes for that system, if you place it on the market under your own name — and fine-tuning a third-party model on your own data can do the same. That matters because providers of high-risk systems carry conformity assessment, technical documentation, quality management and post-market monitoring duties. Most SaaS features are not high-risk, and establishing that clearly is itself the work worth doing before a customer asks.
What will enterprise customers ask us about our AI?
Which models you use, where data is processed, whether their data trains anything, what your evaluation practice is, how you handle model changes and what happens on exit. Having documented answers is a sales advantage before it is a compliance requirement — and the alternative is making commitments under deal pressure that your engineering team then has to honour.
Should we charge for AI usage?
It is a real trade-off rather than a best practice. Usage pricing aligns cost with revenue and makes customers ration the feature, which suppresses the adoption you wanted. Bundling drives adoption and exposes you to your heaviest users. The wrong answer is inheriting whichever model the rest of your product uses without modelling what heavy AI usage does to the margin on your largest accounts.
Related verticals
Organisations in SaaS and software usually share data, buyers or regulators with these. All fourteen are listed on the Technology & Telecom page.
AI for Startups
Defensibility, evaluation before scale, and what diligence will ask — including when not to build.
Read more →AI for Developer Tools Companies
Evaluation for code-facing features, benchmark honesty and unit economics — for buyers who will check.
Read more →AI for IT Services and System Integrators
Estimation, knowledge reuse and proposal production — plus building capability you can genuinely sell.
Read more →Tell us what the problem looks like.
Thirty minutes, no charge, no deck. We will tell you whether this is an AI problem, a data problem, or a process problem — and we will say when the honest answer is to buy something rather than build it.