Reinforcement Learning
Learning a policy for sequential decisions where each choice changes what happens next, applied only where that structure genuinely exists and a simpler method has been shown to fall short.
Reinforcement learning is the most requested and least often appropriate technique in this pillar. It is genuinely powerful for sequential decision problems, and most business problems presented as reinforcement learning are single-step prediction or constrained optimisation wearing a costume.
Reinforcement learning is the training of a decision policy through interaction with an environment, where actions influence future states and rewards. It suits problems in which decisions are sequential, delayed consequences matter, and either a reliable simulator or substantial logged decision data exists.
Does your problem actually need it?
Four conditions have to hold. If any fails, something simpler and more reliable will do better:
- Decisions are sequential. Today's action changes tomorrow's situation. A one-off prediction with an immediate outcome is supervised learning.
- Consequences are delayed. The value of an action only becomes clear several steps later, so credit has to be assigned across time.
- Exploration is possible safely. Either a faithful simulator exists, or trying suboptimal actions in production is affordable and reversible.
- The reward is definable. Somebody can write down, precisely, what good looks like as a number, including the things that must not be sacrificed to achieve it.
In our assessments, most candidate problems fail the third or fourth condition. Where they do, we say so and propose the alternative: usually operations research, a contextual bandit, or a straightforward predictive model feeding a rule.
Reward specification is where these projects fail
An agent optimises exactly what you wrote, not what you meant, and it will find the interpretation you did not consider. Reward design, including explicit penalties for the outcomes you would find unacceptable, is the majority of the intellectual work and the part that most needs your domain experts rather than ours.
How we approach it
Start with bandits where the problem allows
Many problems framed as reinforcement learning are contextual bandits: choose an action, observe the reward, no lasting state change. Bandits are far simpler to implement, evaluate and operate safely, and they solve a surprising share of what gets proposed. We check this first.
Build or validate the simulator
Where exploration in production is unsafe, everything depends on the fidelity of the simulator, and a policy trained in a flawed one fails in reality with confidence. We validate the simulator against historical outcomes before trusting any policy learned inside it, and we report where it diverges.
Learn offline from your logs where possible
Offline reinforcement learning uses historical decision data without live exploration. It carries its own difficulty, because your logs only contain actions your current policy chose, but it is far safer as a starting point and often sufficient to demonstrate value.
Evaluate off-policy before deploying
Estimating how a new policy would have performed using data collected under the old one, with the uncertainty stated. This is technically demanding and it is the difference between a defensible deployment decision and a hopeful one.
Constrain the policy in production
Hard limits on what actions the policy may take, regardless of what it has learned. Safe fallback behaviour, a bounded exploration budget and a tested switch that reverts to the incumbent policy immediately. The same operational discipline described under AgentOps applies here.
Deploy gradually and measure against the incumbent
A small share of traffic or a limited set of assets first, with the existing policy running alongside as the control. Reinforcement learning policies fail in ways that are hard to anticipate, so the rollout is designed to make failure cheap and visible.
Where it genuinely fits
| Problem | Why RL suits it | Usual prerequisite |
|---|---|---|
| Inventory and replenishment policy | Sequential, delayed cost, stock state carries over | Reliable simulator from demand history |
| Dynamic resource allocation | Actions change the state of the system | Simulator or safe production exploration |
| Sequential treatment or offer paths | Value emerges over a customer journey | Substantial logged decision history |
| Industrial process control | Continuous control with delayed effect | High-fidelity process simulator |
| Energy dispatch and storage | Decisions now constrain decisions later | Physical model plus price history |
| Single-step targeting decisions | Poor fit | Use supervised learning or a bandit |
How the engagement runs
Feasibility, reward design and simulator validation come before any policy is trained.
Feasibility and framing
The four conditions tested, alternatives compared, and a written recommendation on whether to proceed with RL at all.
Reward design and constraints
Reward and penalties specified with your domain experts, safety constraints and fallback behaviour defined.
Simulator or offline data
Simulator built and validated against history, or offline dataset assembled from decision logs with coverage assessed.
Policy learning and off-policy evaluation
Policy trained, evaluated off-policy against the incumbent with uncertainty stated, and stress-tested against edge cases.
Constrained deployment
Limited rollout with the incumbent as control, action constraints enforced, monitoring and revert switch tested.
What you receive
A policy that is safe to deploy, and an honest account of where it is not.
Feasibility assessment
Whether RL fits, what the alternatives are, and the recommendation in writing.
Reward specification
The objective, the penalties and the behaviours explicitly ruled out, agreed with your experts.
Validated simulator or offline dataset
With divergence from historical reality reported honestly.
Trained policy
With off-policy evaluation against the incumbent and the uncertainty around that estimate.
Safety constraints
Hard action limits, fallback policy, exploration budget and a tested revert switch.
Rollout and monitoring plan
Staged deployment with the incumbent as control, and the metrics that trigger reversion.
Is this the right engagement?
Worth being direct. Reinforcement Learning is the wrong spend in some situations, and those are listed rather than buried.
Good fit if
- Decisions are genuinely sequential and today's action changes tomorrow's state.
- A reliable simulator exists or can be built, or exploration in production is safe.
- Domain experts can specify the reward and what must never be traded away.
- The incumbent policy can run alongside as a control.
- There is appetite for a longer engagement with a real chance of a negative finding.
Choose something else if
- The decision is single-step with an immediate outcome. Use supervised learning.
- The constraints are known and the objective is fixed. Use optimisation.
- Exploration is unsafe and no faithful simulator can be built.
- Nobody can define the reward precisely enough to write down.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
When should we use reinforcement learning instead of supervised learning?
When decisions are sequential and consequences are delayed, so an action changes the situation the next decision is made in. If the outcome of each decision is observed immediately and independently, supervised learning or a contextual bandit will be simpler, faster and more reliable.
Do we need a simulator?
Either a simulator or substantial logged decision data. Where exploration in production is unsafe, everything rests on simulator fidelity, so we validate it against historical outcomes before trusting any policy learned inside it and report where it diverges.
How do you keep a learned policy safe in production?
With hard constraints on the actions it may take regardless of what it has learned, a bounded exploration budget, a safe fallback policy, staged rollout against the incumbent as control, and a tested switch that reverts immediately.
What is off-policy evaluation?
Estimating how a new policy would have performed using data collected under a different one, so you can make a deployment decision before exposing customers or assets to it. It is technically demanding and the uncertainty has to be stated, but it is the difference between a defensible decision and a hopeful one.
Will you tell us if reinforcement learning is the wrong choice?
Yes, and in our experience that is the more common finding. Most problems presented as reinforcement learning are single-step prediction or constrained optimisation, and the first three weeks are designed to establish which you have before the expensive part begins.
Often paired with this
Most clients combine two or three engagements from the Machine Learning & Predictive Analytics pillar. These are the ones that most often run immediately before or after.
Optimisation and Operations Research
Scheduling, routing and allocation solved against your real constraints, with solutions planners will actually run.
Read more →Dynamic Pricing Optimisation
Prices moved on measured elasticity, inside guardrails, proven by controlled experiment rather than by simulation.
Read more →Custom ML Model Development
A bespoke model for a problem no off-the-shelf product fits, benchmarked against a simple baseline and deployed properly.
Read more →Is this the right engagement?
Tell us what you are trying to build. If a different service fits better, or if you do not need us at all, we will say so.