Pose Estimation and Activity Recognition
Understanding what people are doing rather than merely that they are present, using skeletal representations that support safety, ergonomics and process analysis without retaining identifiable footage.
Detecting that a person is in a frame is a solved problem. Determining whether they are lifting correctly, whether a two-person procedure was actually performed by two people, or whether someone has fallen rather than crouched, requires understanding movement over time, and that is a substantially harder question.
Pose estimation locates body keypoints such as joints in each frame; activity recognition interprets sequences of those poses over time to classify what a person is doing. Together they support safety monitoring, ergonomic assessment, procedure verification and process timing without necessarily retaining identifiable imagery.
The privacy advantage is real and worth designing for
Once pose is extracted, the analysis runs on coordinates rather than pixels. A skeleton shows that a person bent at the waist while lifting; it does not show who they are. Where the pose extraction happens on the device and only keypoints are retained, the privacy position becomes considerably simpler:
- No identifiable imagery is stored or transmitted, only anonymous coordinate streams.
- The purpose limitation is easier to demonstrate, because the retained data cannot support identification.
- Employee and works council conversations, which are frequently the binding constraint, become materially easier.
- Retention obligations are lighter, since keypoint data is not biometric identification.
We design for this by default in workplace deployments, because it is both the better privacy outcome and, in practice, the difference between a system being approved and being blocked.
How we build activity recognition
Pose extraction tuned to your viewpoint
Camera angle, distance, occlusion and clothing all affect keypoint quality, and industrial settings are harder than the datasets these models were trained on. We measure keypoint reliability on your own footage first, because everything downstream inherits its errors.
Model the sequence, not the frame
Activities are defined by movement over time. A single frame cannot distinguish sitting down from standing up, and a frame-by-frame classifier produces flickering output that no operational system can use. Temporal models with sensible smoothing are the minimum bar.
Define activities operationally
'Unsafe lifting' has to become a testable definition: trunk flexion beyond a threshold, sustained for a duration, while a load is present. We write these with your safety or process specialists, and the process of writing them frequently exposes that the organisation had two different definitions in use.
Accept and design for ambiguity
Some activities genuinely cannot be distinguished from a given viewpoint. We identify these during the feasibility stage and either change the camera position, accept the limitation explicitly, or route ambiguous cases for review rather than pretending to a certainty the data does not support.
Alert to what is actionable
The same alert-fatigue arithmetic applies here as in video analytics: an ergonomic system flagging every imperfect lift produces a number nobody reads. Aggregate reporting with trends by area and shift usually delivers more than individual alerts, and it avoids the surveillance framing that sinks these deployments.
Report on areas and processes, not individuals
The most successful deployments we have run report ergonomic risk by workstation and process step rather than by person. It produces the same operational insight, removes the individual-monitoring objection, and is the version that survives consultation with employee representatives.
Where it is applied
| Setting | What is recognised | What it supports |
|---|---|---|
| Manufacturing and warehousing | Lifting posture, reaching, repetition | Ergonomic risk assessment by workstation |
| Construction and heavy industry | Proximity to hazards, protective equipment use | Safety intervention and near-miss analysis |
| Healthcare and care settings | Falls, mobility, procedure steps | Incident response and care quality |
| Retail and hospitality | Queue behaviour, service interactions | Staffing and layout decisions |
| Sport and rehabilitation | Technique, range of motion, symmetry | Coaching and recovery tracking |
| Process industries | Sequence and duration of manual steps | Cycle time and standard work analysis |
How the engagement runs
Keypoint quality on your own footage is measured before any activity model is built on top of it.
Feasibility and definitions
Pose quality measured on your footage and viewpoints; activity definitions written with safety or process specialists.
Data collection and labelling
Sequences collected across shifts and conditions, labelled to the agreed definitions with agreement measured.
Temporal modelling
Sequence models trained and evaluated per activity, with ambiguous cases identified and handled explicitly.
Edge deployment and privacy
Pose extraction on device, keypoint-only retention, aggregate reporting designed with employee representatives consulted.
Pilot and handover
Limited area pilot with accuracy reviewed against observation, then rollout and handover.
What you receive
Movement understood well enough to act on, without a surveillance footprint.
Pose extraction pipeline
Deployed on device where possible, with keypoint quality measured per camera and viewpoint.
Activity definitions
Operational, testable definitions agreed with your safety or process specialists.
Temporal models
Per-activity accuracy with the ambiguous cases identified and their handling documented.
Privacy design
Keypoint-only retention, on-device processing and the consultation record.
Aggregate reporting
Risk and timing by area, workstation, process step and shift rather than by individual.
Monitoring and retraining
Drift in keypoint quality and activity distribution, with the retraining trigger.
Is this the right engagement?
Worth being direct. Pose Estimation and Activity Recognition is the wrong spend in some situations, and those are listed rather than buried.
Good fit if
- You need to know what people are doing, not just that they are present.
- Ergonomic, safety or process-timing questions are currently answered by manual observation.
- Cameras can be positioned to see the movement clearly.
- Aggregate reporting by area or process is acceptable and preferable.
- Employee representatives can be consulted before deployment.
Choose something else if
- The activities cannot be distinguished from any feasible camera position.
- The intent is individual performance monitoring, which we would advise against.
- Consultation obligations cannot or will not be met.
- Simple presence detection is sufficient, which is object detection.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
Does activity recognition require identifiable video?
No, and it should not. Pose extraction can run on the device with only skeletal keypoints retained, which supports the analysis without storing imagery that identifies anyone. That design is both the better privacy outcome and, in workplace settings, often the difference between approval and rejection.
How accurate is pose estimation in industrial settings?
Lower than in the clean datasets these models were trained on, because of occlusion, clothing, camera angle and lighting. We measure keypoint reliability on your own footage in the first fortnight, since every downstream activity model inherits those errors.
Can it tell the difference between a fall and someone crouching?
Often, using the movement over time rather than a single frame, which is exactly why temporal modelling is required. Some pairs of activities are genuinely indistinguishable from certain viewpoints, and we identify those during feasibility rather than discovering them in production.
Will employees accept this?
More readily when it reports by workstation and process rather than by individual, when only skeletal data is retained, and when representatives are consulted before installation. Deployments framed as individual monitoring get resisted, and in several jurisdictions consultation is a legal precondition rather than good practice.
How is this different from video analytics?
Video analytics detects objects and events across camera feeds, typically for security and operations. This interprets human movement over time to determine what someone is doing, for safety, ergonomics and process analysis, usually with skeletal rather than image data.
Often paired with this
Most clients combine two or three engagements from the Computer Vision pillar. These are the ones that most often run immediately before or after.
Video Analytics and Surveillance AI
Video analytics tuned against alert fatigue, with tracking across cameras and privacy designed in from the start.
Read more →Visual Quality Inspection and Defect Detection
Defect detection engineered from the imaging up, tuned to escape rate and false reject cost on your line.
Read more →AR and VR AI Experiences
Spatial experiences with AI inside, built to a frame budget and to a use case that justifies the hardware.
Read more →Is this the right engagement?
Tell us what you are trying to build. If a different service fits better, or if you do not need us at all, we will say so.