Markerless Motion Capture: Fixing Occlusion Dropouts in Fast Rotational Movements

The clip is eight frames long, maybe seventy milliseconds, right at the top of a golf downswing. On frame 214 the trail hip reads eighteen degrees of internal rotation. On frame 215 it reads negative fifty-one. By frame 218 it has settled back onto a number that fits the rest of the swing. Nobody's pelvis reverses direction and snaps back in under a tenth of a second, but the joint-angle graph says otherwise, and the peak pelvis rotational velocity pulled from that rep now reads north of five thousand degrees per second - a number no human hip has ever produced. The lead arm swept in front of the trail hip for exactly those frames, the pose model lost a point it never physically had to begin with, and it filled the gap with a guess now sitting in a training log next to real data, indistinguishable to anyone who only checks the summary number.

Markerless motion capture has no physical marker to lose - it estimates joint centers frame by frame from video, running a 2D pose-estimation model on each synchronized camera view, then triangulating those estimates into a 3D position wherever enough views agree. Occlusion happens when a body segment, a piece of equipment, or another person blocks a joint from enough views that triangulation can no longer resolve a reliable position.

What happens next depends on the vendor, and none of it announces itself as a problem. Some pipelines drop the keypoint outright, leaving a gap that only shows up if you go looking. Some hold the last known position, reading as a brief freeze rather than a spike. And some - the dangerous case - keep producing a confident-looking estimate anyway, because the underlying network was trained on thousands of typical human poses and will guess where a hip or shoulder should sit even when the actual pixels are occluded. That guess can be wildly wrong while carrying a deceptively high confidence value, because the model reports how plausible its guess looks against its training, not how occluded the joint truly is.

The tell that separates dropout from ordinary noise is the shape of the error. Noise is small, continuous jitter around a true value. Dropout is a step - in-trend, then a frame or two wildly off, then back as if nothing happened. Differentiate that step into an angular velocity, exactly what most software does for peak rotational speed, and the artifact gets worse: a position jump divided by one frame's duration produces a velocity an order of magnitude past anything biomechanically plausible.

Two effects stack in a fast rotational movement, each raising the odds of a dropout on its own.

The first is self-occlusion: as the body rotates through a large range in a short window, limbs cross in front of the torso and each other from the camera's view, even with nothing else in the room blocking the shot. A trail arm sweeping across the chest in a golf downswing, a stride leg crossing the trail hip during a pitching delivery, a throwing arm swinging behind the torso mid-spin in discus or hammer - each puts a joint behind another segment for part of every rep, from whichever camera angles do not happen to sit in exactly the right place at exactly the right instant. Colyer et al. (2018, Sports Medicine - Open), reviewing vision-based motion analysis toward markerless systems, name self-occlusion during fast, multi-planar movement as one of the central unresolved limitations still separating markerless systems from marker-based labs, worse for throwing and rotational sports than for movements confined to one plane, like walking or a controlled squat.

The second effect is motion blur: at high angular velocity a limb moves several pixels within a single exposure, softening the edges a pose model relies on to locate a joint, which lowers confidence even without literal occlusion. Both effects compound at exactly the moment that matters most - the acceleration phase into impact or release.

Kanko et al. (2021, Journal of Biomechanics) compared a markerless, multi-camera deep-learning system against a marker-based reference during overground gait and found agreement was not uniform across planes - transverse, rotational joint angles disagreed meaningfully more than sagittal flexion-extension angles, consistent with rotation being harder to resolve even without heavy occlusion. The caveat: walking is a slow, low-occlusion task filmed inside a full-coverage lab volume, so this is a floor for rotational-plane error under near-ideal conditions, not a ceiling - a field rig filming a throw or swing with fewer cameras should do worse.

Needham et al. (2021, Scientific Reports) evaluated several 2D pose-estimation-based triangulation methods against a marker-based reference for 3D joint-center accuracy and found error was not evenly distributed - it clustered around segments where a joint was partially occluded or moving quickly, running well above the error seen in clean, unobstructed frames of the same trial. Their setup used a fixed multi-camera lab volume and a defined set of test movements, so absolute figures will not transfer to a two-camera field rig filming a live pitch or swing, but the pattern - error concentrating where occlusion and speed overlap - is the same mechanism at work here.

The damage shows up in three places. Joint rotation angle takes the direct hit, reading a physically impossible value for the dropout's duration. Peak angular velocity takes a worse hit, because differentiating a position series amplifies single-frame error - a hip that never exceeds 500 deg/s can show a computed peak of 5,000 or more from one bad frame, and that inflated number is what lands in a training report, not the trace it came from. And event timing - the frame marking peak pelvis rotation velocity, used for kinematic sequencing or X-factor stretch - can shift onto the artifact frame instead of the true peak, corrupting downstream timing metrics even when the raw angle trace looks fine at a glance.

More cameras help, but the gain depends on where they sit relative to the specific rotation a movement produces, not just how many there are. A two-camera sagittal-plane rig, fine for a squat or treadmill gait trial, leaves a joint edge-on to both cameras at once for a real portion of any movement that rotates the trunk ninety degrees or more - exactly the geometry a golf swing or discus throw produces. The table below is a starting allocation, not a hard rule; a taller athlete or tighter filming space can still need more coverage than the minimums listed.

The target underneath all of these numbers is the same: keep every joint visible, unobstructed, in at least three cameras throughout the movement, so triangulation has a spare if one loses the joint. Frame rate matters separately from camera count - raising it shortens each dropout window, but it does not fix a genuine geometric occlusion the way an added camera angle does.

Run this on any rep from a fast rotational movement before trusting its peak angular velocity, and repeat it whenever camera count, placement, or the movement changes.

1. Pull per-frame keypoint confidence alongside the 3D angle output, not just the final smoothed trace.
2. Flag any frame where confidence drops below roughly 0.4-0.5 on the vendor's 0-1 scale, adjusted to that model's typical clean-frame confidence.
3. Independently flag any frame-to-frame angle change implying an angular velocity above what the joint can plausibly produce - roughly 1,500-2,000 deg/s for hip and shoulder rotation, since even elite rotational sports rarely exceed a few hundred to just over a thousand deg/s at these joints.
4. Treat a frame as a genuine dropout only when both flags line up - either alone can be a real, if uncertain, estimate.
5. Re-scrub the footage at the flagged timestamp and confirm visually that an occluding limb, object, or another athlete is actually in frame.
6. Repair only the confirmed window with a cubic-spline interpolation, not a hold-last-value fix, and avoid bridging more than 4-5 frames at 120 fps (roughly 35-40 ms) without flagging the result as an estimate.
7. Recompute peak angular velocity after the fix; a change over 15-20% confirms the original number was dropout noise, not performance.

A collegiate golfer's downswing, filmed on a four-camera markerless rig at 120 fps, showed exactly this pattern at the top of the transition, right as the lead arm crossed the trail hip. Raw pelvis rotation, per-frame confidence, and implied angular velocity appear below alongside the corrected values.

Frames F4 through F6 carry confidence values between 0.19 and 0.33, well under the 0.4-0.5 trigger, and the angle swings from 12 degrees to negative 61 to negative 6 in three frames - physically incoherent, not a fast but genuine rotation. Re-scrubbing confirmed the lead forearm crossing directly in front of the trail hip marker on the camera closest to square, leaving only one partial view - not enough for confident triangulation. Raw velocity spikes as high as 8,795 deg/s, an order of magnitude past anything a human pelvis produces. After cubic-spline interpolation across the flagged frames, the corrected trace resumes a smooth trend near 542 deg/s into F7 - and F7's own raw value, captured once occlusion cleared, lands almost exactly where the interpolation predicted, confirming the repair reconstructed the true trajectory rather than just papering over the gap.