DENEB SYSTEMSA correlation filter tracker that keeps small targets
Correlation filter trackers are the classic fast way to follow an object through video. Sky Tracker's correlation profile takes 2.8 ms per 1080p frame on one CPU core, and on 224 labelled public clips it keeps the target's centre within 20 px on 85% of frames, where OpenCV's KCF manages 46% and MOSSE 21%. Add the optional ML check and it reaches 90%.
One bird, five trackers
Every tracker starts from the same box on a bird far off against moving clouds. KCF gives up within a second and MOSSE within three. When the hand-held camera swings, CSRT loses the bird and MIL drifts onto empty sky. Sky Tracker's correlation profile stays on it.
The close-up in the corner shows exactly what Sky Tracker's box contains.
Correlation filter trackers compared
| Tracker | Public dataset 224 clips | Field footage 18 clips | ms / frame 1080p |
|---|---|---|---|
| Sky Tracker correlation with ML | 90% | 65% | 11.7 |
| CSRT (OpenCV) | 87% | 48% | 19.8 |
| Sky Tracker correlation | 85% | 60% | 2.8 |
| KCF (OpenCV) | 46% | 18% | 2.1 |
| MOSSE (OpenCV) | 21% | 30% | 0.2 |
Share of labelled frames with the reported centre within 20 px of the label; a frame the tracker reports as lost counts as a miss. Time is the tracker's update call per frame on one core of an Apple M3 Pro. Method, NanoTrack, MIL and box overlap: the full OpenCV tracker comparison.
What it does differently
It says when it has lost the target
Confidence is 1 while tracking and 0 once the target is gone, instead of a box drawn on the background. Your controller can hold rather than chase noise.
Built for small, distant targets
Tuned on drones, aircraft and birds a few pixels wide, where KCF and MOSSE with OpenCV’s defaults give up on most clips.
An optional second opinion
With ML on, NanoTrack (a 1.8 MB neural tracker) runs alongside. Where the two agree you get the correlation box; where they disagree an arbiter picks one from appearance, contrast, motion and both scores, and restarts the tracker that drifted.
Moving cameras
On the command line, --correlation-motion rejects a box that moves with the background instead of with the target, coasts through the gap and re-locks on the target’s own motion.
Use it from the command line, Python or C++
# The correlation tracker on its ownsky_tracker --source video.mp4 --bbox 469,409,26,38 --profile correlation --csv tracks.csv# With the ML check on every frame, or on every 3rd frame while both agreesky_tracker --source video.mp4 --bbox 469,409,26,38 --profile correlation --mlsky_tracker --source video.mp4 --bbox 469,409,26,38 --profile correlation --ml --ml-every 3import sky_trackertracker = sky_tracker.Tracker("correlation") # correlation filter alonetracker = sky_tracker.Tracker("correlation", ml=True) # plus NanoTracktracker.lock(frame, (469, 409, 26, 38))result = tracker.update(next_frame)print(result.center(), result.confidence, result.lost)sky_tracker::sdk::TrackerOptions options;options.ml = true; // add NanoTrack's second opinionoptions.mlEvery = 1; // check every frame (the default)sky_tracker::sdk::Tracker tracker("correlation", options);More in the docs and on theC++ library page.
Questions
What is a correlation filter tracker?
It learns a filter from the target’s appearance in the first box, then looks for the strongest response to that filter in each new frame, using fast Fourier transforms. MOSSE, KCF and CSRT in OpenCV all work this way. It is fast because one frame costs a few transforms, not a neural network.
Is Sky Tracker’s correlation profile KCF or CSRT?
No. It is its own correlation filter tracker, written for small, low-contrast targets, with its own loss detection. On the same 224 public clips it keeps the target on 85% of frames, against 46% for KCF and 87% for CSRT, at 2.8 ms a frame against 2.1 and 19.8 ms.
When should I turn on ML?
On the hardest footage: clutter, fast camera motion, targets that change shape. It costs about 12 ms a 1080p frame instead of 3. If that is too slow, --ml-every N checks every Nth frame while the two trackers agree; on fast footage the box can then jump between checks, so compare on your own video.
Does ML need a GPU or extra files?
No GPU. It needs OpenCV 4.7 or later with the dnn module and two small model files that ship with the SDK. The Linux and macOS Python wheels include both; sky_tracker.has_nanotrack says whether a build has them.
Correlation or default?
default is a template tracker tuned for small, fast targets in the sky and leads on our fast field footage. correlation works on any target and leads on the public dataset. Try both on labelled frames of your own camera; the free demo runs both.
Try it on your own footage
The free desktop demo runs correlation, correlation with ML and default on ten seconds of any video.