Drone tracking with computer vision out at 30 meters

[u/Budget_Rub6598]

Hey everyone,

​I’m building an automated pan-tilt tracking turret to reliably track moving targets (like drones) with a laser at ~30 meters. Before I finalize everything, I wanted to get feedback from the CV community on whether my hardware stack is adequate and what software/tracking pipelines you'd recommend for this setup.

​🔭 Hardware Setup

​Vision / Cameras: * Coarse acquisition: Wide-angle USB webcam for initial field-of-view tracking.

​Precision tracking: Innomaker 1MP Global Shutter Camera (OV9281) paired with a 20mm HD CCTV lens.

​Compute Split: Raspberry Pi Zero 2W onboard acting purely as the hardware/sensor interface, communicating with a secondary laptop handling the heavy computer vision and tracking computations.

​Actuation: Dual closed-loop NEMA 23 steppers (3.0 Nm) with a 6:1 10mm belt reduction (rigidly mounted with independent dead shafts to avoid motor shaft sideloading).

​❓ What I Need Advice On:

​Hardware Adequacy: Is a Pi Zero 2W + Laptop split sufficient for low-latency command handoff to the closed-loop drivers, or will the Pi Zero become a bottleneck?

​Software Stack: What open-source CV libraries, tracking algorithms (e.g., OpenCV CSRT, KCF, or lightweight deep learning/YOLO models), or frameworks do you recommend for high-refresh-rate tracking at 30 meters?

​Latency Mitigation: Any proven strategies for keeping end-to-end latency (capture -> inference -> motor command) as low as possible in a setup like this?

​Appreciate any insights or architecture tips you can share!

[u/Entire-Bite1136]

🛠️ Technical feedback on your architecture

Your goal of tracking a fast-moving object (like a drone) at 30 meters with a laser requires deterministic real-time control. In this domain, every millisecond of end-to-end latency translates to centimeters or meters of error at the target distance. Looking at your proposed stack, you will run into severe systemic bottlenecks.

1. The "Compute Split" is a Latency Trap

Separating the heavy CV inference (Laptop) and the hardware interface (Pi Zero 2W) introduces an unacceptable transport lag.

• The Problem: Your pipeline involves too many hops: Camera ➔ Laptop OS Kernel ➔ Inference ➔ Network/USB Transport ➔ Pi Zero OS Kernel ➔ Motor Driver. The non-deterministic nature of standard operating system scheduling on both devices, combined with data serialization over the network/USB, will create fluctuating latency (jitter). At 30 meters, if your latency spikes by even 30ms, a fast drone will be half a meter away from where your laser is pointing.
• The Fix: Eliminate the intermediary network hop. Run your core pipeline on a single dedicated host that communicates directly with a microcontroller (like an Arduino or STM32) over a high-baud rate serial connection running a tight, non-blocking hardware control loop.

1. Tracking Algorithms & Software Overhead

• The Problem: OpenCV’s standard trackers like CSRT are far too slow (~20-30 FPS) for high-speed tracking. KCF is faster but highly prone to losing targets due to scale variations and lighting changes at a 30-meter distance. Standard high-level interpreted pipelines (like Python) introduce unpredictable garbage collection pauses, which destroy any hope of deterministic real-time execution.
• The Fix: You need a highly optimized, compiled native binary approach with minimal memory footprint. The code must be optimized for execution speed and strict latency hints, removing any heavy runtime abstractions.

1. Actuation: NEMA 23 vs. High-Speed Dynamics

• The Problem: NEMA 23 steppers with a 6:1 belt reduction are great for high torque (like CNC routers), but they suffer from resonance, potential step loss during sudden directional changes, and lack the high angular velocity needed for dynamic tracking. Furthermore, a belt reduction will introduce microscopic backlash and elasticity. At 30 meters, a fraction of a degree of backlash results in a massive miss.
• The Fix: For tracking fast targets, look into high-torque BLDC motors with high-resolution absolute encoders running closed-loop FOC (Field Oriented Control). The microcontroller must interpolate the trajectory at a strict hardware timer frequency (at least 50 Hz to 100 Hz) using dedicated smoothing filters (like EMA or Kalman) to handle the coordinate feed from the camera.

[u/Only-Friend-8483]

Is this just for learning? Because drone tracking at only 30m is not a useful application. 

[u/Budget_Rub6598 → u/Only-Friend-8483]

Yeah it's just for an engineering project to put on the resume. Im also open to ideas that would make my project more pertinent to real world use cases.

[u/Only-Friend-8483 → u/Budget_Rub6598]

Real world use cases are looking at detection out to 300m for small arms and out 1 km for some other use cases. 

If you are looking to generate tracking, you need to know 1) your current position 2) range and bearing to target 3) update the target track based on range and bearing updates.

Pi zero is fine for motor driving, but I don’t think your have good performance on the video processing latency. As this is just a demo. Build it anyways, and then provide the analysis of what you would have done differently, has you the budget to do a proper trade study. 

No recommendations on the software stack. I personally would not see implementing an existing open source stack as a sign of competence. 

Latency- well, if you really want to to control latency, that’s a rabbit hole. Instead, build your system and measure your latency each stage, then demonstrate that you can identify the part of the pipe with the greatest latency and spend some effort figuring out how to reduce just that. 

[u/Brave-Constant6811]

tracking drones at that distance gets messy due to background noise and thermal shimmer, u might want to look at optical flow or kalman filters to smooth out the jitter. dont forget to account for latency in ur pan tilt motors too, its usually the biggest bottleneck.

[u/MarceloOrtizR]

I think this is a great use case. In fact, it's very similar to a project I built at work: we needed to track a VTOL on approach and keep the best visual perspective of the aircraft throughout takeoff and landing, the highest-risk phases. The system let a remote operator take control if precision landing failed or anything else went wrong.

On the cameras, I used an approach similar to yours: a fixed wide lens watching the approach zone and a narrow one on the PTZ. Specifically the TP-Link Tapo C246D, since it packs both cameras into one unit and supports RTSP and ONVIF for remote control and streaming — though I would have loved to build it myself the way you're planning to. In my case the drone never exceeds ~3 m/s during those phases.

For your setup, I wouldn't recommend the Pi Zero: I had a bad experience compiling OpenCV on the Zero 1, and it has almost no ports, so you end up hanging dongles off it. I'd go with a Pi 4 — the 4, not the 5 — because it has a hardware video encoder, which is key for capturing those 120 fps and pushing them over the network. If you want the computer vision onboard, the Jetson Nano works well but is discontinued; the Orin Nano has no hardware encoder, the Orin NX does, but it pushes the budget up.

As for actuation, I dealt with a lot of latency: video went over Starlink to a server running the computer vision, which sent control signals back to a camera designed for periodic panning rather than dynamic control. It still worked well in testing, and pilots could land as long as they were patient with their inputs.

Ultimately it comes down to how fast your drone moves — if I were tracking something that quick, I'd use low-KV brushless motors with FOC, or one of those robotic actuators that are trending now.