How Computer Vision Spots Small Drones, and Why It's Hard

In short: Computer vision finds drones by running a trained object detector, usually a YOLO-family or R-CNN-family neural network, on every frame of a camera feed, then linking the detections over time with a tracker. The hard part isn't the model. It's that a distant drone may cover only a few hundred pixels, and birds look almost the same. The best published systems split each frame into tiles, generate extra training examples, and check that a detection holds up across several frames. In real deployments, cameras are rarely the only sensor. Radar or radio-frequency (RF) sensors usually find the object, and the camera confirms what it is.

Key takeaways

• Detection is a computer vision problem: find a small object in a busy frame, fast enough to keep up with video.
• Size is the main enemy: in the Drone-vs-Bird challenge dataset, most labeled drones are smaller than 32 by 32 pixels.
• Birds are the main false alarm. Researchers name them as the biggest source of confusion.
• Thermal infrared helps at night and against cluttered skies, and has its own public benchmarks.
• Cameras are one sensor of several. Radar, RF and acoustic sensors each cover a weakness of the others.
• Students can build a working version with a webcam, Python, OpenCV and a small indoor drone.

Why anyone needs to detect drones

Airports, stadiums, prisons, power plants and large events all need to know when a drone is nearby. Most drones broadcast Remote ID, but not every drone in the air is compliant, and a broadcast only helps if someone is listening. Detection gives the people responsible for a site a picture of what's flying around them.

In 2021 the FAA picked five airports, including Atlantic City International in New Jersey, to test drone detection and mitigation technologies in a real airport environment (AAAE summary). In December 2023, an FAA official said 62% of the detection technologies and 33% of the mitigation technologies completed the program (Defense Daily). Detecting drones reliably in a real environment is still an open engineering problem.

This article covers detection only: seeing and identifying a drone. Interfering with a drone in flight is a separate matter, tightly restricted by US law, and not something students or schools do.

The sensor lineup, and where cameras fit

No single sensor does the whole job, so real systems combine them.

Sensor	Good at	Weak at
Radar	Long range, any weather, day or night	Cost; small birds can look like small drones
RF (radio) sensors	Picking up the link between a drone and its controller; sometimes locating the pilot	Drones flying pre-programmed routes with no radio link
Acoustic (microphones)	Hearing propellers behind trees or buildings	Short range; wind and traffic noise
Cameras (visible and thermal)	Confirming what an object actually is; a picture people can act on	Shorter range; fog, glare and darkness for visible cameras

A common pattern is to let radar or RF raise the alert, then point a camera at the spot to confirm it. That makes the camera's job classification as much as detection: is this a drone, a bird, a balloon or a plastic bag?

Why it's hard for a camera

Drones are small, fast, and look like birds. The Drone-vs-Bird Detection Challenge, run since 2017 by the WOSDETC workshop, publishes the difficulties plainly:

• Tiny targets. In the 2020 dataset, drone sizes range from 15 pixels to over a million, but most are under 32 by 32 pixels. A standard detector trained on everyday photos isn't built for objects that small.
• Birds. The organizers call birds the main disturbing object. At a few dozen pixels, a gull and a quadcopter have a similar size, shape and path.
• Moving cameras and distance. Sequences filmed from a moving camera, and very distant drones, were the hardest for every team.
• Busy backgrounds. Trees, buildings and clouds produce shapes and motion that look like targets.

Look-alikes aren't only birds. In a school hallway, a small drone flying next to a ceiling fixture of about the same size and color was enough to confuse two standard detectors.

How the detectors work

Almost every modern system uses a convolutional neural network trained on labeled video frames. There are two main families:

• One-stage detectors, like the YOLO ("You Only Look Once") family, predict boxes and labels in a single pass. They're fast enough to run on live video, which is why they're popular.
• Two-stage detectors, like Faster R-CNN, first propose regions that might hold an object, then classify each one. They're slower but often more precise on small objects.

Both families have won the challenge. In 2020, first place went to a two-stage Cascade R-CNN at 80% average precision, and second to a YOLOv3 model at 79.8% that split each frame into tiles. In 2025, the 8th edition was won by a YOLOv11 system (Laroca et al.) built on three ideas worth learning:

1. Tiling (multi-scale processing): run the detector on the whole frame and on zoomed-in pieces of it, then merge the results. A drone that's 20 pixels wide in a full frame is much larger relative to a tile.
2. Copy-paste augmentation: cut drones out of training images and paste them onto new backgrounds, so the model sees far more examples than were filmed.
3. Temporal consistency: a real drone shows up in frame after frame along a smooth path, while a lot of false alarms flicker. Checking several frames removes many of them.

We tried the first idea on one of our own photos, using a standard detector trained on everyday objects. Run on the whole photo, it found nothing. Run on a 640-pixel tile, it found the drone, but it called it a kite, because its training data has no drone class. That's why real systems also train on drone datasets.

Thermal cameras

Thermal infrared cameras see heat, not light, so they work at night and pick out warm motors and batteries against a cool sky. The Anti-UAV workshop series built public benchmarks for this. The first edition, at CVPR 2020, released 160 video sequences in visible and infrared. The second, at ICCV 2021, grew to 280 thermal infrared sequences. A newer dataset, CST Anti-UAV (2025), has 220 thermal sequences and more than 240,000 labeled boxes, focused on tiny drones in complex scenes. These benchmarks also include decoys like kites, balloons and birds, because that's what real skies contain.

From detection to tracking

A detector looks at one frame at a time, and a tracker turns those snapshots into a path. The classic approach is tracking-by-detection: run the detector on each frame, then use a Kalman filter to predict where each object should be next and match new detections to existing tracks. A track gives you speed and heading, which helps with the bird problem. Birds flap, glide and turn in ways most drones don't. A good track also lets the system keep its lock when the detector misses a frame.

Where the research is heading

• Synthetic data. Researchers render drones in simulators like AirSim and Unity to create training images that would be expensive to film. A 2023 review cites work where a YOLOv5 model performed reasonably after training only on synthetic images. NVIDIA's Isaac Sim has a tool, Replicator, built for this: it renders labeled images with randomized lighting, textures and positions.
• Visible plus thermal. Combining both cameras, even when they aren't perfectly aligned, covers day, night and glare.
• Running on the edge. Moving detection onto small computers mounted at the camera, so a site doesn't need a server room.
• Fusing sensors with learning. Models that take radar, RF, acoustic and camera data together and output one confident answer.

Does computer vision use reinforcement learning?

Mostly no. Drone detectors are trained with supervised learning: every training image comes with the right answer, a box around the drone, and the model is corrected toward it. Reinforcement learning (RL) is for problems where nobody can label the right answer at each step and you can only score the outcome, like flying a racing lap. None of the detection methods above use RL. The two still meet in three places:

• In the same machine. A detector sees, and an RL-trained controller acts on what it sees. The Swift racing drone, which beat human champions in 2023, used a gate detector trained on labeled images feeding a flight controller trained with PPO, the same RL algorithm used to fine-tune chatbots.
• Behind the training tricks. Google's AutoAugment (2018) used PPO to search for the best image augmentations, the same family of trick as copy-paste augmentation. Before that, neural architecture search (2016) used RL to design an image network that matched the best human-designed ones.
• In the newest models. Researchers now fine-tune vision models with RL on the score they're actually judged by. Visual-RFT (2025) rewarded a vision-language model for how well its predicted box overlapped the true one and beat its baseline by 21.9 points on a COCO detection test with only two examples per class.

For the RL side of the same problem, see our explainer on how reinforcement learning teaches drones and robots.

How students can try it

The core ideas fit in a classroom: a webcam, a laptop, Python and a small indoor drone. A project that builds in steps:

1. Motion first. Use OpenCV background subtraction to flag anything moving against a still background. It's simple, and it shows right away why motion alone isn't enough: people, curtains and shadows trigger it too.
2. Add a trained detector. Fine-tune a small pretrained YOLO model on a public drone dataset, or on frames you label yourself.
3. Measure where it breaks. Fly a small drone toward and away from the camera and record the distance where detection drops off. Then try tiling and see how much farther you get.
4. Add tracking. Connect detections across frames with a simple Kalman filter and compare false alarms before and after.

It touches the skills drone and robotics employers ask for: Python, computer vision, working with data, and testing ideas against real hardware. Fly safely and within the rules. Recreational flying outdoors needs the free TRUST certificate, and anything that isn't purely recreational needs a Part 107 remote pilot certificate.

Where Drone Edge fits

We build drone courses for schools that start with the rules and safety every operator needs and lead into hands-on flying and building. If you teach and want projects like this in your program, see how we work with schools.

Questions answered

Can a camera tell a drone from a bird?

Often, but not from one frame at long range. Systems get better results by tracking the object over several frames and looking at how it moves, and by pairing the camera with radar or RF sensors.

What is YOLO?

YOLO ("You Only Look Once") is a family of fast object detection models that predict boxes and labels in a single pass over an image. It's widely used for drone detection because it runs in real time.

How far away can a camera detect a drone?

It depends on the lens, the camera's resolution, the drone's size and the weather. Reviews give optical ranges of roughly half a kilometer to a few kilometers for good equipment. A webcam in a classroom gets a few meters to tens of meters.

Is it legal to detect drones?

Passively watching or listening, the way a camera or a Remote ID receiver does, is generally fine. Jamming, spoofing or otherwise interfering with a drone in flight is restricted under US law to specific authorized agencies. Check current rules for your situation.

Do I need a thermal camera to learn this?

No. A regular webcam teaches the core skills. Thermal comes in when you work at night or need to cut down on visual clutter.

Last updated: October 2026. Research results cited are from the papers linked above. Benchmarks and challenge results change every year.