This page covers the engineering behind Pétanque Scanner. If you're after the app itself — what it does, what it costs, where to get it — the app page is the one you want.
The problem
Measuring the gap between boules and the jack needs a 3D position, accurate to the centimetre, on uneven ground. With nothing but a phone, none of the obvious routes hold up: a single image carries no scale, and depth APIs vary too much between devices to be trusted on fine gravel at a metre and a half. Placing a marker on the ground would fix the scale, but it would force an accessory on the player — which is precisely what the app is meant to replace.
The approach: accumulate rays
The user's natural movement becomes the source of information. Every camera frame runs through an object detector that finds the boules and the jack in 2D. Each detection is turned into a ray cast from the camera, whose pose comes from AR Foundation (ARCore / ARKit). Walking around the game accumulates dozens of rays from different viewpoints, and where they intersect is the 3D position of a boule.
The by-product is as useful as the result: how tightly the rays converge is a free confidence metric. Rays that cross cleanly mean a reliable position; scattered rays mean an unstable detection. That is the score shown live during the scan, and it is what stops the app from announcing a wrong number with confidence.
The data: fully synthetic
Shooting and hand-labelling a dataset covering every surface, every lighting condition and every finish of boule was not realistic for one person. So the training set is rendered in Unity, with domain randomisation and pixel-exact labels produced by the engine — no manual annotation at all.
A first pass on real photos set the floor: across 471 cleaned photos, moving from YOLOX tiny @416 to YOLOX-s @640 was worth +9.25 mAP@.50:.95 (67.19 → 76.44), and confirmed along the way that model capacity mattered more than resolution, augmentation or epoch count.
What worked, and what failed
Synthetic data turned out to be far less obliging than expected. Three experiments, one win and two losses:
- +1500 simple synthetic scenes: a win. Roughly +3.5 AP on the jack, the class that bottlenecks everything else.
- A richer generator with decor and clutter: a loss. The intent was to bring the renders closer to a real scene. It scored worse than the simple generator.
- 1500 scenes built around jack occlusion: a loss. And this is the dominant failure case. Jack recall dropped to 86.1% against 88.9% for the incumbent model. Experiment abandoned, model never shipped.
The lesson I take from it: with synthetic data, covering the distribution pays and targeting the hard cases punishes. I ran the occlusion experiment a second time, convinced I had botched the first — the "mine the hard examples" instinct is stubborn, and it was wrong here.
The shipped model
The on-device model is a YOLOX-s at 448 px, exported to ONNX (34 MB) and run by OpenCV DNN on the phone. It is evaluated on a validation set deliberately built to match real use: photos taken with a phone held over the pitch — not images scraped from the web, which over-represent competition shots and product photography.
| Class | Recall | Precision |
|---|---|---|
| Boule | 99.4% | 100% |
| Jack | 88.9% | 91.4% |
Methodology: confidence threshold 0.25, IoU > 0.5, greedy one-to-one matching against ground truth. The validation set holds only 34 photos — enough to pick between two models, not enough to publish as a definitive figure, and the first thing I would grow.
The remaining jack failures are nearly all the same two: too small in frame, or hidden behind a boule. That is an acceptable limit here because the app does not work on a single image but on a sequence — a missed frame is caught by the next ones.
The robustness work
Most of the development time went not into the model but around it: clustering successive detections into distinct boules, locking the scene onto the ground plane, filtering shaky poses before they pollute the accumulation, and offering a "magnet" mode to nudge a boule back into place when the light plays tricks. That is the gap between a demo that works and an app you're willing to publish.
On-device constraints
Everything runs on the phone: no image is uploaded, and there is neither a server nor a user account. The price is binary size — the ONNX models discarded along the way were pulled out of the shipped assets, giving about 145 MB back to the APK.
Availability
The app is available on Android and iOS:
See also: the app page, its privacy policy and terms of use.
