Rural Clinics Are Auditing Their Diagnostic Models — and Finding the Training Data Wanting

When the Kilifi-based clinic network Uzima Health began buying diagnostic AI three years ago, the vendor accuracy figures were reassuring: 94% sensitivity on chest imaging, validated on tens of thousands of studies. What nobody asked was where those studies came from.

Uzima's own retrospective audit, covering 18,400 scans across 60 clinics in Kenya, Uganda, and Rwanda, put real-world sensitivity at 81%. The shortfall clustered in predictable places: patients with concurrent tuberculosis, images from older portable units, and anything captured outside a climate-controlled room. Two of the three vendors, Uzima says, could not produce a demographic breakdown of their training sets at all.

"We are not saying the tools are useless — they caught things our clinicians missed," said Dr. Faridah Okeyo, who led the audit. "We are saying that a number produced in a teaching hospital in another hemisphere is not a promise about what happens in Kilifi at 4 p.m. with a generator running."

Uzima has begun publishing its audit methodology so other networks can replicate it, and has made local validation a condition of renewal in its next procurement cycle. If enough buyers follow, vendors will have to start treating deployment-site performance as a shipping requirement rather than a footnote — which is, in the end, how every other piece of medical equipment already works.