Blog


— Via AIMed
The life-saving reason why medical annotation company Centaur Labs wants us all to start analyzing medical images
In 1987, Jack Treynor, Finance Professor at the University of Southern California, conducted an experiment with his class in an effort to prove market efficiency. Treynor asked each of his 56 students to estimate the number of jelly beans inside a jar. The jar contained 850 beans, and the median answer Treynor’s students gave was 870.
Only one student managed to give an estimate that was closer to the true value than the group median. The experiment became a classic example of the wisdom of crowds, where the average answer obtained from a group of individuals tends to be more accurate than the answers of the individuals themselves.
Erik Duhaime, Co-Founder and CEO of medical data labeling company Centaur.ai, knew the experiment. But when he studied it again during his PhD at the MIT Center for Collective Intelligence, he saw another potential use. At the center, researchers look at how humans and computers can be better connected to become more powerful.
“I was partly inspired by my wife, who was attending medical school and residency at that time,” says Duhaime. “My PhD research focused on how to aggregate the opinions of multiple experts. Particularly, overcoming the challenge of making the wisdom of crowds work for certain tasks like analyzing medical images, where some people might have the professional knowledge and skills to do so, while others do not.”
Read the full article at AIMed »
Noisy evaluation data undermines LLM performance, hides real risks, and wastes engineering effort. Metrics cannot fix flawed ground truth. High-quality, expert-labeled evaluation data aligns scores with real outcomes, enabling trustworthy decisions, regulatory readiness, and scalable AI systems. Centaur.ai delivers the expert data infrastructure LLMs require.
Listen to Co-founder and CEO Erik Duhaime talk about the origins of Centaur Labs and the future of medical data labeling.
Centaur.ai CEO Erik Duhaime joined SegMed’s Bites of Innovation to explain how healthcare AI teams can achieve data quality at scale. He discusses collective intelligence, why credentials do not guarantee labeling quality, competitive annotation, dynamic escalation using disagreement, confidence scoring, continuous QC, regulatory datasets, de-identification, and how to use LLMs without blind trust.