Microsoft Research and the University of Alicante release PadChest-GR dataset with support from Centaur Labs

Researchers from Microsoft Research and the University of Alicante released PadChest-GR (Grounded-Reporting) in late 2024, an innovative dataset designed to improve the quality of generative AI models for chest X-ray (CXR) imaging. The team used the Centaur platform to complete all the annotations for this dataset, and we’re thrilled to have been able to contribute to their success. This dataset is now available to researchers globally.

‍

‍

Grounded Radiology Report Generation (GRRG): Evaluating model quality

If Radiology Report Generation (RRG) aims to create free-text radiology reports from clinical images, Grounded Radiology Report Generation (GRRG) takes it a step further by including the localization of individual findings in the image. The 2024 paper introduces both the first model to demonstrate the power of GRRG (MAIRA-2) as well as the task of GRRG and the output of a "Grounded Radiology Report."

‍

The MAIRA-2 research team defines a "grounded radiology report" as "a list of sentences from the Findings section [of a radiology report], each describing at most a single observation from the image(s), and associated with zero or more spatial annotations indicating the location of that observation if appropriate." An example of a "grounded radiology report" is below.

‍

‍

‍

By spatially grounding radiological findings, AI teams will be able to more easily verify the quality of the draft radiology reports their models generate. This verification is essential, as model quality and explainability are critical to building both clinician and patient trust in AI, particularly in generative AI.

‍

Today, there are many CXR image datasets that are labeled for diagnosis and finding classification tasks or that come with the associated text-based radiology reports for automated draft report generation. Some datasets also include spatial annotations to localize labels (for finding, anatomy, or device; e.g., ‘pneumothorax’) or single finding phrases, such as ‘left retrocardiac opacity.’ 

‍

What has been missing - and needed - to enable AI teams to build GRRG models are datasets that have both the spatial annotations and the direct links to the complete sets of descriptive sentences from the findings.

‍

‍

PadChest-GR: The first manually curated dataset for GRRG

PadChest-GR is the first manually curated dataset for Grounded Radiology Report Generation (GRRG). It includes:

‍

• 4,555 CXR studies  (3,099 abnormal, 1,456 normal)
• Bilingual reports (English and Spanish) 
• 7,037 positive and 3,422 negative finding sentences
• Dual-reader bounding box annotations for positive findings
• Categorical labels for finding type, location, and progression

‍

Our collaboration with the PadChest-GR team

We collaborated closely with researchers from Microsoft Research, the University of Alicante, and the rest of the team to ensure seamless annotation of this novel dataset. Radiologist annotators used our HIPAA-compliant annotation platform to complete all data annotation.

‍

Annotation was completed in two stages:

‍

1. Study-level quality control: The study was assessed for image quality, report quality, selection of the prior image, and list of findings; also for the presence of unfiltered protected health information or pediatric patients, or if lateral projection was needed to assess findings. Annotators used the original text-based radiology report, as well as the list of extracted positive finding sentences, in this evaluation.
2. Box annotations for each extracted positive finding: Only images that met the quality bar progressed to stage 2. If a positive finding sentence was localizable, annotators drew one or more bounding boxes around the area(s) where it was visible in the image

‍

For both stages, every study or finding was analyzed independently by two professionals. The frontal image was always displayed beside the prior image (when available) so findings regarding progression could be identified.

‍

‍

The development of PadChest-GR was also supported by Microsoft Research, the Department of Radiology at University Hospital Sant Joan d’Alacant, Universitat d'Alacant, MedBravo, and the University of Cambridge. The research was financially supported by the University of Alicante-Microsoft research collaboration, which is funded by Microsoft.

‍

‍

Using PadChest-GR and building datasets for GRRG‍

You can read the complete preprint about the PadChest-GR dataset here:

PadChest-GR: A Bilingual Chest X-ray Dataset for Grounded Radiology Report Generation