REPORT-GUIDED PSEUDO-LABEL GENERATION AND TRANSFORMATION-BASED DETECTION FOR CHEST X-RAY PATHOLOGY LOCALIZATION WITHOUT MANUAL ANNOTATION

Author

• James Antwi Adinkrah, Southern Illinois University CarbondaleFollow

Date of Award

5-1-2026

Degree Name

Master of Science

Department

Computer Science

First Advisor

Ahmed, Khaled

Abstract

Automated detection and segmentation of thoracic abnormalities in chest radiographs remains a critical challenge in medical imaging, primarily due to the scarcity of pixel-level annotations. This thesis presents a report-guided multimodal framework that eliminates the need for manual radiologist annotations by leveraging free-text radiology reports to generate pseudo ground-truth segmentation masks for two clinically significant pathologies: atelectasis (lung collapse) and cardiomegaly (cardiac enlargement).The proposed pipeline integrates Grounding DINO, an open-set object detection model, with MedSAM, a medical image segmentation model fine-tuned on over one million medical image-mask pairs. Radiology reports from the MIMIC-CXR dataset (approximately 377,000 chest X-rays) are parsed and summarized to produce disease-specific text prompts. These prompts guide Grounding DINO to localize pathological regions, and the resulting bounding boxes are used as prompts for MedSAM to generate pixel-level segmentation masks. A tiered quality control system filters the generated masks, yielding 5,006 high-quality image-mask pairs. Downstream segmentation models, including SegFormer-B5 and Swin Transformer-Base, are trained on these pseudo labels using MMSegmentation. SegFormer-B5, trained on the Grounding DINO-generated dataset, achieves a mean Intersection over Union (mIoU) of 43.17%, with per-class IoU of 26.63% for atelectasis and 19.40% for cardiomegaly. A complementary detection study benchmarks three DETR-family transformer architectures DETR, RF-DETR, and Conditional DETR on a three-class subset of the VinBigData Chest X-Ray dataset (Aortic Enlargement, Cardiomegaly, Pleural Thickening). RF-DETR achieves the strongest overall performance with an F1 score of 0.636, precision of 0.814, and accuracy of 0.775, driven by its DINOv2 backbone and exponential moving-average training. A key finding is that over 87% of annotated images contain duplicate multi-radiologist bounding boxes, identifying annotation merging via Weighted Boxes Fusion as the highest-impact improvement. This work demonstrates that clinically meaningful segmentation and detection models can be developed without manual annotations, paving the way for scalable computer-aided diagnosis systems.