Over recent years, developers and researchers have made progress in efficiently building AI applications. Google Research has contributed to this effort by providing easy-to-use embedding APIs for radiology, digital pathology and dermatology to help AI developers train models in these domains with less data and compute. However, these applications have been restricted to 2D imaging, while physicians often use 3D imaging for complex diagnostic decision-making. For example, computed tomography (CT) scans are the most common 3D medical imaging modality, with over 70 million CT exams conducted each year in the USA alone. CT scans are often essential for a variety of critical patient imaging evaluations, such as lung cancer screening, evaluation for acute neurological conditions, cardiac and trauma imaging, and follow-up on abnormal X-ray findings. Because they are volumetric, CT scans are more involved and time-consuming for radiologists to interpret compared to 2D X-rays. Similarly, given their size and structure, CT scans also require more storage and compute resources for AI model development.
CT scans are commonly stored as a series of 2D images in the standard DICOM format for medical images. These images are then recomposed into a 3D volume for either viewing or further processing. In 2018, we developed a state-of-the-art chest lung cancer detection research model trained on low dose chest CT images. We’ve subsequently improved the model, tested it in clinically realistic workflows and extended this model to classify incidental pulmonary nodules. We’ve partnered with both Aidence in Europe and Apollo Radiology International in India to productionize and deploy this model. Building on this work, our team explored multimodal interpretation of head CT scans through automated report generation, which we described in our Med-Gemini publication earlier this year.
Based on our direct experience with the difficulties of training AI models for 3D medical modalities, coupled with CT’s importance in diagnostic medicine, we designed a tool that allows researchers and developers to more easily build models for CT studies across different body parts. Today we announce the release of CT Foundation, a new research medical imaging embedding tool that accepts a CT volume as input and returns a small, information-rich numerical embedding that can be used for rapidly training models with little data. We developed this model for research purposes only and as such it may not be used in patient care, and is not intended to be used to diagnose, cure, mitigate, treat, or prevent a disease. For example, the model and any embeddings may not be used as a medical device. Interested developers and researchers can request access to the CT Foundation API, and use it for research purposes at no cost. We have included a demo notebook on training a model for lung cancer detection using the publicly available NLST data from The Cancer Imaging Archive.