Deep Learning for Inverse Problems: Spectral Computed Tomography Image Reconstruction (DL-spectral CT)

An AAPM Grand Challenge

Overview

The American Association of Physicists in Medicine (AAPM) is sponsoring a Grand Challenge on deep-learning for image reconstruction leading up to the 2022 AAPM Annual Meeting. The DL-spectral CT Challenge follows up on last year’s DL-sparse-view CT Challenge. DL-spectral CT will provide an opportunity for investigators in CT image reconstruction, using data-driven or iterative techniques, to compete with their colleagues on the accuracy of their methodology for solving the inverse problem associated with spectral CT acquisition. Please note, an individual from each of the two top-performing teams will be required to present on their results during the AAPM 2022 Annual Meeting (July 10-14, 2022, Washington, DC) Grand Challenge Symposium and will be awarded complimentary meeting registration in order to attend. Following the Annual Meeting, results from all of the challenge participants will be highlighted in a challenge report.

Background

Last year’s DL-sparse-view CT Challenge [1] tested the ability of deep-learning-based image reconstruction to solve a CT-related inverse problem for which a solution is known to exist, by exploiting gradient sparsity in the true images. For the 2022 DL-spectral CT Challenge, we address an inverse problem that is of current interest and is challenging for any image reconstruction technique. Background information on the DL-spectral CT Challenge with the details of the spectral CT physics modeling will be included on the challenge website.

Briefly, this challenge will focus on a form of spectral CT known as dual-energy CT, where the subject is scanned by X-ray beams with two different spectra. One way to accomplish such a scan is to vary the kilovoltage peak (kVp) as the X-ray source circles the scanned subject; in particular this challenge models an ideal form of fast kVp-switching where the X-ray tube voltage alternates between two settings for consecutive projections. For each kVp setting, the beam spectrum is broad and the polychromatic nature of the X-ray beam makes quantitative imaging in CT quite challenging when only one kVp setting is used. With dual-energy CT, i.e. two kVp settings, it is possible to reconstruct quantitative images. If the subject is known to be comprised of only two tissues, the density map of each tissue can in fact be reconstructed accurately from dual-energy CT data. This challenge focuses on reconstructing tissue maps from dual energy CT data when there are three tissues present in the scanned subject.

As with DL-sparse-view-CT, the DL-spectral-CT set-up models the breast CT application. Dual-energy CT scans are simulated for a breast model [2] that contains three tissue types: adipose, fibroglandular, and calcification. The calcifications are assumed to be composed of hydroxyapatite [3]. Truth images and simulated data will be provided for 1000 cases so that participants can decide to either use a data-driven or optimization-based technique.

The DL-spectral CT Challenge seeks the image reconstruction algorithm that provides the most accurate reconstruction of the adipose, fibroglandular, and calcification spatial maps from dual-energy CT data.

[1] E. Y. Sidky and X. Pan, “Report on the AAPM deep-learning sparse-view CT grand challenge,” Medical Physics (early access: https://doi.org/10.1002/mp.15489), 2022
[2] I. Reiser and R. M. Nishikawa, “Task-based assessment of breast tomosynthesis: Effect of acquisition parameters and quantum noise,” Medical Physics, Vol. 37, pp. 1591-1600, 2010. (http://doi.org/10.1118/1.3357288).
[3] B. Ghammraoui, Ah. Zidan, A. Alayoubi, As. Zidan, and S. J. Glick, “Fabrication of microcalcificiations for insertion into phantoms used to evaluate x-ray breast imaging systems,” Biomedical Physics and Engineering Express, Vol. 7, article number 055021, 2021. (https://doi.org/10.1088/2057-1976/ac1c64).

Objective

The overall objective of the DL-spectral CT Challenge is to determine which deep-learning or optimization-based technique provides the most accurate recovery of a three-tissue test phantom from ideal 512-view dual-energy transmission data with no noise. To this end, we will provide 1000 data/tissue-map pairs based on a 2D breast CT simulation that can be used for training the algorithm. During the validation phase, 10 cases without the true tissue maps will be provided so that participants can try out their algorithms in an unlimited fashion and post results on a leaderboard. In the final two weeks of the challenge -- the test phase -- testing data will be provided for 100 cases without the corresponding ground truth images. Participants will submit their reconstructed tissue maps for these testing cases.

Get Started

Register to get access via the Challenge website
Download the data/image pairs after approval
Train or develop your image reconstruction algorithm
Download the testing data
Submit your results

Important Dates

Mar 17, 2022: Training set release
Mar 31, 2022: Release of 10-case validation set for the challenge leaderboard
May 17, 2022: 100-case Testing set release
May 31, 2022: Final submission of test results (midnight UTC, 5pm PST)
June 7, 2022: Top two teams invited to present at Grand Challenge Symposium
July 10-14, 2022: Grand Challenge Symposium, AAPM 2022 Annual Meeting
August 2022: Challenge report will be drafted and datasets made public

Challenge Data

Figure 1: Top row: fast kVp-switching dual-energy CT images reconstructed using standard processing and filtered back-projection. The tissue contrast for both images is different as the 50 and 80 kVp images are shown in the gray scale windows of [0.19,0.35] and [0.18,0.26] cm-1, respectively. Bottom row: ground truth images in the form of the X-ray linear attenuation map at 60 KeV (left) and a tissue label map (right) with adipose (blue), fibroglandular (green), and microcalcifications (red).

The details of how the simulation images and data are generated will be provided on the DL-spectral CT Challenge website. The 512x512 pixel training images are from a breast phantom simulation that has complex random structure modeling fibroglandular tissue in circular cross-section of a breast model (see bottom row of Figure 1). Also, a variable number of microcalcification-like objects of variable location and shape amplitude are inserted into the phantom. X-ray circular fan-beam CT acquisition is simulated where the X-ray source spectrum alternates between 50 and 80 kVp for successive views. As there are a total of 512 projections evenly distributed over the circle, there are 256 views acquired at each of the kVp settings and the respective projections are interlaced. Using standard negative logarithm processing and filtered back-projection (FBP) image reconstruction results in the images shown in the top row of Figure 1, which show streaking and cupping (non-uniform background gray level) artifacts and thus do not provide an accurate quantitative map of the subject. The goal of the competing dual-energy CT image reconstruction algorithms is to estimate the tissue label maps shown in the RGB composite image in the lower right of Figure 1. Image reconstruction algorithms need to overcome the streaking/cupping artifacts and exploit the different tissue contrasts in the 50 and 80 kVp transmission datasets to arrive at the tissue-map images. (Note that the microcalcifications appear larger in the 60 keV monochromatic image than in the tissue label map; this is a result of the highly attenuating nature of the Hydroxyapatite compound used to model the microcalcifications.)

Figure 2: Gray scale material maps for the breast phantom; blow-ups of a region of interest are shown in the top row and the full images are shown in the bottom row. Grayscale maps range from 0 to 1, where 0 and 1 indicate, respectively, no and full occupation of the designated material. At the interface of materials there is partial occupation due to finite with of the pixels; within the breast phantom support the material maps sum to 1, except for the outer edge of the skin line (modeled with fibroglandular tissue) where the phantom transitions to air.

For the challenge input data we provide the 256 view x 1024 detector-pixel X-ray transmission data (not shown) for 50 and 80 kVp dual-energy acquisitions. Also provided as input data are the FBP images obtained after standard processing (example shown in the top row of Figure 1). The image reconstruction algorithms will estimate material maps shown in the RGB-color composite image of Figure 1 and in the separated gray scale images of Figure 2.

We will generate 1,000 training cases, providing the input data (transmission and FBP images), and the output truth data (tissue maps). DL-spectral CT participants can choose to use whatever image reconstruction approach they wish to. For those interested in a deep-learning approach, participants can choose whether they will take an end-to-end (dual-energy transmission data to tissue map images), an artifact-removal (dual-energy FBP images to tissue map images) approach, or some combination of the two. Unlike DL-sparse-view CT, all geometric and physical parameters of the simulation will be provided in DL-spectral CT so that participants can generate transmission data from the tissue maps. This also means that all participants will be able to generate additional training data on their own that will be consistent with the challenge dual-energy CT data model.

In the training phase, the DL-spectral CT participants will have access to the 1,000 training data cases, transmission/FBP-images/tissue-maps. In the validation and testing phases, 10 and 100, respectively, new cases will be generated, and the participants will be given only transmission/FBP-image data. Final test rankings of the image reconstruction algorithms will be based purely on RMSE over the 100 test material map images. During the validation phase, ten validation data sets will be released for the purpose of creating a leaderboard, which will be visible only from March 17th to May 17th, the day when the testing set is released. Final results test results may not reflect the leaderboard ranking because the final testing phase depends on RMSE performance over a test set containing 100 images. During the validation phase, the number of submissions per participant will be unlimited, but during the test phase only three submissions will be allowed. Test phase submission is expected to be accompanied by a technical note describing the DL-spectral CT image reconstruction algorithm.

Quantitative evaluation

Submitted test-phase reconstructed tissue-map images will be evaluated by computing

Root-mean-square-error (RMSE) averaged over the 100 test cases:

where r and t represent the triplet of reconstructed and test tissue-map images, respectively, and n is the number of pixels in a single image.
Worst-case 25x25 pixel ROI-RMSE over all 100 test tissue-map images:

where b_c is a masking indicator function for the 25x25 ROI centered on coordinates c in the image, m=625 is the number of pixels in the ROI.

Mean RMSE will be the primary metric that determines the algorithm ranking. In case of a numerical tie (to be determined based on the distribution of results with the mean RMSE), the worst-case ROI-RMSE will be used to rank the algorithms.

Results, prizes and publication plan

At the conclusion of the challenge, the following information will be provided to each participant:

The evaluation results for the submitted cases
The overall ranking among the participants

The top two performing teams:

Will present their algorithm and results at the 2022 AAPM Annual Meeting
Will receive complimentary registration for one individual on each team to attend the 2022 AAPM Annual Meeting
Will receive a Certificate of Merit from AAPM

A manuscript summarizing the challenge results will be submitted for publication following completion of the challenge which will summarize the approaches taken by participating teams. Note that participants will not be coauthors on the challenge report; the report will only briefly describe results and methods so that participants can publish their own manuscripts based on the DL-spectral CT Challenge.

Terms and Conditions

The DL-spectral CT Challenge is organized in the spirit of cooperative scientific progress. The following rules apply to those who register a team and download the data:

Anonymous participation is not allowed.
Participants from the host institution and colleagues of the organizers may participate, but they will not be eligible for prizes or recognition as a ranking participant.
Entry by commercial entities is permitted but should be disclosed.
Once participants submit their outputs to the DL-spectral CT Challenge organizers, they will be considered fully vested in the challenge, so that their performance results will become part of any presentations, publications, or subsequent analyses derived from the Challenge at the discretion of the organizers.
The downloaded datasets or any data derived from these datasets, may not be given or redistributed under any circumstances to persons not belonging to the registered team.
The full data, including reference data associated with the test set cases, is expected to be made publicly available after the publication of the DL-spectral CT Challenge. Until the official public release of the data, data downloaded from this site may only be used for the purpose of preparing an entry to be submitted for the DL-spectral CT Challenge. The data may not be used for other purposes in scientific studies and may not be used to train or develop other algorithms, including but not limited to algorithms used in commercial products.

Organizers and Major Contributors

Emil Sidky (Lead Organizer) (The University of Chicago)
Xiaochuan Pan (The University of Chicago)
Samuel Armato and the AAPM Working Group on Grand Challenges

Contact

For further information, please contact the lead organizer, Emil Sidky (sidky@uchicago.edu).