Encrypted | Login

 

Benchmarking methods, sharing solutions, and advancing patient care through collaborative competition.

An AAPM Grand Challenge

Overview

The  American Association of Physicists in Medicine (AAPM)  is sponsoring the “Grace Kim Memorial Grand Challenge: Advancing Total Body, Marrow, and Lymphoid Irradiation Treatment Planning (Grace Challenge)”. Organized by the AAPM Working Group on Grand Challenges (WGGC) and Task Group No. 379 (TG-379), the challenge honors the legacy of Grace Gwe-Ya Kim, PhD, whose leadership and mentorship advanced the practice of radiotherapy and collaborative innovation. We invite participants to develop and benchmark treatment planning strategies that improve plan dosimetry quality, planning efficiency, and treatment delivery efficiency for intensity-modulated total body irradiation (IM-TBI), total marrow irradiation (TMI), and total marrow and lymphoid irradiation (TMLI). Participants will receive a CT dataset, with pre-defined planning target volume (PTV) and organ at risk (OAR) structures and are asked to produce clinically deliverable IM-TBI, TMI, and/or TMLI plans using manual, scripted, or fully automated approaches. The top performer in each category (IM-TBI, TMI, TMLI) will receive complimentary registration to present their methodology at the AAPM Annual Meeting & Exhibition in Vancouver, BC July 19–22, 2026 (in-person attendance required). Following the meeting, the organizers will summarize the results of this challenge in a journal publication.

Background

Total body irradiation (TBI), along with the emerging TMI and TMLI, are critical conditioning regimens in hematopoietic stem cell transplantation. Intensity-modulated approaches to TBI, TMI, TMLI (IM-TBI/TMI/TMLI) represent a significant advance over conventional TBI, employing sophisticated beam modulation to intensify target dose while sparing organs at risk, thereby improving cancer cytotoxicity efficacy and reducing normal-tissue toxicity. Planning IM-TBI, TMI, TMLI is inherently complex: plans must (1) cover exceptionally large target volumes, (2) protect numerous critical organs, and (3) remain deliverable within clinically acceptable treatment times from head to toe. In current clinical practice, IM-TBI/TMI/TMLI planning is time-consuming, often taking days. Furthermore, the plan quality can vary due to reliance on the experience of planner, the availability of optimization tools, and the time allocated for planning. These prolonged timelines and variability can hinder workflow efficiency and consistency of care.

Objectives

The proposed Grace Challenge aims to investigate best practices to improve plan dosimetry quality, planning efficiency, and treatment delivery efficiency for IM-TBI/TMI/TMLI through this large-scale planning study. Specifically, we aim to:

  1. Identify effective, generalizable planning strategies and automation approaches across treatment planning platforms, and
  2. Establish practical clinical benchmarks and create an openly available resource, including datasets, scoring routines, and method summaries, for community use.

Challenge Details

  1. Phase I (learn and practice phase)

    • Dataset: Participants will have access to three sets of CT images along with their corresponding structure sets through the Grace Challenge ProKnow Systems Platform. These CT images have been anonymized and represent a pediatric patient, a female adult patient, and a male adult patient. For each CT set, we provide three structure sets corresponding to IM-TBI, TMI, and TMLI, respectively. Participants can choose to participate in just one, two, or all three categories, by downloading and making 1-3 plans using 1-3 structure sets. Each structure set includes both the Planning Target Volume (PTV) and Organs at Risk (OARs). The treatment prescription is set at 12 Gy in 8 fractions (1.5 Gy per fraction) for all plans. Detailed dosimetric objectives and constraints can be found in “Dosimetry Scorecard” document.
    • Planning guidance document: To facilitate the learning of IM-TBI, TMI, and TMLI planning, we provide a “Planning Guidance” document. This document contains step-by-step instructions for manually creating IM-TBI, TMI, and TMLI plans from scratch, as well as links to the publicly available scripts to help automate treatment planning.
    • How to participate: In this phase, participants can download 3 CT and structure sets to learn and practice planning. Moreover, participants can use these data sets to develop semi-automatic or fully-automatic planning strategies. For each category, participants will upload only 1 plan (on the male adult patient), that is, the combined DICOM RTDose and DICOM RTPlan files to the Grace Challenge-ProKnow Systems Platform under the “Upload, Revise and Submit” tab to calculate their plan dosimetry quality score (QScore) in real-time based on the predetermined dosimetry scoring criteria. The other 2 patients (pediatric and female adult patients) are only for practice purposes and the system was not designed to provide real-time plan dosimetry quality scores for these two additional patients. Instructions on how to create acceptable DICOM RTDose and DICOM RTPlan files can be found in “DicomRT upload and download Instructions” document. The QScore will be displayed in real time. In this phase, the number of submissions to the Grace Challenge-ProKnow Systems Platform to calculate the plan's dosimetry score is unlimited.

  2. Phase II (competition and final score phase)

    • Dataset: A new CT image set and corresponding 3 structure sets for the competition phase will be provided.
    • How to participate: In this phase, participants will download the new CT and structure sets to test their developed planning strategies. Similar to Phase I, participants can upload their created DICOM RTDose and DICOM RTPlan files to the Grace Challenge-ProKnow Systems Platform an unlimited number of times under the “Upload, Revise and Submit” tab to display their plan QScore and iteratively try to improve their score, if desired. Once a satisfactory score is achieved, the plan can be submitted, and it will appear on the “Your Submission” tab. Participants can continue to upload plans under the “Upload, Revise and Submit” tab to view their scores without hitting “Submit”. If a better score was achieved, participants can submit the revised plan, and the later submission will replace any previous submissions. The last submission made before the deadline will be considered the final submission and will be used in the calculation of the final scores and ranking. Additionally, participants are required to submit the survey to calculate the TScore (delivery time or delivery efficiency) and PScore (planning automation or planning efficiency) for their final scores. If you develop any template, script and/or algorithms to automate planning and indicated their use in the survey, you are required to provide certain evidence for your template, script and/or algorithm to receive the appropriate PScore. One evidence could be a screen-capture of your template, script and/or algorithm. Submit your evidence files to gracechallenge@aapm.org. After the conclusion of the challenge, the challenge organizer will schedule a meeting with the top 5 performers from each category (IM-TBI, TMI, TMLI) to confirm the plan deliverability as well as ensure the functionality and performance of the developed template, scripts and/or algorithm.

    Required submission in Phase II:

    • One (combined) DICOM RTDose file and one (combined) DICOM RTPlan file
    • Survey
    • Evidence of templates, scripts and/or algorithms used to improve planning efficiency, if you indicate its usage in the survey.

    Note: Participants with missing items (in Phase II) will not be eligible for the award.

  3. Evaluation metrics

    • QScore: Plan dosimetry quality score (range: 0-100). Please see the comprehensive scoring criteria in the “Dosimetry Scorecard” document. This document gives a full breakdown of the plan scoring criteria. Some metrics are tagged as “critical” which means that if you do not meet the minimal requirement for dosimetric criteria, then your total plan score will be set to zero. The plans must be practical, realistic, and something you would plan for a real patient in your clinic. The challenge organizer will review plan parameters of all top scoring plans and reserve the right to disqualify any plans they suspect to not be deliverable, overly complicated, or too inefficient to be practical. Note that this challenge's scoring algorithm is a little more challenging than prior plan studies, so your total score may be less than other plan studies you may have done. That's okay! Remember that the scores are all relative, and seeing where you fall in relation to your peers is what matters, not the absolute score. This metric is automatically generated on the Grace Challenge-ProKnow Systems Platform in both Phase I and Phase II.
    • TScore: Treatment delivery efficiency score (range: 0-100). Please see the scoring criteria in the “Planning and Delivery Efficiency Scorecard” document. Briefly, the objectives outlined in this scorecard are designed to evaluate treatment delivery time based on plan parameters, such as numbers of isocenters, numbers of beams, total MU, etc. This metric is calculated by the organizers after the submission is closed in Phase II. Note that dose rate will not be a factor in calculating delivery efficiency to respect variations in dose rate preferences by different institutions and clinics.
    • PScore: Planning efficiency score (range: 0-100). Please see the scoring criteria in the “Planning and Delivery Efficiency Scorecard” document. Briefly, the objectives outlined in this scorecard are designed to evaluate the level of automation in the planning process. This metric is calculated by the organizers after the submission is closed in Phase II.
    • Final score: S = 0.7*QScore + 0.15*TScore + 0.15*PScore
    • Based on feedback received from participants in the practice phase, the organizers of the Challenge reserve the right to adjust the scoring criteria for the competition phase.

Participating in Challenge

  1. Who Should Participate

    • Medical physicists and dosimetrists,
    • Academic researchers and industry teams who develop planning and automation tools, and
    • Trainees and students interested in large-volume RT planning and optimization.

  2. Participants

    3. Each participant registers independently using his or her own name. Multiple participants from a single institution or clinic are permitted.

  3. Participation Method

    • Register via the Grace Challenge-ProKnow Systems Platform.
    • Get access to the data and guidance document after approval
    • After approval, on the Grace Challenge-ProKnow Systems Platform, select Organization: AAPM to get access to three studies: TBI, TMI, TMLI.
    • Review the "Plan Guidance", "Dosimetry Scorecard", and "Planning and Delivery Efficiency" documents in each study
    • Phase I: Download practice datasets to learn and develop your IM-TBI, TMI, TMLI planning strategies. Submit your plans to the Grace Challenge-ProKnow Systems Platform to see your QScore.
    • Phase II: Download the competition datasets to test your developed IM-TBI, TMI, TMLI planning strategies. Submit your plans to the Grace Challenge-ProKnow Systems Platform and email evidence of your templates, scripts, and/or algorithms to gracechallenge@aapm.org. On the Grace Challenge-ProKnow Systems Platform, you will see your QScore as soon as you submit the plan. Organizer will email you the overall ranking after concluding the challenge.

  4. Important Dates

    • December 1, 2025: Phase I starts. Registration opens. Practice datasets are available.
    • April 1, 2026: Phase II starts. Competition datasets are available.
    • May 01, 2026: Phase II ends.
    • May 14, 2026: Top 5 finalists in each of the 3 categories announced, and virtual meetings arranged for the finalists to demo in their respective treatment planning systems to ensure plan deliverability.
    • May 22, 2026: Participants are notified of their overall ranking. The winners for each of the three categories (IM-TBI, TMI, TMLI) are announced.
    • July 16-22, 2026: AAPM Annual Meeting & Exhibition: The winner for each of three categories (IM-TBI, TMI, TMLI) will present their work during a dedicated Challenge session.
    • September 2026: The Challenge organizers summarize the Grand Challenge in a journal paper.

  5. Results, Prize and Publication Plan

    • Results: Rankings will be determined by the Final Score, with separate leaderboards for each category (IM-TBI, TMI, TMLI). The winner in each category is the participant with the highest Final Score. If multiple participants share the highest Final Score, ties will be broken by QScore (higher is better), then by TScore, then by PScore, and finally by mean lung dose.
    • Prize: The winner will be awarded complimentary registration and are invited to present their planning strategies and results in the 2026 AAPM Annual Meeting & Exhibition (July 16-22, 2026, Vancouver, BC). In-person attendance is required. Travel expenses are not included.
    • Publication and Authorship:
      • The challenge organizers will compile the challenge results and perform detailed analyses across planning categories (IM-TBI, TMI, TMLI), delivery platforms, treatment planning systems (TPS), plan quality (QScore+TScore), planning efficiency (PScore), and other relevant dimensions. Top performers in each dimension will be recognized during the AAPM Grand Challenge presentation.
      • After the 2026 AAPM Annual Meeting & Exhibition, the organizers will prepare and submit a manuscript summarizing the challenge outcomes. Top performers will be acknowledged in the manuscript but will not be listed as co-authors.
      • Winners in the three planning categories are highly encouraged to submit their work to journals.

  6. Terms and Conditions

    7. The following rules apply to those who register and download the data:

    • Anonymous participation is not allowed.
    • Individual organizers of the Grace Challenge cannot participate. (Please review AAPM's participant COI statement.)
    • Entry by commercial entities is encouraged but should be disclosed.
    • Once participants submit their plans to the Grace Challenge, they will be considered fully vested in the challenge, such that their plan may become part of any presentations, publications, or subsequent analyses derived from the Challenge at the discretion of the organizers.
    • If identified as the top 5 in each category, participants will be asked to summarize their strategies in a 1-2 page document and submit them to the organizer.
    • Data and code downloaded from this site may only be used for the purpose of scientific studies and may not be used for commercial purposes.  Please acknowledge the source of data by:
      • <AAPM Task Group Report 379: technical guidelines for total body irradiation, total marrow irradiation, and total lymphoid irradiation> under review
      • <TBD paper> to be submitted by the challenge organizers

Organizers

Contacts

For further information, please contact the Grace Challenge Committee (gracechallenge@aapm.org), or AAPM staff member, Emily Townley (emily@aapm.org). We welcome participants to submit suggestions. The suggestions proposed in Phase I (practice) could be incorporated into Phase II (competition).

Disclaimer

Dates, awards, and publication plans are subject to AAPM WGGC approval and may be adjusted based on participation and sponsor support. All data and materials are for research purposes only and not for clinical use.