| Question 1: What limits use of autocontouring in clinical practice? |
| Reference: | Yang, J., Veeraraghavan, H., Armato, S.G., III, Farahani, K., Kirby, J.S., Kalpathy-Kramer, J., van Elmpt, W., Dekker, A., Han, X., Feng, X., Aljabar, P., Oliveira, B., van der Heyden, B., Zamdborg, L., Lam, D., Gooding, M. and Sharp, G.C. (2018), Autosegmentation for thoracic radiation treatment planning: A grand challenge at AAPM 2017. Med. Phys., 45: 4568-4581. |
| Choice A: | Poor quality of auto-segmentation results |
| Choice B: | Poor workflow integration |
| Choice C: | Poor image quality |
| Choice D: | (a) and (b) |
| Choice E: | (a) and (c) |
| Question 2: Commissioning AI-based auto-segmentation tools for clinical use should include the following: |
| Reference: | Cardenas CE, Yang J, Anderson BM, Court LE, Brock KB. Advances in Auto-Segmentation. Semin Radiat Oncol. 2019 Jul;29(3):185-197. |
| Choice A: | Extensive test for segmentation accuracy with local institution data |
| Choice B: | Extensive test for expected image types and patient anatomies |
| Choice C: | Seamless integration with treatment planning system |
| Choice D: | Clearly document the limitations on insufficient accuracy |
| Choice E: | All of the above |
| Question 3: Which of the following segmentation metrics is most sensitive to outliers? |
| Reference: | Taha, A.A., Hanbury, A. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 15, 29 (2015). |
| Choice A: | Dice similarity |
| Choice B: | Average boundary distance |
| Choice C: | Hausdorff distance |
| Choice D: | Jaccard index |
| Question 4: Which of the following segmentation metrics correlates best with correction time? |
| Reference: | Femke Vaassen, Colien Hazelaar, Ana Vaniqui, Mark Gooding, Brent van der Heyden, Richard Canters, Wouter van Elmpt. Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiotherapy, Physics and Imaging in Radiation Oncology, Volume 13, 2020, Pages 1-6
Kiser, K.J., Barman, A., Stieb, S. et al. Novel Autosegmentation Spatial Similarity Metrics Capture the Time Required to Correct Segmentations Better Than Traditional Metrics in a Thoracic Cavity Segmentation Workflow. J Digit Imaging 34, 541-553 (2021). |
| Choice A: | Dice similarity |
| Choice B: | Average boundary distance |
| Choice C: | Hausdorff distance |
| Choice D: | Surface DSC |
| Question 5: An in-house AI model trained to contour organs-at-risk using patients simulated and treated for non-small cell lung cancer can directly be applied clinically to patients undergoing radiotherapy for breast cancer since the organs-at-risk are largely the same. |
| Reference: | Sharp, G., Fritscher, K.D., Pekar, V., Peroni, M., Shusharina, N., Veeraraghavan, H. and Yang, J. (2014), Vision 20/20: Perspectives on automated image segmentation for radiotherapy. Med. Phys., 41: 050902. |
| Choice A: | True |
| Choice B: | False |
| Question 6: Assessing the impact of an AI model after clinical release should include |
| Reference: | Michael V. Sherer, Diana Lin, Sharif Elguindi, Simon Duke, Li-Tee Tan, Jon Cacicedo, Max Dahele, Erin F. Gillespie, Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review, Radiotherapy and Oncology, Volume 160, 2021, Pages 185-191, ISSN 0167-8140 |
| Choice A: | Analysis of geometric measures |
| Choice B: | Expert ratings of AI based contours |
| Choice C: | Quantification of time savings |
| Choice D: | Review of cases with poor segmentation performance |
| Choice E: | All of the above |
