Open Science Practices Need Substantial Improvement in Prognostic Model Studies in Oncology Using Machine Learning

Abstract

Objective: To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using machine-learning methods in the field of oncology.

Study design and setting: We conducted a systematic review, searching the MEDLINE database between 01/12/2022 and 31/12/2022 for studies developing a multivariable prognostic model using machine-learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices.

Results: We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data-sharing statements, with 21 (46%) indicating data were available on request to the authors and 7 declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code-sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their to model to be used in practice. The use of reporting guidelines was rare: 8 studies (18%) mentioning using a reporting guideline, with 4 (10%) using the TRIPOD statement, 1 (2%) using MI-CLAIM and CONSORT-AI, 1 (2%) using STROBE, 1 (2%) using STARD, and 1 (2%) using TREND.

Conclusion: The adoption of open science principles in oncology studies developing prognostic models using machine-learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science is needed for prediction research in oncology.