Best practices to evaluate the impact of biomedical research software-metric collection beyond citations.

Autor:	Afiaz A; Department of Biostatistics, University of Washington, Seattle, WA, 98195, United States.; Biostatistics Program, Public Health Sciences Division, Fred Hutchinson Cancer Center, Seattle, WA, 98109, United States., Ivanov AA; Department of Pharmacology and Chemical Biology, Emory University School of Medicine, Emory University, Atlanta , GA, 30322, United States., Chamberlin J; Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, 84108, United States., Hanauer D; Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, 48109, United States., Savonen CL; Biostatistics Program, Public Health Sciences Division, Fred Hutchinson Cancer Center, Seattle, WA, 98109, United States., Goldman MJ; UC Santa Cruz Genomics Institute, University of California Santa Cruz, Santa Cruz, CA, 95060, United States., Morgan M; Roswell Park Comprehensive Cancer Center, Buffalo, NY, 14263, United States., Reich M; University of California, San Diego, La Jolla, CA, 92093, United States., Getka A; University of Pennsylvania, Philadelphia, PA, 19104, United States., Holmes A; Jonsson Comprehensive Cancer Center, University of California, Los Angeles, CA, 90095, United States.; Institute for Precision Health, University of California, Los Angeles, CA, 90095, United States.; Department of Human Genetics, University of California, Los Angeles, CA, 90095, United States.; Department of Urology, University of California, Los Angeles, CA, 90095, United States., Pati S; University of Pennsylvania, Philadelphia, PA, 19104, United States.; Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, 46202, United States.; Center for Federated Learning, Indiana University School of Medicine, Indianapolis, IN, 46202, United States., Knight D; Jonsson Comprehensive Cancer Center, University of California, Los Angeles, CA, 90095, United States.; Institute for Precision Health, University of California, Los Angeles, CA, 90095, United States.; Department of Human Genetics, University of California, Los Angeles, CA, 90095, United States.; Department of Urology, University of California, Los Angeles, CA, 90095, United States., Boutros PC; Jonsson Comprehensive Cancer Center, University of California, Los Angeles, CA, 90095, United States.; Institute for Precision Health, University of California, Los Angeles, CA, 90095, United States.; Department of Human Genetics, University of California, Los Angeles, CA, 90095, United States.; Department of Urology, University of California, Los Angeles, CA, 90095, United States., Bakas S; University of Pennsylvania, Philadelphia, PA, 19104, United States.; Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, 46202, United States.; Center for Federated Learning, Indiana University School of Medicine, Indianapolis, IN, 46202, United States., Caporaso JG; Pathogen and Microbiome Institute, Northern Arizona University, Flagstaff, AZ, 86011, United States., Del Fiol G; Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, 84108, United States., Hochheiser H; Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, 15206, United States., Haas B; Methods Development Laboratory, Broad Institute, Cambridge, MA, 02141, United States., Schloss PD; Department of Microbiology and Immunology, University of Michigan, Ann Arbor, MI, 48109, United States., Eddy JA; Sage Bionetworks, Seattle, WA, 98121, United States., Albrecht J; Sage Bionetworks, Seattle, WA, 98121, United States., Fedorov A; Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, 02138, United States., Waldron L; Department of Epidemiology and Biostatistics, City University of New York Graduate School of Public Health and Health Policy, New York, NY, 10027, United States., Hoffman AM; Biostatistics Program, Public Health Sciences Division, Fred Hutchinson Cancer Center, Seattle, WA, 98109, United States., Bradshaw RL; Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, 84108, United States., Leek JT; Biostatistics Program, Public Health Sciences Division, Fred Hutchinson Cancer Center, Seattle, WA, 98109, United States., Wright C; Biostatistics Program, Public Health Sciences Division, Fred Hutchinson Cancer Center, Seattle, WA, 98109, United States.
Jazyk:	angličtina
Zdroj:	Bioinformatics (Oxford, England) [Bioinformatics] 2024 Aug 02; Vol. 40 (8).
DOI:	10.1093/bioinformatics/btae469
Abstrakt:	Motivation: Software is vital for the advancement of biology and medicine. Impact evaluations of scientific software have primarily emphasized traditional citation metrics of associated papers, despite these metrics inadequately capturing the dynamic picture of impact and despite challenges with improper citation. Results: To understand how software developers evaluate their tools, we conducted a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We found that although developers realize the value of more extensive metric collection, they find a lack of funding and time hindering. We also investigated software among this community for how often infrastructure that supports more nontraditional metrics were implemented and how this impacted rates of papers describing usage of the software. We found that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seemed to be associated with increased mention rates. Analysing more diverse metrics can enable developers to better understand user engagement, justify continued funding, identify novel use cases, pinpoint improvement areas, and ultimately amplify their software's impact. Challenges are associated, including distorted or misleading metrics, as well as ethical and security concerns. More attention to nuances involved in capturing impact across the spectrum of biomedical software is needed. For funders and developers, we outline guidance based on experience from our community. By considering how we evaluate software, we can empower developers to create tools that more effectively accelerate biological and medical research progress. Availability and Implementation: More information about the analysis, as well as access to data and code is available at https://github.com/fhdsl/ITCR_Metrics_manuscript_website. (© The Author(s) 2024. Published by Oxford University Press.)
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu