Skip to main content
SpringerLink
Log in

The Unseen Hand: AI-Based Prescribing Decision Support Tools and the Evaluation of Drug Safety and Effectiveness

  • Current Opinion
  • Published:
Drug Safety Aims and scope Submit manuscript

Abstract

The use of artificial intelligence (AI)-based tools to guide prescribing decisions is full of promise and may enhance patient outcomes. These tools can perform actions such as choosing the ‘safest’ medication, choosing between competing medications, promoting de-prescribing or even predicting non-adherence. These tools can exist in a variety of formats; for example, they may be directly integrated into electronic medical records or they may exist in a stand-alone website accessible by a web browser. One potential impact of these tools is that they could manipulate our understanding of the benefit-risk of medicines in the real world. Currently, the benefit risk of approved medications is assessed according to carefully planned agreements covering spontaneous reporting systems and planned surveillance studies. But AI-based tools may limit or even block prescription to high-risk patients or prevent off-label use. The uptake and temporal availability of these tools may be uneven across healthcare systems and geographies, creating artefacts in data that are difficult to account for. It is also hard to estimate the ‘true impact’ that a tool had on a prescribing decision. International borders may also be highly porous to these tools, especially in cases where tools are available over the web. These tools already exist, and their use is likely to increase in the coming years. How they can be accounted for in benefit-risk decisions is yet to be seen.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Shreve JT, Khanani SA, Haddad TC. Artificial intelligence in oncology: current capabilities, future opportunities, and ethical considerations. Am Soc Clin Oncol Educ Book. 2022;42:842–51. https://doi.org/10.1200/edbk_350652.

    Article  Google Scholar 

  2. Weissman GE. FDA regulation of predictive clinical decision-support tools: what does it mean for hospitals? J Hosp Med. 2021;16(4):244–6. https://doi.org/10.12788/jhm.3450.

    Article  PubMed  PubMed Central  Google Scholar 

  3. In-house manufacture of medical devices in Great Britain. 31 December 2020. https://www.gov.uk/government/publications/in-house-manufacture-of-medical-devices/in-house-manufacture-of-medical-devices.

  4. Tricco AC, et al. Implemented machine learning tools to inform decision-making for patient care in hospital settings: a scoping review. BMJ Open. 2023;13(2): e065845. https://doi.org/10.1136/bmjopen-2022-065845.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Tran KA, Kondrashova O, Bradley A, Williams ED, Pearson JV, Waddell N. Deep learning in cancer diagnosis, prognosis and treatment selection. Genome Med. 2021;13(1):152. https://doi.org/10.1186/s13073-021-00968-x.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Armando LG, Miglio G, de Cosmo P, Cena C. Clinical decision support systems to improve drug prescription and therapy optimisation in clinical practice: a scoping review. BMJ Health Care Inform. 2023;30(1): e100683. https://doi.org/10.1136/bmjhci-2022-100683.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Naor GM, et al. Screening for medication errors and adverse events using outlier detection screening algorithms in an inpatient setting. J Med Syst. 2022;46(12):88. https://doi.org/10.1007/s10916-022-01864-6.

    Article  PubMed  Google Scholar 

  8. Kessler S, et al. Economic and utilization outcomes of medication management at a large Medicaid plan with disease management pharmacists using a novel artificial intelligence platform from 2018 to 2019: a retrospective observational study using regression methods. J Manag Care Spec Pharm. 2021;27(9):1186–96. https://doi.org/10.18553/jmcp.2021.21036.

    Article  PubMed  Google Scholar 

  9. B Chang et al. ARPNet: antidepressant response prediction network for major depressive disorder. Genes. 2019;10(11):907. https://mdpi-res.com/d_attachment/genes/genes-10-00907/article_deploy/genes-10-00907-v2.pdf?version=1574076353.

  10. R Li et al. G-net: a recurrent network approach to g-computation for counterfactual prediction under a dynamic treatment regime. In: Proceedings of machine learning for health. 2021; PMLR, vol. 158, pp. 282–299.

  11. Jie Z, Zhiying Z, Li L. A meta-analysis of Watson for Oncology in clinical application. Sci Rep. 2021;11(1):5792. https://doi.org/10.1038/s41598-021-84973-5.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  12. Beaubier N, et al. Integrated genomic profiling expands clinical options for patients with cancer. Nat Biotechnol. 2019;37(11):1351–60. https://doi.org/10.1038/s41587-019-0259-z.

    Article  PubMed  CAS  Google Scholar 

  13. Hasan MM, Young GJ, Patel MR, Modestino AS, Sanchez LD, Noor-E-Alam M. A machine learning framework to predict the risk of opioid use disorder. Mach Learn Appl. 2021;6: 100144. https://doi.org/10.1016/j.mlwa.2021.100144.

    Article  Google Scholar 

  14. Koesmahargyo V, et al. Accuracy of machine learning-based prediction of medication adherence in clinical research. Psychiatry Res. 2020;294:113558. https://doi.org/10.1016/j.psychres.2020.113558.

    Article  PubMed  Google Scholar 

  15. LE Walker et al. The DynAIRx Project Protocol: Artificial Intelligence for dynamic prescribing optimisation and care integration in multimorbidity. J Multimorbid Comorbid. 2022;12:26335565221145493. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9761229/pdf/10.1177_26335565221145493.pdf.

  16. Baysari MT, et al. Supporting deprescribing in hospitalised patients: formative usability testing of a computerised decision support tool. BMC Med Inform Decis Mak. 2021;21(1):116. https://doi.org/10.1186/s12911-021-01484-z.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Perri GA, et al. MedSafer to support deprescribing for residents of long-term care: a mixed-methods study. Can Geriatr J. 2022;25(2):175–82. https://doi.org/10.5770/cgj.25.545.

    Article  PubMed  PubMed Central  Google Scholar 

  18. EM Powers, RN Shiffman, ER Melnick, A Hickner, M Sharifi. Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review. J Am Med Inform Assoc. 2018;25(11):1556–1566. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6915824/pdf/ocy112.pdf.

  19. Van Laere S, Muylle KM, Cornu P. Clinical decision support and new regulatory frameworks for medical devices: are we ready for it? a viewpoint paper. Int J Health Policy Manag. 2021;11(12):3159–63. https://doi.org/10.34172/ijhpm.2021.144.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Clinical decision support software: guidance for industry and food and drug administration staff. 2022. www.fda.gov/media/109618/download. Accessed 2 June 2023.

  21. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447–53.

    Article  PubMed  CAS  Google Scholar 

  22. American Society of Clinical Oncology. mCODE: minimal common oncology data elements. Alexandria: American Society of Clinical Oncology; 2020.

    Google Scholar 

  23. Nazer LH, et al. Bias in artificial intelligence algorithms and recommendations for mitigation. PLOS Digit Health. 2023;2(6): e0000278. https://doi.org/10.1371/journal.pdig.0000278.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Lyell D, Coiera E. Automation bias and verification complexity: a systematic review. J Am Med Inform Assoc. 2016;24(2):423–31. https://doi.org/10.1093/jamia/ocw105.

    Article  PubMed Central  Google Scholar 

  25. Oliva JD. Dosing discrimination: regulating PDMP risk scores. Cal L Rev. 2022;110:47.

    Google Scholar 

  26. Lu JH, Callahan A, Patel BS, Morse KE, Dash D, Shah NH. Low adherence to existing model reporting guidelines by commonly used clinical prediction models. MedRXiv. 2021. https://doi.org/10.1101/2021.07.21.21260282.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Alilou M, et al. A tumor vasculature-based imaging biomarker for predicting response and survival in patients with lung cancer treated with checkpoint inhibitors. Sci Adv. 2022;8(47):eabq4609. https://doi.org/10.1126/sciadv.abq4609.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  28. Kaushal A, Altman R, Langlotz C. Geographic distribution of US cohorts used to train deep learning algorithms. JAMA. 2020;324(12):1212–3. https://doi.org/10.1001/jama.2020.12067.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Petri H, Urquhart J. Channeling bias in the interpretation of drug effects. Stat Med. 1991;10(4):577–81.

    Article  PubMed  CAS  Google Scholar 

  30. Rotalinti Y, Tucker A, Lonergan M, Myles P, Branson R. Detecting drift in healthcare AI models based on data availability. In: Machine learning and principles and practice of knowledge discovery in databases. Cham: Springer; 2023. p. 243–58.

    Chapter  Google Scholar 

  31. Vasey B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924–33. https://doi.org/10.1038/s41591-022-01772-9.

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Gilead Sciences, Uxbridge, UK

    Harriet Dickinson

  2. Takeda, Cambridge, MA, USA

    Dana Y. Teltsch

  3. Merck Healthcare KGaA, Darmstadt, Germany

    Jan Feifel

  4. Institute of Pharmaceutical Sciences, ETH Zurich, Zurich, Switzerland

    Philip Hunt

  5. AstraZeneca, Gaithersberg, MD, USA

    Enriqueta Vallejo-Yagüe

  6. Institute of Primary Health Care (BIHAM), University of Bern, Bern, Switzerland

    Enriqueta Vallejo-Yagüe

  7. Kidney Center School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Arti V. Virkud

  8. AstraZeneca BeLux, Groot-Bijgaarden, Belgium

    Katoo M. Muylle

  9. Department of PharmacoTherapy, Epidemiology and Economics, Groningen Research Institute of Pharmacy, University of Groningen, Groningen, The Netherlands

    Taichi Ochi

  10. Center for Innovation in Medicine, Bucharest, Romania

    Taichi Ochi

  11. Ohio State University, Columbus, OH, USA

    Macarius Donneyong

  12. OM1, Inc., Boston, MA, USA

    Joseph Zabinski

  13. Boehringer Ingelheim, Binger Str. 173, 55218, Ingelheim am Rhein, Germany

    Victoria Y. Strauss

  14. Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Juan M. Hincapie-Castillo

Authors
  1. Harriet Dickinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dana Y. Teltsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Feifel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philip Hunt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Enriqueta Vallejo-Yagüe
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Arti V. Virkud
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Katoo M. Muylle
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Taichi Ochi
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Macarius Donneyong
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Joseph Zabinski
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Victoria Y. Strauss
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Juan M. Hincapie-Castillo
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

the Actionable AI group

Corresponding author

Correspondence to Harriet Dickinson.

Ethics declarations

Funding

This project was unfunded and workshops were conducted on personal time.

Conflicts of interest

Taichi Ochi, Macarius Donneyong, Enriqueta Vallejo-Yagüe, Arti V. Virkud and Juan M. Hincapie-Castillo have no conflicts of interest to declare. Harriet Dickinson is an employee and stockholder in Gilead Sciences Inc. Jan Feifel is an employee and stockholder of Merck KGaA. Katoo M. Muylle is an employee of AstraZeneca BeLux and her PhD research was supported by a grant from the Research Foundation Flanders (FWO) under grant number 1S39820N. Joseph Zabinski is an employee and stockholder of OM1. Victoria Y. Strauss is an employee of Boehringer Ingelheim Pharma GmBH & Co. KG. Philip Hunt was an employee and stockholder of AstraZeneca at the time of writing. Dana Y. Teltsch is an employee of Takeda, and owns stocks in Takeda and Aetion.

Ethics approval

Ethical approval was not required as this work does not contain any human subjects or patient-level data. Ethics approval was not sought for these workshops as no human data were involved.

Consent to participate

Consent to participate was also not required as no human subjects were involved.

Consent for publication

Consent for publication is not applicable as no patients were involved.

Availability of data and material (data transparency)

Data sharing is not applicable to this article as no datasets were analysed during the current study. No electronic supplementary material is provided.

Code availability

Code sharing is not applicable to this article as no code was generated during the current study. No electronic supplementary material is provided.

Authors’ contributions

HD conceived the idea for this manuscript and organised structured workshops. All authors decided on key themes and defined the scope of the manuscript, and HD, DT, JF, JZ, EVY, AV, KM, TO, MD, PH, VYS and JHC participated in workshops and subsequent offline discussions around key concepts. HD wrote the first draft from notes taken in the workshops. All authors reviewed the manuscript and approved the final version before submission.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dickinson, H., Teltsch, D.Y., Feifel, J. et al. The Unseen Hand: AI-Based Prescribing Decision Support Tools and the Evaluation of Drug Safety and Effectiveness. Drug Saf 47, 117–123 (2024). https://doi.org/10.1007/s40264-023-01376-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40264-023-01376-3

Search

Navigation