FormalPara Key Points

Difficulties finding and using information from clinical guidelines (especially those produced by local healthcare organisations) contribute to thousands of prescribing and medication administration incidents.

Some of these incidents are associated with substantial patient harm.

Further development and implementation of techniques that may be effective in preventing such incidents (such as iterative user testing) is required.

1 Introduction

Medication errors are common, with an estimated 237 million occurring each year in England alone, costing the National Health Service (NHS) £98 million and causing or contributing to an estimated 1708 deaths [1]. Many factors have been identified that can contribute to medication errors [2], one of which is the plethora and complexity of medication-related guidelines.

Guidelines can play a key role in ensuring healthcare professionals provide safe and consistent patient care, and can also themselves contribute to both prescribing [3,4,5,6,7,8,9] and medication administration errors [10,11,12,13]. For example, Jones et al. [14] previously analysed video recordings of medication preparation and administration in a simulated paediatric emergency setting to find which steps of using the NHS Injectable Medicines Guide were susceptible to misinterpretation and caused clinically significant medication errors. The study suggests that 70% of discrepancies in the guideline use process arose from steps involved in finding the correct guideline (with using the incorrect guideline for the patient’s age a significant issue), contributing to 14 medication errors. As a result of evidence such as this, written guidance that is contradictory, incomprehensible or otherwise of poor quality [15], as well as problems with instructions about procedures [16], have been included in two frameworks describing factors that contribute to patient safety incidents. Additionally, the clarity of writing and formatting of guidelines have been recognised as two of six key domains influencing their uptake [17].

Given the challenges of interpreting and using guidelines, several studies have explored iterative ‘user testing’ and subsequent modification of documents as a structured approach to improving the usability and utility of health-related guidelines. While much of this work has studied information aimed at patients and the public [18], more recent work has tested this approach with clinical guidelines aimed at healthcare staff [19,20,21]. For example, Jones et al. [21] found participants understood significantly more information when interpreting medication administration guidelines that had been modified following iterative user testing, and that nurses using the revised guidelines made fewer errors in an in situ simulation [22]. A cost-effectiveness analysis concluded that user testing and modifying this guideline had a 99% chance of being cost effective [23].

However, no published study has examined guideline-related errors across a whole health system or even a whole organisation. Previous studies have focused on specific scenarios requiring a limited number of guidelines. We therefore lack an overall picture of how and where mistakes are made with the use of guidelines, what medication errors occur as a result, and which guidelines are most often affected. An understanding of this would allow the identification and prioritisation of areas in which to apply user testing or other approaches to the improvement of guidelines to support patient safety.

We therefore aimed to address this gap in the literature. Our objectives were to identify guideline-related medication errors reported via a national incident reporting database; to describe the types of error, stages of medication use, types of guideline, drugs, specialties and clinical locations most commonly associated with guideline-related medication errors; and to make recommendations for both practice and research in this area.

2 Methods

We carried out a retrospective analysis of medication errors relating to clinical guidelines that had been reported to the National Reporting and Learning System (NRLS) for England and Wales.

2.1 Database and Searches

The NRLS collects data on patient safety incident reports that are voluntarily and anonymously reported by staff working in the NHS and other healthcare organisations in England and Wales. These reports include incidents and near misses. The report template allows the input of categorical data (e.g. incident type and location) as well as free-text fields that allow staff to describe the incident, its perceived causes and the actions taken. Incidents within the database are classified by the reporter as having been linked to death, severe harm, moderate harm, low harm or no harm.

To identify incident reports relating to guideline use, a free-text field search was conducted by the national NRLS team using the following search terms related to medicines guidelines commonly used in England and Wales: BNF, British National Formulary, Guideline, NICE, SPC, SmPC, Summary of Product Characteristics, CKS, Clinical Knowledge Summary/ies, Medusa, Injectable Medicines Guide and IMG. The search was restricted to incidents reported to the NRLS between 1 January 2016 and 31 December 2021 (inclusive) that had been classified by the reporter as medication incidents. The identified incident reports, including both categorical and free-text data fields, were then supplied to the research team.

2.2 Hierarchical Task Analysis (HTA)

Hierarchical task analysis (HTA), a method that describes the workflow of a task or activity [24], was used as the basis of this study. To develop this HTA, data from the identified incidents reported to have resulted in moderate harm, severe harm or death were first inductively coded to identify the stage of the process of guideline use to which the incident related. These inductive codes were then used to inform the adaptation of an HTA from a previous study that was specific to one guideline [14] to consider the use of guidelines in general. The adapted HTA describing the process of retrieving information from any medicines guideline is shown in Fig. 1.

Fig. 1
figure 1

The hierarchical task analysis describing the process of retrieving information from a medicines guideline

Full size image

2.3 Coding of Medication Incident Reports

A list of all medication incidents that were linked by the reporter to moderate or severe harm or death (608 reports), plus a random sample of 5100 linked to no harm and low harm, was generated using Microsoft Excel (Microsoft Corporation, Redmond, WA, USA). This sample size was based on pragmatic considerations and resources available. Each report was then coded by one of a team of five researchers (all final-year Master of Pharmacy students), based on the free-text descriptions of the incident, its causes and preventive actions. Any deviations from expected practice (as described by the HTA) when retrieving information from a medicines guideline were defined as ‘step discrepancies’. Each step discrepancy was then further coded using a subset of error modes (Box 1) adapted from those used in a previous study [14] that were drawn from a generic human error taxonomy [25]. To focus on the initial cause of difficulties using guidelines and to enable clear presentation of results without ‘double counting’, only the first discrepancy of a series of step discrepancies was coded. When a step discrepancy was identified, the guideline(s) to which it related was also coded. Separate codes were assigned to reports that did not describe a step discrepancy or those for which it was not clear whether or not a step discrepancy had occurred. Coding was supported by operational definitions agreed by the research team before coding commenced (electronic supplementary material [ESM] Tables S1 and S2).

Box 1 Error modes used to code step discrepancies from expected practice when retrieving information from a medicines guideline, as described in the hierarchical task analysis
Full size table

Prior to coding the data, the team of five researchers each read relevant articles from the research and practice literature and attended a series of meetings with experienced medication safety pharmacist-researchers (BDF, SG and MDJ) to discuss how the operational definitions should be applied. To evaluate interrater reliability of coding among the researchers, the team of five researchers, plus a sixth researcher with specific expertise in this area, first independently coded the same 100 randomly selected reports as a pilot. These six researchers then discussed cases where there was disagreement and agreed some amendments to the operational definitions (ESM Table S2). This revised framework was subsequently used to code the remaining reports.

Following this, the team of five researchers each coded 1000 no-harm and low-harm cases and approximately 120 of the moderate, severe and death cases. After coding was completed for all the cases, each reviewer double-coded a further randomly selected 100 cases to further establish the level of interrater reliability.

2.4 Data Analysis Processes

The coded data were analysed with descriptive statistics using SPSS (version 27; IBM Corporation, Armonk, NY, USA). The interrater reliability for the coding of error modes was calculated using Fleiss’ kappa. The frequencies of different types of step discrepancy, error mode, guideline and medicine were calculated. The total numbers of no-harm and low-harm discrepancies in the entire NRLS data extract were estimated using Eq. 1 (rounded to the nearest integer):

$${E}_{h,s}= {D}_{h,s}\times \frac{{T}_{h}}{{C}_{h}},$$
(1)

Eh,s = estimated total number of discrepancies from HTA step ‘s’ in the entire NRLS data extract linked to harm level ‘h’

Dh,s = number of incidents coded as discrepancies from HTA step ‘s’ linked to harm level ‘h’

Th = total number of incidents in the entire NRLS data extract linked to harm level ‘h’

Ch = number of incidents coded linked to harm level ‘h’

h = level of harm linked to an incident, either ‘no harm’ or ‘low harm’

s = HTA step number

This was not necessary for higher levels of harm, as all the reports extracted from the NRLS were coded. Cross tabulations of step discrepancies were made against the guideline involved, stage in the medicines use process, and the care setting.

2.5 Ethics

We used exclusively anonymous data and therefore after review through the University of Bath Ethical Implications of Research Activity process, no Ethics Committee review was required. Data-sharing agreements were in place between NHS England and each data analysis site. Identifiable data were encrypted and were only accessible from secured university systems.

3 Results

3.1 National Reporting and Learning System (NRLS) Database Search

Our search strategy identified 28,217 incident reports of a total of 1,238,649 medication errors reported to the NRLS over the 6-year study period [26]. Of these 28,217, 23 were reported to be linked to death, 69 to severe harm, 516 to moderate harm, 3968 to low harm, and 23,641 to no harm. All reports linked to death and severe and moderate harm were coded, along with the random sample of 5100 low- and no-harm reports, giving a total of 5708 reports coded (Fig. 2).

Fig. 2
figure 2

Summary of the incident report search and coding process followed during this study. HTA hierarchical task analysis, NRLS National Reporting and Leaning System

Full size image

3.2 Interrater Reliability

Fleiss’ kappa for interrater reliability during the pilot coding (100 reports each coded by six researchers) was 0.43. During the main coding, this was 0.46 (random sample of 500 reports, each coded by two researchers).

3.3 HTA Step Discrepancies

A total of 642 reports of the 5708 coded described an HTA step discrepancy leading to a medication incident (Table 1 and Fig. 2). Finding the right guideline was the HTA step most often associated with a step discrepancy. The remaining 5066 reports did not describe an HTA step discrepancy, although in many cases this largely reflected lack of information in the incident report. Scaling up using Eq. 1 produced an estimate of approximately 3200 HTA step discrepancies described in all 28,217 incident reports (Table 2).

Table 1 Number of incident reports coded as describing or not describing an HTA step discrepancy relating to use of guidelines
Full size table
Table 2 Estimated number of incident reports describing or not describing an HTA step discrepancy relating to use of guidelines in the total NRLS data extract
Full size table

All six error modes (Box 1) were involved in HTA step discrepancies (Table 3). The most frequent type of discrepancy was misunderstanding the correct guideline (HTA step 3, error mode R2). Other common types of discrepancy included mistakenly using the wrong guideline (HTA step 1, error mode R2), relying on memory rather than checking a guideline (HTA step 1, error mode R4), an organisation not providing an appropriate guideline (HTA step 1, error mode R5), using information from the wrong section of a guideline (HTA step 2, error mode R2) and a guideline not including the required information (HTA step 2, error mode R5). These were also the types of discrepancy associated with reports of death and severe harm (Table 3), along with an appropriate guideline being provided by the organisation but not found by staff (HTA step 1, error mode R1).

Table 3 Number of HTA step discrepancies coded with each error mode, and an example of each
Full size table

HTA step discrepancies were associated with five main types of guideline (Table 4), of which the most frequently identified were guidelines produced by a specific local organisation or group of organisations (‘local guidelines’).

Table 4 Number of HTA step discrepancies associated with specific types of guidelines
Full size table

Most reported HTA step discrepancies occurred during one of two stages of the medicines use process, i.e. prescribing or administration (Table 5). Similarly, the most common types of medication error associated with HTA step discrepancies were wrong or unclear doses, frequencies or quantities, and the omission of medicines (ESM Table S3).

Table 5 Number of HTA step discrepancies associated with each stage of the medicines use process and the associated types of guideline
Full size table

Fourteen medicines were mentioned in at least 10 incidents that described an HTA step discrepancy (Table 6 ESM Table S4), including four antibiotics and three anticoagulants.

Table 6 Medicines mentioned in at least 10 incidents that described an HTA step discrepancy (for a list of all medicines, see ESM Table S4)
Full size table

The vast majority of reported HTA step discrepancies occurred in acute hospitals (590 reports). Other settings associated with one or more discrepancies were ambulance services (19 reports), community services (19 reports), mental health services (9 reports), community pharmacies (3 reports), general practice (1 report) and learning disability services (1 report). Medical specialties were most frequently associated with HTA step discrepancies occurring in acute hospitals (Table 7).

Table 7 Number of HTA step discrepancies associated with various acute hospital specialties
Full size table

4 Discussion

A wide variety of difficulties occur in finding and using information in clinical guidelines, some of which contribute to patient harm. Difficulty finding the correct guideline was the most common type of guideline-related incident, and several deaths were reported in incident reports following guideline-related medication errors. Difficulties finding and using information were most frequently associated with guidelines produced by specific local organisations (particularly acute hospitals) in relation to either the prescribing or administration of typical ‘high risk’ parenteral drugs, particularly inappropriately omitted medicines and incorrect drug doses or frequencies. Approximately 3200 guideline-related incidents are a small proportion of the more than 1.2 million medication errors reported to the NRLS during the study period, let alone the 237 million medication errors estimated to occur in England each year [1]. However, this figure is likely to be a significant underestimate, as 40% of reports contained insufficient information to determine whether they were linked to difficulties finding and using information in clinical guidelines. In addition, it is likely that the initial search of the NRLS database did not recover all reports of guideline-related incidents, especially as awareness of the contribution to medication errors of difficulties finding and using information is likely to be low among incident reporters.

4.1 Comparison with Previous Literature

The present study makes an important contribution to the literature by providing the first detailed examination of how difficulties in the use of guidelines contribute to medication errors in clinical practice. Previous studies of medication errors have identified that difficulties using guidelines can contribute to both prescribing [3,4,5,6,7,8,9] and administration errors [10,11,12,13]. However, these have not considered the guidelines use process in sufficient detail to identify that problems with both finding and using information can contribute to medication errors. The exception is a study of simulated rather than actual practice, which identified 21 guidelines-use discrepancies contributing to a medication error, of which 14 discrepancies related to finding the guideline and seven discrepancies related to reading it [14]. Similarly, previous user testing studies have identified problems finding and understanding information in guidelines, but were not designed to investigate a subsequent contribution to medication errors [19, 20, 27, 28].

The present study is also the first to quantify the reported level of harm associated with guideline-related medication incidents in practice, although a previous in situ simulation study that compared the use of ‘standard’ and ‘modified following iterative user testing’ intravenous administration guidelines rated the potential clinical significance of 280 guideline-related errors [22]. Six were likely to result in severe consequences, 214 in moderate consequences and 60 in minor consequences. As this previous study considered only the high-risk route of intravenous drug administration, it is unsurprising that the present study (which includes all routes of drug administration) observed a lower proportion of harmful incidents.

The contribution of guidelines produced by local organisations to medication errors has not been investigated by previous studies, although this has been highlighted in relation to national guidelines such as the British National Formulary (particularly dose and frequency errors) [4, 29] and the NHS Injectable Medicines Guide [14, 22]. The present study is also the first to investigate all types of guideline-related medication incident and has thus demonstrated an approximately equal contribution to both prescribing and administration errors, whereas previous studies were not able to draw comparisons between types of medication error.

4.2 Implications for Practice

The results suggest that actions to reduce the contribution of difficulties finding and using information in clinical guidelines to medication errors may reduce medication-related harm. This especially applies to local guidelines relating to both the prescribing and administration of high-risk drugs in acute settings. Such work should focus on how easily accessible guidelines are, as well as the guidelines themselves. Both simplicity of the message given within guidelines and the formatting of guidelines may be important to address [17], and iterative user testing and subsequent modification of guidelines may be an effective technique to address these concerns [19, 20, 22, 27]. While local hospital guidelines were the most commonly identified as contributing to errors, and focusing on these is therefore important, the same principles should also be applied to all guidelines. In addition, those reporting, monitoring and investigating medication-related incidents should be aware of guidelines as a potential contributing factor and ensure details are reported and appropriate actions taken.

4.3 Implications for Future Research

The process of iterative user testing and modification of intravenous drug administration guidelines has been shown to reduce the frequency of medication administration errors and is a cost-effective approach [22, 23], but this has not been investigated for the prescribing errors linked to local guidelines that were prevalent in the findings of the present study. This question should therefore be addressed by future research, as user testing and subsequent modification has the potential to improve local prescribing guideline design and thus prevent medication errors. Such research should also focus on the development of user testing processes that are feasible given the resources available within individual healthcare providers. While iterative user testing and subsequent modification has been shown to make it easier for health professionals to find and understand information within one specific guideline [19, 20, 27], it does not address the process of finding the correct guideline to use, which contributed to many of incidents reported in the present study. Future research should therefore investigate whether techniques such as iterative usability testing of wider guideline information systems [28] are able to prevent further medication errors. Given the limitations of research based on voluntary reporting of medication incidents, future studies utilising prospective qualitative or quantitative data collection from staff involved in incidents would give greater certainty about the characteristics and prevalence of guideline-related medication errors. This is especially important in sectors such as primary care, which are underrepresented in the voluntary incident report data used in the present study. In addition, all future research into the causes of medication errors should consider the potential contribution of difficulties finding and using information in guidelines.

4.4 Strengths and Limitations

A retrospective analysis of a national database has the advantage of giving a broad pool of evidence across different locations and settings within England and Wales. However, incident reports have a number of limitations. The most significant of these is underreporting, with as few as 0.12% of patient safety incidents reported [30,31,32], leading to underestimation of the number of medication errors. In particular, rates of reporting are influenced by local organisational culture and awareness, and are much higher from secondary care. In addition, it is only mandatory for NHS organisations to report incidents leading to death or severe harm to the national NRLS system. Reports from patients and carers are generally not included. Incident reports are often submitted before completion of a local investigation and hence there may be missing data, especially on contributory factors. Therefore, many reports did not contain sufficient information to determine whether or how guidelines contributed to errors. Furthermore, the level of harm resulting from the incident may not have been clear at the time of report, hence NRLS database reports may relate to actual or potential harm based only on the judgement of the reporter.

It is likely that the initial database search missed some relevant reports. An inductive approach to this free-text field search, with additional search terms identified from the results of initial searches, might have increased the sensitivity of this stage. The fact that so many errors were related to guidelines, despite these limitations, strengthens our finding that guidelines are an important area to address in medication safety.

Another limitation was the relatively low interrater reliability for coding to the stages of the HTA process. This may also have been due to the limited level of information in the reports. As a result of this, findings need to be interpreted with some caution. As only the first of a series of step discrepancies occurring in one incident was coded, any chains of discrepancies were not recorded. Finally, the database relates to England and Wales only, therefore it is uncertain as to how far the findings are generalisable to other countries.

5 Conclusion

Difficulties finding and using information in clinical guidelines (especially those produced by local healthcare organisations) could contribute to thousands of prescribing and medication administration incidents, some of which are associated with substantial patient harm. Further development and implementation of techniques that may be effective in preventing such incidents (such as iterative user testing and subsequent guideline modification) is required.