External quality assessment (EQA) of performance of laboratories participating in the European Antimicrobial Resistance Surveillance Network (EARS-Net)

This report includes a short conclusion on the capacities of the participating laboratories, and recommendations for improvement. All 30 European Union/European Economic Area (EU/EEA) countries participated in this EARS-Net EQA exercise. 

The aims of the EARS-Net EQA exercises are: 1) to assess the accuracy of species identification reported by individual participating laboratories; 2) to assess the accuracy of qualitative AST results reported by individual participating laboratories, and 3) to evaluate the overall comparability of routinely collected test results, between laboratories and EU/EEA countries. In EARS-Net EQA exercises, eligible laboratories are identified by national EARS-Net EQA coordinators, designated by the Coordinating Competent Body in each EU/EEA country. Participating laboratories identify the species of six bacterial strains and submit AST results for the antimicrobial agents included in EARS-Net surveillance, using the methods routinely applied. 

In 2024, the panel of six EQA strains consisted of Acinetobacter baumannii, Enterococcus faecium, Escherichia coli, Pseudomonas aeruginosa, Klebsiella pneumoniae , and Staphylococcus aureus (Table 1). The E. coli and the K. pneumoniae strains had been included in previous EARS-Net EQA exercises. The E. coli strain (‘2024 EARS-Net 3’) was the most challenging strain in 2022 (strain ‘2022 EARS-Net 2’) and 2023 (strain ‘2023 EARS-Net 1’) − i.e. the strain with the most incorrect results. The K. pneumoniae strain (strain ‘2024 EARS-Net 5’) was a challenging strain in 2023 (strain ‘2023 EARS-Net 2’) [2,3]. 

On 10 June 2024, the six strains were distributed via the national EARS-Net EQA coordinators to 980 laboratories in all 30 EU/EEA countries. An EQA webpage was opened to receive submission of results between 11 June and 11 August 2024. 

As in previous EARS-Net EQA exercises [2-4], concordance of species and AST interpretations with the expected results was defined as ‘excellent’ (≥95% of interpretations in concordance with expected results), ‘very good’ (>90% to <95%), or ‘good’ (>85 to ≤90%). There was also the category ‘satisfactory’ (>80 to ≤85%) for results that could be improved.

Results were submitted by 912 laboratories. Species identification was evaluated for the laboratories, and 5 408 (99.2%) of the 5 451 reported species were correct. There was ‘excellent’ concordance for each of the six strains (98.5 to 99.5% concordance). Two laboratories reported the wrong species for every submitted strain. 

The interpretation of AST results was only evaluated if the species had been correctly identified. The evaluation was performed according to the clinical breakpoints in the European Committee on Antimicrobial Susceptibility Testing (EUCAST) Clinical Breakpoints Tables v14.0 [6], with the EUCAST categories ‘susceptible, standard dosing regimen’ (S), ‘susceptible, increased exposure’ (I), and ‘resistant’ (R). 

In the 2024 EARS-Net EQA exercise, the scoring system for the evaluation of interpreted results included an assessment of the ‘level of difficulty’ and the ‘severity of error’ of the submitted AST result for each strain-antimicrobial agent combination. The scoring system was the same as in the 2023 EARS-Net EQA. There were two ‘levels of difficulty’ (‘easy’ and ‘difficult’), reflecting the magnitude of the risk of getting the AST result wrong. ‘Easy’results were those with expected AST results far from the breakpoint, where the categorisation was obvious. Conversely, ‘difficult’ results were those close to the breakpoint or inside the area of technical uncertainty (ATU), or those for which breakpoints had been recently changed or added. Consequently, the scoring system allocated a higher score to ‘difficult’ results than ‘easy’ results, and penalised errors for ‘easy’ results more severely than errors for ‘difficult’ results. The severity of error was divided into three levels: very major error (VME), which indicated reporting false susceptibility (i.e. reporting S or I, instead of R); major error (ME), which indicated reporting false resistance (i.e. reporting R, instead of S or I) and no error. The scoring system penalised VMEs more severely for ‘easy’ results than for ‘difficult’ results and did not penalise MEs if the test was considered ‘difficult’. 

The reported interpretations of AST results were evaluated for 910 laboratories (excluding the two laboratories that reported the wrong species for all submitted strains). Among the 54 044 AST results evaluated, the most frequently reported methods for AST had very good concordance with the expected result (Table 13). These were automated systems (55.8% of all tests, 91.5% of which were correct), followed by disk or tablet diffusion (26.0% of all tests, 91.9% of which were correct) and minimum inhibitory concentration (MIC) methods, including broth microdilution and gradient test (17.6% of all tests, 92.1% of which were correct).

Overall, the AST interpretations submitted achieved a 'very good' level of concordance with the expected results, with 91.7% (49 579 out of 54 044) correct. Otherwise, MEs and VMEs were observed for 4.1% and 4.2% of interpretations, respectively. At country level, all countries achieved a ‘very good ’ level of concordance with the expected interpretation of AST results except for two countries (Cyprus and Latvia) which achieved a ‘good’ level concordance. At laboratory level, 15.7% (n=143) of the laboratories achieved an ‘excellent’ level of concordance; 57.9% (n=527) achieved a ‘very good’ level of concordance; 23.4% (n=213) achieved a ‘good’ level of concordance; 2.5% (n=23) achieved a ‘satisfactory’ level (>80 to ≤85%), and 0.4% (n=4) were below the ‘satisfactory’ level (<80%).

There were 71 strain-antimicrobial agent combinations tested for antimicrobial susceptibility in the 2024 EARS-Net EQA exercise, and the vast majority had results in ‘excellent’ concordance with the expected results (n=53 or 74.6% of the combinations). A ‘very good’ level of concordance was achieved for four combinations (5.6%). 

Overall, the results of the 2024 EARS-Net EQA exercise did not show a systematic overestimation or underestimation of AMR in the EU/EEA, with deviations distributed across both types of errors (MEs and VMEs). However, these results show that there are still inconsistencies between laboratories and that there has been no improvement in the prediction of AST profiles for beta-lactam antimicrobials for the E. coli and K. pneumoniae EQA strains since the previous EARS-Net EQAs. The results also support a continuing trend, across different species, of difficulties in predicting AST results for aminoglycosides. The results imply that AST of S. aureus and P. aeruginosa bacterial isolates with difficult or unexpected resistance profiles can be problematic.

There were three situations where a specific method seemed to influence the percentage of correct results. The use of disk/tablet diffusion for AST of cephalosporins resistance in E. coli did not perform as well as other methods; the use of MIC methods (broth microdilution and gradient test) and of automated systems was not adequate for the prediction of cefoxitin susceptibility in S. aureus ; and gradient tests had particularly poor performance for the prediction of AST results of beta-lactam agents for P. aeruginosa. As standard practice, laboratories should confirm that their laboratory protocols are in accordance with the latest EUCAST recommendations and guidelines, applying the most recent EUCAST breakpoints. In addition, AMR surveillance and control activities should note and consider the specific deviations in AST results observed for each species and antimicrobial agent/group during this EQA exercise.