Enhancing large-scale clinical data analysis with Pair at the National Heart Centre Singapore

The National Heart Centre Singapore (NHCS) is using Pair, a fast and secure AI Assistant developed by Open Government Products (OGP), to revolutionise how it analyses vast amounts of clinical text data. This innovative project tackles the historically challenging task of extracting key insights from patient case notes, which comprise information that has been difficult to process due to its sheer volume, medical jargon complexity, and unstructured nature. The collaboration has already demonstrated significant success, with the potential to speed up data analysis, and produce not only more advanced evidence-based clinical decision-making but in less time.

The challenge: analysing thousands of patient case notes

NHCS faced a significant hurdle: a database of patient case notes multiple times larger than any other within the hospital, including those for diagnoses, demographics, or lab results. However, IT policy restrictions have made it challenging to develop in-house analytics for processing such a large volume of text data. The work that NHCS started with the Pair team was born out of this operational need to better analyse clinical data.

Figure 1: Lau Yee How leveraged Pair for large-scale clinical data analysis

One important type of information that NHCS chose to examine involved heart attack patients' increasing length of stay over the years. To determine the cause, Associate Professor Anders Sahlen, Senior Consultant, Cardiology, and Director of Echocardiography Core Laboratory, NHCS, along with Lau Yee How, Senior Manager, Singapore Cardiac Data Bank, NHCS, conducted a comprehensive audit across thousands of patient case notes retrospectively. Insights obtained were used to optimise chronic disease management, and plan for higher resource utilisation in hospitals. The key first step was to extract 19 components of the Charlson Comorbidity Index (CCI), a general predictor of one-year mortality risk.

Matching human auditors with 96% accuracy

Initial tests were conducted to evaluate Pair's accuracy in extracting key insights from clinical case notes. To do this, the NHCS team had Pair extract the necessary components of the CCI from the discharge summaries of 600 patients.

The results obtained were remarkable: Pair achieved an overall accuracy of 96% across all 19 CCI components when compared with manual reviews performed by human auditors. This level of accuracy proved that Pair was highly effective at reading and extracting key medical information from complex clinical text. The team also discovered that when Pair made errors, they were randomly distributed around a mean of zero, suggesting that any errors introduced would tend to cancel each other out when applied across numerous patient records, providing overall reliable results.

From 286 hours to 1 hour: a 99% efficiency gain

Following the successful pilot, NHCS worked with the Pair team to increase its processing ability of handling batch jobs of up to 5,000 case note entries simultaneously in a single document. This enhanced capability proved to be a game-changer for NHCS. With a cost optimisation and efficiency gain of 99.6%, the net lead time for 6,128 discharge summaries was reduced from 286 hours of manual interpretation to just 1 hour for prompt generation and upload in Pair. Previously, each manual interpretation required 2.8 minutes of clinician time on average. Additionally, multiple Pair batch jobs can be executed simultaneously without human intervention, with the entire batch processing completed in just 2.5 hours, and email notifications sent upon completion. This enables the NHCS team to focus on the high-value work of turning data into actionable insights, instead of spending time on routine tasks.

Figure 2: Batch job feature on Pair

Furthermore, the proven prompt framework created for the use case can be easily shared with other teams within NHCS and SingHealth. This allows for scaling and customisation while preventing duplicative development efforts, and lowering the barrier to entry for a wider range of users without the need for writing code or model training.

This new batch-processing capability allows NHCS to overcome a long-standing hurdle, while bringing wide-ranging benefits. While Pair will still be applied under oversight and input with a human in the loop to validate its output to ensure accuracy and trustworthiness, the ability to rapidly analyse large datasets holds great promise for advancing patient care. The absence of systematic bias when using Pair to derive CCI demonstrates the robustness of its analytical approach, which is important for scaling the application to large volumes of text entries.

Looking ahead

This initial success with the CCI has paved the way for future applications. The NHCS team has moved on to apply Pair to multiple new areas where extraction from unstructured text is required, such as clinic entries, radiology reports, and even patient feedback. The goal is to continue strengthening NHCS’s capabilities, ultimately enhancing our understanding and delivery of patient care through advanced, secure, and efficient data analysis.

Batch job and other Pair features are available to all Singapore Public Healthcare Institutions and Government agencies. Log in at pair.gov.sg to try it out today.