Proph Abby: Singapore General Hospital Leverages Pair to Revolutionise Surgical Antibiotic Prophylaxis

Singapore General Hospital (SGH) has harnessed the power of Pair to create Proph Abby – an AI-assistant that streamlines surgical antibiotic prophylaxis guidelines for doctors. SGH utilised Pair's unique features to enhance patient care and operational efficiency in a critical healthcare setting.

Addressing a critical challenge in surgical care

SGH performs over 100,000 surgeries annually. Each requires prophylactic antibiotics, which are used to prevent post-operation infections in surgical sites. Often, the challenge is prescribing the right type and dose of antibiotics, particularly in complex cases where patients have multiple drug allergies and poor comorbidities. Recognising that AI could be tapped to address this, Pair was utilised to develop a solution – Proph Abby – that can scour existing surgical antibiotic prophylaxis guidelines and provide a recommendation within five minutes.

Figure 1: Prescription of surgical antibiotic prophylactics can be challenging, especially in complex cases. Dr Ong Si Ling, Medical Officer, Department of Anaesthesiology, is a part of the team that saw a gap that could be addressed with LLMs like Pair.

Dr. Aaron Lee, Senior Consultant, Department of Anaesthesiology, had experimented with other LLM platforms to develop tools that can help clinicians, and was one of the pilot users of the Pair Assistant feature. Among these projects, Dr. Lee and his team thought LLMs could help clinicians with surgical antibiotic prophylaxis prescriptions. They needed a solution that could meet the stringent security requirements of the healthcare sector and leveraged Pair, which was designed explicitly for public service use cases.

Security and data protection are paramount in the public service, and doubly so for healthcare. Pair's clearance for classified data and its ability to meet government- specific data handling requirements made it the perfect fit for the needs of SGH.

Leveraging Pair Assistants for specialised healthcare tasks

Pair's Assistant feature allows for easy creation, testing, and sharing of LLM-based assistants specialised to perform high-volume, repetitive, and niche tasks. This capability aligned perfectly with the team’s goal of streamlining the complex process of surgical antibiotic prophylaxis decision-making.

Dr. Jonathan Cheng, Consultant, Department of Surgical Intensive Care, and a key member of the development team, notes, "Pair Assistants enabled us to create a tool tailored to our specific guidelines and workflows. The ability to easily test and iterate on our assistant was crucial in developing a solution we could trust in a clinical setting."

Impressive results and time savings

The impact of Proph Abby has been substantial. When tested against 100 cases using a dataset of past patients, the Assistant achieved a 95% recommendation compliance rate with the established guidelines. This was a significant improvement over the previous 80% compliance rate, where doctors may have been overly cautious and prescribed more antibiotics incongruent with the guidelines. Moreover, it received a 74% agreement score (Cohen's Kappa) when rated against human evaluators. This indicates a high level of consistency between Proph Abby's outputs and the human evaluators' assessments.

Dr. Cheng explains, "Proph Abby allows doctors to access and apply complex guidelines quickly and accurately, saving valuable time – up to 90%, especially for complex cases where more references and calculations are required. It's important to emphasise that Proph Abby guides the doctor rather than giving instructions, ensuring that medical professionals remain at the centre of decision-making."

With Proph Abby, clinicians can quickly and accurately access complex guidelines on surgical antibiotic prophylaxis.

Rigorous testing made simple

The SGH team leveraged Pair's built-in features, such as its Assistant Preview function and direct user feedback and usage mechanisms, for testing and evaluation to ensure Proph Abby's reliability. They conducted extensive tests, including comparing outputs against expert opinions for complex cases and evaluating common scenarios against senior residents' decisions. This thorough validation process was crucial in ensuring patient safety.

Seamless sharing and feedback loop

Once developed and tested, Pair's sharing features allowed for easy distribution of Proph Abby across the department. The platform's ability to facilitate direct feedback from users to assistant creators ensures ongoing improvement and refinement.

"The ease of sharing Proph Abby with our colleagues and gathering their feedback has been invaluable," notes Dr. Cheng. "It allows us to continuously improve the assistant based on real-world use."

Future-proofing healthcare innovation

Encouraged by the success of Proph Abby, the SGH team is already exploring other applications of Pair in healthcare. They have created another assistant that helps interpret medical results, discharge summaries, and scan reports in layman's terms, demonstrating the versatility of the Pair platform in addressing various healthcare challenges.

As SGH continues to innovate in healthcare delivery, Pair stands out as a secure, efficient, and user-friendly platform for creating specialised assistants. The success of Proph Abby serves as a compelling example of how public sector organisations can leverage AI to enhance their services while maintaining the highest standards of security and efficiency. By improving antibiotic prescription and saving valuable time for healthcare professionals, Proph Abby is helping SGH provide better care for the thousands of patients who undergo surgery each year.