Streamlining Legislative Research for Compliance at the People's Association

In the fast-paced world of public service, quick access to accurate information is crucial for efficient decision-making and policy implementation. Pair Search, a feature of the Pair platform developed for Singapore's public officers, has emerged as a transformative tool in this regard. By leveraging advanced AI technology, Pair Search offers rapid, precise, and contextually relevant information retrieval across vast repositories of government documents, legislation, and policy papers.

Figure 1: Comprehensive results for search query

For Choon Hooi, an officer in PA's Finance Division, Pair Search has become an indispensable tool for navigating complex legislative documents, particularly the Charity Act and Regulations. The platform's ability to quickly sift through vast amounts of information has revolutionised how PA supports its operational units in interpreting the legislation for compliance as registered charities.

Previously, Choon Hooi spent considerable time manually researching and answering queries about regulatory and compliance requirements. With Pair Search, he can now provide accurate information efficiently, backed by relevant sources for verification. This not only saves time but also enables operational staff to conduct initial research independently, fostering a culture of self-help and reducing dependency on centralised support.

How does Pair Search work?

Figure 2: Pair Search

Pair Search leverages on Vespa.ai, a highly versatile and scalable open-source big data serving engine. This advanced technology enables Pair Search to offer state-of-the-art search capabilities, incorporating extensive text-search functionalities and continuously integrating cutting-edge techniques and models.

The search process in Pair Search is meticulously divided into three phases: document processing, retrieval, and re-ranking. The retrieval mechanism combines keyword-based and semantic search strategies, utilizing Vespa's weakAnd operator alongside nativeRank and BM25 text matching algorithms. On the semantic front, Pair Search incorporates e5 embeddings, offering a balance of speed, cost-effectiveness, and enhanced performance.

To maintain speed despite the use of complex re-ranking algorithms, Pair Search employs a three-phase approach to iteratively refine search results. This includes cost-effective initial filtering, resource-intensive re-ranking using the ColbertV2 model, and a final phase that creates a hybrid score integrating semantic, keyword-based, and ColbertV2 scores.

This sophisticated backend ensures that Pair Search delivers fast, accurate, and relevant results, setting a new standard for information retrieval in the public sector.

Direct Access to Information and Future Potential

The introduction of Pair Search has led to tangible improvements in both accuracy and time savings in PA. By providing more relevant results and reducing the time spent combing through information, Pair Search has become an essential tool for public officers dealing with large volumes of legislative and policy documents.

The impact of Pair Search is particularly notable when dealing with lengthy documents with multiple versions. Choon Hooi explains, ”Before Pair Search, we were heavily reliant on our team to decipher complex information and advise users. Now, users can access and understand the information directly.”

As government agencies such as PA continue to explore the potential of Pair's suite of tools, there's growing interest in expanding their capabilities. The success of Pair Search demonstrates the team's commitment to developing tailored AI solutions for the public sector. The Pair team is constantly innovating, working on new features and improvements that go beyond search functionality to address various challenges faced by public officers.