Building an AI-enabled insights platform for the UK Foreign, Commonwealth and Development Office

The Challenge: Providing up-to-date insights for FCDO on emergent global trends

The Foreign, Commonwealth and Development Office (FCDO) takes an evidence-informed approach to policy and strategic decision making to ensure delivery relating to UK priorities at home and abroad is supported by up-to-date and accurate information. Edge Health has recently supported the FCDO by creating a bespoke AI-enabled insights platform for media analysis. This collects, processes and displays data, providing an accurate evidence base for decision making in a cross-cutting area and saving civil servant’s valuable time. 

Our Approach: End-to-End Product Development

Our approach leveraged public data sources and applied advanced AI algorithms to generate key insights.

Methodology for Insight Extraction and Validation

1. API calling: The tool ingests a large number of publicly available news articles into the platform through an API.

2. AI powered filtering: The sheer volume of articles ingested makes any attempts to manually filter for relevant data points infeasible. This necessitates the development and application of advanced AI and Machine Learning algorithms. We created a sequential filter of:

• A Boolean keyword search algorithm.
• A Support vector machine (SVM) to classify the vectorised free text fields based on semantic meaning.
• A Transformer based text classifier.

These algorithms ensure that the ‘firehose’ of data from our API is effectively filtered to only include relevant data.

3. Article Labelling: Relevant articles need to be tagged and summarised to ensure pertinent information is retained. We harnessed state-of-the-art large language models (LLMs)to label and summarise text from our input source.

4. Manual validation web app: Human-in-the-loop validation is essential to maintain high quality datasets. We built a bespoke React-based web app that allows the validator to accept, reject or update data based on the initial inputs.

5. Insights dashboard: a bespoke Power BI dashboard was developed to highlight key insights, enabling researchers and analysts to consider geographical and seasonal trends in the subject matter.

This end-to-end product was subsequently handed over to the FCDO analytical team, along with robust and comprehensive technical documentation, to ensure continued in-house monitoring and upkeep.

The Impact: Data-Driven Insights to inform policy making and strategic planning

Historically, building an evidence base for policy decisions has relied heavily on manual desk-based research, which is a time-consuming task, limited by staff capacity. This reduces the number of policy areas that can be served at any one time.

The creation of AI-driven tools, such as this one, presents an opportunity to increase efficiency, saving researchers’ time and allowing them to focus on analysis rather than data collection. Usage data from our tool suggests that it increases the speed that data is collected significantly, as highlighted by our client:

“Before, I had to chase down data—spending days searching and compiling what I needed for my analysis. Now, the data comes to me automatically, accessible in just a few clicks.”

Overall, this enables more data to be collected, allowing policymakers to make more informed decisions and understand global trends more efficiently.