Is Keir Starmer being advised by AI? The UK government won’t tell us

Thousands of civil servants at the heart of the UK government, including those working directly to support Prime Minister Keir Starmer, are using a proprietary artificial intelligence chatbot to carry out their work, New Scientist can reveal. Officials have refused to disclose on the record exactly how the tool is being used, whether the prime minister is receiving advice that has been prepared using AI or how civil servants are mitigating the risks of inaccurate or biased AI outputs. Experts say the lack of disclosure raises concerns about government transparency and the accuracy of information being used in government.

After securing the world-first release of ChatGPT logs under freedom of information (FOI) legislation, New Scientist asked 20 government departments for records of their interactions with Redbox, a generative AI tool developed in house and trialled among UK government staff. The large language model-powered chatbot allows users to interrogate government documents and to “generate first drafts of briefings”, according to one of the people behind its development. Early trials saw one civil servant claim to have synthesised 50 documents “in a matter of seconds”, rather than a full day’s work.

All of the contacted departments either said they didn’t use Redbox or declined to provide the transcripts of interactions with the tool, claiming that New Scientist’s requests were “vexatious”, an official term used in responding to FOI requests that the Information Commissioner’s Office defines as “likely to cause a disproportionate or unjustifiable level of distress, disruption or irritation”.

However, two departments did provide some information about their use of Redbox. The Cabinet Office, which supports the prime minister, said that 3000 people in its department had taken part in a total of 30,000 chats with Redbox. It said that reviewing these chats to redact any sensitive information before releasing them under FOI would require more than a year of work. The Department for Business and Trade also declined, stating that it held “over 13,000 prompts and responses” and reviewing them for release would not be feasible.

When asked follow-up questions about the use of Redbox, both departments referred New Scientist to the Department for Science, Innovation and Technology (DSIT), which oversees the tool. DSIT declined to answer specific questions about whether the prime minister or other cabinet ministers are receiving advice that has been prepared using AI tools.

A DSIT spokesperson told New Scientist: “No one should be spending time on something AI can do better and more quickly. Built in Whitehall, Redbox is helping us harness the power of AI in a safe, secure, and practical way – making it easier for officials to summarise documents, draft agendas and more. This ultimately speeds up our work and frees up officials to focus on shaping policy and improving services – driving the change this country needs.”

But the use of generative AI tools concerns some experts. Large language models have well-documented issues around bias and accuracy that are difficult to mitigate, so we have no way of knowing if Redbox is providing good-quality information. DSIT declined to answer specific questions about how users of Redbox avoid inaccuracies or bias.

“My issue here is that government is supposed to serve the public, and part of that service is that we – as taxpayers, as voters, as the electorate – should have a certain amount of access to understanding how decisions are made and what the processes are in terms of decision-making,” says Catherine Flick at the University of Staffordshire, UK.

Because generative AI tools are black boxes, Flick is concerned that it isn’t easy to test or understand how it reaches a particular output, such as highlighting certain aspects of a document over others. The government’s unwillingness to share that information further reduces transparency, she says.

That lack of transparency extends to a third government department, the Treasury. In response to the FOI request, the Treasury told New Scientist that its staff doesn’t have access to Redbox, and that “GPT tools internally available within HM [His Majesty’s] Treasury do not retain prompt history”. Exactly which GPT tool this refers to is unclear – while ChatGPT is the most famous example, other large language models are also known as GPTs. The response suggests that the Treasury is using AI tools, but not keeping comprehensive records of their use. The Treasury didn’t respond to New Scientist’s request for clarification.

“If they’re not retaining the prompts that are being used, it’s hard to get any sort of idea of how to replicate the decision-making processes there,” says Flick.

Jon Baines at UK law firm Mishcon de Reya says choosing not to record this information is unusual. “I find it surprising that the government says it can’t retrieve prompts inputted into its internal GPT systems.” While courts have ruled that public bodies don’t have to keep public records prior to archiving, “good information governance would suggest that it can still be very important to retain records, especially where they might have been used to develop or inform policy,” he says.

However, data protection expert Tim Turner says the Treasury is within its rights not to retain AI prompts under FOI laws: “I think that unless there’s a specific legal or civil service rule about the nature of the data, they can do this.”