Better information means better decisions: This is the principle that makes the rise of machine learning, new technology and algorithmic decision making so exciting to policymakers and private businesses.
One part of that revolution that is particularly exciting to UK ministers and officials is Palantir’s foundry software. From the perspective of the UK government the joy of Foundry is the ability to understand, use, monitor and run experiments and projections with different operating systems and methods of data collection. At the start of the COVID-19 pandemic, a system originally developed to allow information to flow seamlessly from agents in the field proved perfect to allow the British government to monitor That’s what was going on in the complex tangle of overlapping health trusts of the NHS and GP practices.
Civil servants have described the use of the foundry in monitoring the progress of Covid-19 as one of the few things in the UK’s pandemic response that worked as advertised. Many in government now hope that the software may hold the secret to something they have previously failed to do: harness the NHS’s data sources to better understand how to administer treatment and improve patient care.
But better information has a price to pay. This makes it difficult for governments or individuals to engage in polite speculations about how services actually operate. The UK government’s prison data dashboard, for example, is a win for transparency, but it shows that the UK prison system is generally very poor at actually rehabilitating prisoners.
The NHS is more popular than most peer systems, despite the fact that it provides roughly the same results for the same amount of money. One reason for this is that the various ugly things health care systems have to do – deciding who gets treated first and what kind of treatment to offer – are kept hidden by the healthcare model. A better system-wide view of how treatment outcomes and waits differ for people with learning disabilities or different ethnic minorities is vital to improving how the system operates. But it serves states or governments better if those realities are kept out of sight.
Which is why the British state may be increasing the number of contracts awarded to Palantir, but it is slowing the public’s ability to make the software work for itself. Information collected by the foundry helped create the government’s COVID-19 dashboard, which made publicly available online what was going on in British hospitals. Since then the UK government has tapped Palantir to manage the underlying system for its vaccine rollout and critical capacity. But the original dashboard risks becoming a historical artifact – even though it could become an all-purpose tool for monitoring the performance of the UK’s healthcare infrastructure as a whole.
Foundry and the use of the COVID-19 dashboard are good examples of how desperation drives governments to innovate. In normal times, the British government would have been more reluctant to overcome public resistance to allow an American company, and intelligence agencies with deep ties to the NHS, to enter. Nor, in normal times, would it have ever done something so transparent as using that company’s tools to create a publicly available dashboard that measures a country’s effectiveness in fighting a new disease.
But now that the acute phase of the pandemic is over, the government has decided that it relishes the benefits of better data-sharing and modelling, but dislikes the opportunity for better testing that would make that information available to others. comes with
Politicians are right to assume that better and more transparent data will mean tackling awkward questions. If it is not clear that the UK is managing to effectively rehabilitate most people, say, it raises uncomfortable questions about, in fact, why we expect shorter prison sentences to be meted out. And a more transparent NHS might find it difficult to engage in the secret rationing that the service has practiced in the past. But the reality is that you don’t need the government to be transparent to know that the UK health service is in bad shape or that most of the public sector is in bad shape.
Around the world, the data revolution will make it harder for governments to hide the truth about how they provide services. But they would do better to become enthusiastic advocates for the benefits of transparency and share the fruits of better information with voters, rather than simply hoarding them for the political class.
