Public sector data: unlocking the not so hidden gem

Since March 2020, the coronavirus pandemic has created unprecedented challenges for local government.

There has been few aspects of local government that have not required at least temporary reform; from areas ranging from social care and children’s services, to waste collection and housing.

Local councils have found that better use of data has been key in helping them to rise to the challenge and respond effectively.  In a sector which is not always renowned for its pioneering use of data, local authorities have taken a vast leap forward in these past 11 months.  This has included acquiring new data, which they did not have access to previously, and also utilising existing data more efficiently and deploying it in novel ways. Examples of such data include:

• Identifying those most clinically and economically vulnerable to the effects of COVID-19;
• Predicting demand and pressures on local services;
• Informing d irect public health responses to COVID-19 outbreaks, including local-level; and

• collaboration on NHS Test and Trace.

Notwithstanding the success with which local authorities have acquired and utilised data during the pandemic, it raises important questions of whether this progress can be sustained in the long run, and whether it would even be beneficial and indeed ethical to continue to do so.

To shed light on these issues, the Centre for Data Ethics and Innovation (CDEI) hosted a forum in 2020 to speak to local authorities to see what lessons have been learnt, and what their concerns are for the future. The CDEI has just published a report summarising the information the forum had gathered, particularly regarding:

• How data use had changed in local authorities since the outset of the pandemic, including lessons learned and ambitions for the future;
• How recent achievements and new practices might be maintained and improved upon beyond the duration of the pandemic; and 

• How local authorities had approached questions of data governance and ethics during this period, and what extra support they might need in the future to use data to the highest ethical standards.

How has the pandemic changed data use?

The main ways data has been used by local councils is:

• by successfully been combining datasets from different local public service partners in order to better identify at-risk individuals, such as families experiencing multiple problems. Thereby having the information required to provide more targeted interventions. Hackney Council has used combining datasets to identify residents who are most clinically and economically vulnerable to Covid-19. This includes older residents and people with disabilities who live alone, as well as those most susceptible to the economic consequences of lockdown. Hackney was able to use unique property reference numbers (UPRNs) to link datasets that were previously siloed, for example data related to council tax and tenancy deposits;
• by collecting new population-level data, in the form of granular postcode-level data, on infection rates to inform outbreak containment plans and to help them target messages to residents in at risk neighbourhoods. Councils have further used existing population-level data to determine where demand for services was likely to be greatest during the pandemic;
• by using predictive analytics. In using algorithms alongside data to predict residents' demand for services in the near future, such as police; fire and ambulance teams services; and 

• by promoting transparency by allowing residents to understand the state of their local area and how their local authorities are performing; such as in terms of information about business rate charges, procurement decisions and the availability of housing stock. This promotes accountability and allows local authority decisions to be adequately scrutinised.

Key challenges and issues:

Along with the success, local authorities have faced issues along the way and there are a numbers of lessons learnt. Further, the forum highlighted those key ongoing challenges that local authorities must be vigilant of, as they continue to use data analysis to provide a better service.  

1. Data exchanges 

• The forum raised concerns that local authorities were not always able to share or access the data they wanted to. In addition, there could be more clarity in the definitions and terminology used to describe the data shared, as there are many inconsistencies with how data is managed from council to council, which prevents the data from being shared effectively. 
• Local authorities reported they had more success in deploying existing internal datasets in a new way than they had in acquiring new datasets from other bodies. The exception to this is the sharing of specific types of health data, which have been facilitated or enforced by central government, and which have previously been difficult for local authorities to access. 
• Not every local authority had success to the aforementioned mature data strategies and therefore have launched fewer data-driven initiatives in response to the crisis.
• There is much more than can be done to create more systematic and clarified approach in the future. Effort should also be made to upskill staff at all levels and to improve local authority access to trusted data resources 

2. Sustainability

• One of the main concerns, with the spike of data usage, is that the achievements seen in 2020 were only temporary, and after the pandemic is over the use of data practices would revert back to the pre-pandemic status quo. In support of this pessimism, is the fact that the emergency access granted to use datasets will likely be repealed once it is safe to do so. 
• There also has been an insufficient demand from frontline teams, such as housing and social care teams, to use the results of the data analysis. The report found that frontline practitioners can lack faith in the veracity of data-driven analysis, preferring instead to rely on their own judgement. The forum discovered that their data teams lack the feedback loops to know whether their findings are being well used.
• Local authorities may be unwilling to continue new or innovative practices beyond the pandemic, even where there is clear value, if they fear they will be the outlier in such practices. It is also likely that enthusiasm for data-led interventions among senior decision-makers could wane, especially in areas that cause controversy. 

• Further, these issues are combined with the historic barriers to innovation, such as:

• budgetary constraints - It takes time and resource to martial a Council's data, develop and test use cases and then deploy them in the front line. Following ten years of austerity and a global pandemic, the headspace within Councils for such projects is limited;
• poor data quality skills gaps; and 

• lack of legal clarity as to what is legally permissible in the collection and use of (personal) data. Compliance with Data Protection Laws will need to be maintained, and councils must ensure they have up to date Privacy Policies to allow them to have greater clarity of their lawful basis for using data, and having clearly explained this use to data subjects.

3. Ethical concerns

• A great challenge that local authorities face is not just to maximise the use of data, but to do so whilst upholding the highest ethical standards. There is a risk that engaging in new practices could lead to new risks; particularly with regard to the use of health data, where preserving the privacy of data subjects is critical.
• When society can return to "business as usual" those practices of retaining access to certain data which are justified during a pandemic may no longer be justifiable outside of an emergency context. Outside of a pandemic, councils will need to construct clear narratives around the data they are using and the benefits it will bring.

• Obtaining member buy-in and advocacy is therefore vital to continue to use data. The CDEI outlined a framework that consists of five key principles of trustworthy data use, designed to help data controllers think through the ethical implications of new data-driven initiatives, being:

• Value - Is there a clear benefit to individuals and society from the data being used and shared?
• Security - Is data being used securely and is the privacy of data subjects being protected?
• Accountability - Is it clear who is responsible for how the data is being used and shared?
• Transparency - Can the public scrutinise how their data is being used and do they know why?
• Control - Do people have a say in how data about them is used and shared?
• The forum also discussed that one of the barriers they faced in maintaining ethical standards was in implementing data ethics guidance that everyone could understand. Local authorities noted that what is helpful for technical staff may not be for their frontline colleagues, for example guidance that goes too far into specifics may appeal greatly to one team, but be ignored by most others in the organisation.

• Another issue is again the cost of engaging with residents. Local authorities in the forum acknowledged the importance of understanding residents’ concerns as well as increasing public awareness of how local authorities use data, but the cost of doing so means that this option is a luxury that authorities can seldom afford, especially when budgets are already limited. Nonetheless, the value of engaging with the public should not be underestimated, as it also assists councils to build trust and faith in the legitimacy of data-driven projects.

What support to local authorities need?

The forum found that despite their concerns, or limitations of the data being acquired and shared, local authorities had overall changed for the better during the pandemic. These local authorities also talked of attitudinal changes within local government, with the severity of the public health emergency acting as the catalyst. The media’s widespread coverage of data-driven interventions at a national level (e.g. around the contact tracing app), may have helped to alert councillors, chief executives and other decision makers to the power and potential of data.

Having considered the lessons learnt, CDEI and the forum found that, notwithstanding their financial limitations, local authorities could carry of the following measures, within their control, in order to maintain the momentum of the past year:

• Finding new ways to communicate the value of data and in conveying the results of their data analysis;
• Testing data-driven interventions in a safe environment before committing to the rollout of new data-driven interventions, for example  by  using ‘sandbox’ techniques, which invite regulators and others to observe projects as they are being designed;
• Learning from the experiences of other local authorities across the country;
• Utilising external expertise, who can advise local authorities on how to deploy data responsibly, for example data specialists at universities can give data teams confidence that they are using cutting edge techniques, and that they are giving due consideration to ethical risks.

Looking forward

The challenges found by CDEI's forum are systemic in nature, and will require cultural shifts, legal changes and funding decisions from senior leaders that will need to improve over a period of years (not mere months). 

Organisations such as the Local Government Association; the Ministry of Housing, Communities and Local Government; the University of Essex and the Government Digital Service have already offered their support in promoting the use of data and technology to improve the way local authorities operate. 

There will of course be higher barriers that will remain, which are outside of local authorities' control and will require the efforts and influence of central government to resolve. However, the technological advancements of the past year have shown that progress is possible within local authorities, and that the rewards can often justify the effort and expense.