Trowers & Hamlins

Sign up

Home » Resources » Blog » Big Data - the opportunities and challenges

Big Data - the opportunities and challenges
Trowers Public Insight

Big Data - the opportunities and challenges

Much has been written about the value that can be derived from harnessing Big Data – data sets that are so large and complex that they are too difficult to manage through traditional management tools.

Public sector bodies receive, collate and store masses of data sets about their citizens and service users, all of which, if properly analysed and profiled, could provide invaluable information to assist in a range of matters including intervention strategies, service improvement and funding allocation.

Given the proliferation of unstructured data, new technologies have to be employed to ensure that valuable information is not lost or under-utilised.  In a time of austerity, Big Data could assist in reducing costs for the public sector – but is not without its challenges.  Putting aside difficulties in securing the right technology (which is an industry-wide issue) , there are unique challenges in the public sector's use of Big Data.

Given the failure of high profile public sector ICT procurements, Big Data projects are having difficulty in gaining traction.  Whilst the need for a tool to store patient information or child protection interventions is obvious, is the investment in a Big Data technology justifiable?  This in part is answered by how compelling the potential outcomes can be – something that is yet to be fully explored by the public sector.

Also, given the recent controversy of information collection by GCHQ, there is potential for suspicion that Big Data projects are nothing more than "Big Brother" initiatives that have a sinister motive.

These perception challenges are further augmented by legal issues that arise from the ownership and usage of data.  Whilst data protection issues are complex, they are surmountable, assisted in part by the recent launch of the encryption service by the Cabinet Office which removes certain barriers when sharing data in the public sector, the question therefore is can public sector co-operation really happen on the scale required to make analysis of Big Data a realistic goal? 

As the public sector grapples with the potential of Big Data and the value it can deliver, it remains to be seen who will be brave enough to step into the fray and think beyond the immediate pressures of delivery to the long-term benefits that can be secured form effective Big Data analysis.