Blog 4 for the 15-year anniversary of the London Datastore: Celebrating London’s Data Story

London is a wonderful, but complex ecosystem. Almost no city
challenge can be solved by any one organisation on their own. That’s why the
Greater London Authority emphasises partnership working and collaboration in
all we do. Where the solution to a problem requires data sharing, the London Datastore
has been a really important tool to make that happen. We have seen how
important the data on the London Datastore is for researchers, Londoners, and
policy makers as they look to make London better in all sorts of ways.

Nevertheless, data sharing is hard. Even though some really
big problems could be solved by better data sharing, we also need to be careful
because it is important that we protect sensitive data and privacy. The Data
for London programme was set up to make it simpler for people to share and use
data held across London to improve the city and benefit Londoners. However, we
want to do so well: legally, ethically, and in a trustworthy way.

There are a lot of initiatives, in London and nationally, that have shown great ways to do data sharing well. Some of them, such as the Infrastructure Mapping Application, High Streets Data Partnership, and Rough Sleeping project, will contribute blogs of their own during this anniversary period. Others, such as the Information Governance Group for London, or the Damp and Mould IoT Project, are supported by the London Office for Technology and Innovation. Recently, the UK government has also made relevant announcements, such as their investments in the National Data Library and the data (use and access) bill, which directly aim to address public sector data sharing.

However, challenges remain.

This information was gathered in response to a local government collaboration sprint by the new Government Digital Service. We hope it provides some examples of great practice in London, as well as ways we can improve further.

Technical and infrastructure

When sharing data, it is important that you have the
technical infrastructure to actually move data from one place to another. In
London, we are lucky to have a few ways of working that work really well for
us. The London Datastore and Data for London programme provide shared plumbing
for London, which provides a set of tools that all partners can make use of.

For some projects with specific needs, projects are often
supported by third-party suppliers who enable more custom data sharing
infrastructure. London benefits from the skills and scale of its city ecosystem
in terms of technical skills available. We are able to draw on cutting-edge
commercial partners to make use of the latest technologies to share, exchange,
and secure data we are sharing. This vibrant marketplace drives innovation and
ensures value for public money when we procure technical infrastructure.

However, challenges remain. It is particularly difficult for
local authorities to build the required level of digital and data maturity to
make data sharing and cleaning easier. Building a modern data management
solution, including bridging the gap between lots of different systems that are
used for different services, is difficult. It is also expensive and local
authorities struggle to find the long-term investment to enable that jump.
Therefore, instead we are often working with legacy systems, with which it is
harder to effectively exchange high-quality data.

Similarly, the diversity and scale of the ecosystem also makes it difficult to bring data together in a standardised way. As different teams produce similar data in different ways, it can be a technical challenge to make sure the data lines up neatly in the middle. This is particularly hard where local authorities lack dedicated GIS or data teams who can help clarify the minutiae of the data being shared.

Legal

London has invested in making sure our data sharing projects are done in a fully compliant way, while making sure that a misunderstanding of the law does not get in the way of important projects taking place. The Information Governance Group for London brings together IG professionals from across London’s public sector, allowing every organisation to scrutinise plans and ideas. We make sure to focus really clearly on important benefits for Londoners, and only share data that is relevant to that purpose. By being precise about the data we’re sharing and why, we can make sure we are fully complying with our duties under the law.

However, there are some legal restrictions that do not translate well to the local government context. For example, some data sharing agreements are so restrictive that we cannot share data with partners outside the public sector. As most problems in local government are solved by working closely with delivery partners such as commercial companies or charities, this can make it difficult to have on-the-ground impact. Even where agreements were set up with a few select partners, it can be difficult to expand that agreement to further organisations as a service or project grows. This happens when the legal basis for sharing data is only available to the original partners, and not to new ones. This can mean we miss out on important partnerships to solve London’s biggest challenges.

Capability and capacity

It can be difficult to find the right people with the right
skills and a bit of time to work with on solving big problems, particularly in
a sector as resource-constrained as local government. Therefore, the GLA and
London Councils invested in the London Office for Technology and Innovation to
serve as London local government’s collaborative innovation team. LOTI helps
London borough councils and the GLA use innovation, data, and technology to be
high performing organisations, improve services, and tackle London’s biggest
challenges. Having this team available has been revolutionary in helping a
community align themselves around shared problems. That community has then been
able to find those with the capability and capacity to lead on projects that
the wider community benefits from.

However, it is still challenging to find the right individuals, teams, or units with the right expertise and resources to contribute to such projects. London’s public sector is made up of 32 boroughs and the City of London, as well as the GLA and related public sector organisations such as Transport for London, the Metropolitan Police, development corporations, and more. Making the best use of such a diverse network is tricky.

Resourcing

London benefits from the skills and scale of its city ecosystem, not only in terms of infrastructure but also in terms of analytical skills. There is incredible data expertise in London’s public sector, such as the GLA city intelligence unit and GLA economics and domain experts on specific topics in every borough. Moreover, in some areas like planning and infrastructure there are shared resources that can be used by everyone, including data schemas and standards which make data sharing easier going forward. Finally, innovation capacity provided through LOTI has really improved the resourcing available for novel and experimental projects that look for potentially transformative ways of using data to solve problems.

That said, in a resource-constrained sector, it is still difficult to attract and retain staff with the right skillsets. This means that the data that is being shared can be delayed or have quality issues, which makes it harder to use for maximum effect. It also means that project evaluation and evidence may not be systematically collected as often as we would like. Even where central functions, such as the GLA or LOTI teams, are able to support by defining with data standards or schemas, partners often still need help to structure the data to ensure maximum impact. This can hamper data sharing between local authorities, and it can also hamper local authorities’ ability to make use of data being shared with them.

Conclusion

London has achieved an incredible amount in making sure data
sharing projects make real positive change for Londoners. Local authorities are
laser focused on being clear on the end goal of each project, and making sure
data is shared legally, ethically, and in a trustworthy way to achieve that
outcome.

Unfortunately, some great projects still fail because we
cannot share the data needed to solve the problem. Whether we are hampered by
technical infrastructure, legal barriers, capability, capacity, or resourcing,
that is still a wasted opportunity to improve the city and benefit Londoners.

The London Datastore has been a critical tool in unblocking some of these projects and have positive impact. In the past 15 years, we have learned a lot from what has gone well and what has been challenging. The Data for London programme will now rebuild the platform to make it simpler for people to share and use data held across London, and address some of these barriers. We look forward to working with any and all partners who share that vision.