Skip To Content
The Mayor of London The London Assembly

A House Divided: Talk to the Alan Turing Institute Cities Summit

Last week I was privileged to give a talk to the Alan Turing Institute Cities Summit, hosted by UCL’s Bartlett School of Planning.  Knowing that I would be speaking alongside some of the best in the data science business, I opted to cover the cultural and organisational blocks on the adoption of data science in city government.  Anticipating some dramatic, colourful presentations from other speakers, I opted for a speech.  So, in the spirit of these things, do check against delivery….



Thank you for the invite to talk today to the Alan Turing Institute and Bartlett School of Planning Cities Summit.  I am from that complex, non-linear environment called London.  I am not referring to the urban spaces here, as other speakers have done; rather the public services setting and city governance arrangements.

London is in many respects too big and complex to cover in 20 minutes.  I will largely confine my comments today to London City Government and the Boroughs.  I will steer clear of TfL. With over a million contactless journeys alone each day, theirs is a big data story that an audience like this already knows well.

This aside, I will say up front though that here in London we do not have a city data poster child, an enduring, practical example like New York’s buildings inspections or Chicago’s restaurant inspection. But that’s because American Cities are exactly that and we are London.  Large City Government in the states tends to be done more simply – fewer layers tends to reduce complexity; some higher profile mayors have done much to bring data to a more central position in their administrations.

Further, our American cousins also give more political attention to data. It was interesting to see this Guardian piece so early in the presidential nominations:

“Ted Cruz erased Trump’s Iowa lead by spending millions on voter targeting”

Filings reveal Texas senator paid $3m to profiling company as hedge-fund billionaire seeded advantage in ‘military escalation’ of data-powered campaign”

So this will be a presentation free of 3d fly-throughs, dazzling maps and dramatic data visualisations.  I will talk through some of the positive examples of city government sharing and using big data (although I am sure you will rightly contend in some cases whether I am really talking about big data), and where this has worked less well.  I will go on to explore the cultural and organisational issues which I believe are the real influencing factors on the subject matter here.  Finally, I hope to offer some honest conclusions on what this means for city government’s adoption of data science.



This is the London Infrastructure Mapping Application.  Soon to be available in beta form and still to improve.  The idea behind it is to present for the first time a detailed picture of the infrastructure pipeline and how this relates to the trajectories for economic and population growth in London, and plans for developments such as new transport schemes.

Added to our own data – demographic, economic, labour market projections – the platform contains approximately 8,000 data points from a range of infrastructure providers, and developers.  These organisations often have a range of motivations around data release, different from the public sector.  But now we have a tool that:

  • In the short-term, allows for better co-ordination of infrastructure and utility-related activity.
  • In the longer term:
    • Benefits utilities and infrastructure providers because they are better equipped to develop evidence based business plans which meet London’s needs;
    • Benefits developers who are able to see with greater certainly the infrastructure pipeline, which helps unlock development sites more easily;
    • Boroughs who are able to form more cohesive local development plans; and
    • Training providers and colleges who are better able to respond to future industry requirements because they can see the labour market forecasting data as it relates to the infrastructure pipeline.

Interestingly, models like this could even become revenue generating as the various customers for this multi-faceted service see its value.

A similar example is the London Schools Atlas.  It’s easiest to describe the data as a series of great big cubes – each 5,000 schools across, 1m pupils down and 25,000 census output areas deep.  Add in 4 to 5 years as a further dimension and you see, whilst not necessarily ‘big data’, excel is rendered thoroughly redundant.

But what this example illustrates is how harmonised data – in this case, we actually short-circuited the boroughs and went straight to the DfE – can give us a strategic view of a public policy problem that is not constrained (even though the data can be) by administrative boundaries.

It therefore offers a view into the world that greater organisation around data could bring, to areas as diverse as adult social care and waste collection and management.

There are other examples.  Crime hot-spotting using multi-agency data.  Analysing London Ambulance Service data and sharing of non-confidential patient data from A&E so the police can better anticipate crime like knife crimes.  We will soon release a London crime confidence tool, mapping over 8,000 variables at granular, local level.

And now I want to talk about working with the London Boroughs.

Witan is an interactive city modelling platform we have just released in Alpha.  This is the first attempt to draw together all of the social and economic forecasts we use to plan for city growth into one place, and to give us humans the best chance of using our judgement to understand the interdependencies between important inputs like population growth and housing and labour markets.

Our first output is on-demand projections of housing demand, using the GLA’s own models. This means that officers in the 32 London boroughs can enter housing data projections and see how this affects the spread of the population across wards all the way up to 2041.

Previously, this process used to take weeks of specialist GLA staff time, and would typically happen once a year. Now we’ve automated the process, allowing boroughs to run their own projections at any time using GLA’s models, as often as they need to, creating new projections in under five minutes.

The best aspect of this project though is how we are working with the considerable big data brains of Mastodon C. We get to learn from their expertise; they get to develop a commercially viable modelling platform for other cities.

We have just applied to the Cabinet Office’s Data Science Accelerator Programme with a select few London Boroughs.  The point is, this is an ad-hoc exercise –   It involves 4 Boroughs, and it has a (rather unlikely) data request to DWP underpinning its success.

This is the point at which I start to move into organisational culture.  I established the Borough Data Partnership a couple of years ago.  We have shown data owners in Boroughs the art of the possible.  We have given them imported ideas from the New York University. Algorithm-ready analysis for route optimisation and behaviour change in waste management. The ‘math’ is done. We just need some joined-up data.

Now where would we be without a War and Peace reference?  The gory battle scenes lead me to if it is the infantrymen who win wars, or is it the generals with their epaulettes and telescopes up on the hill?  Well the truth is, we need both.  And after 2 years of trying, I am happy to admit defeat and say it is time to take re-engage the generals.



So what is the cultural and organisational reality of public service in London?

We pursue public service reform, but this seems in the main to be a discussion with central government about devolution of powers.  About moving power around in the system.  We seem to find less time for public service modernisation; and innovation is clearly still perceived as risky and still subject to failure which is of course politically dangerous.  Data could be far more relevant to both.  My question is this – for all the decisions made in London’s public services, how many are really made on the basis of first, the best data, and second, the best data with the best analytics applied to them?

Public servants are conditioned to work in silos.  We tend not to move easily beyond administrative boundaries recognised largely only by ourselves.  Problem solving so often focuses on individual services alone and is tactical.  Innovation with data is not and should not be the job of the transport planner, nor is it in the job description of the environmental services manager.  It isn’t necessarily the CTO’s either.  But some form of corporate and political ownership of data-led innovation is needed?

Of course, innovation with data does happen, but it not often enough that full systems thinking occurs; that someone does that apparently simple thing of connecting up data with analytics with service knowledge, for insight and service or societal impact.  Is it acceptable in 2016 we are not at least looking at using waste management data to drive contracting arrangements based on predictive analytics?  Is it not less acceptable still that local councils – and this could be a myth, but no-one has forcefully denied it – are fined by private sector contractors because their communities have not generated enough waste to dispose of?

And then there is the idea of physical space, and community, and how data seeps through every aspect of our lives, and is at the core of city economy and services. This is what Gavin Starks, Chief Executive of the ODI has written in a superb blog on ‘the Porous City‘.  I quote:

“We have failed to create smart cities. We have failed to create truly scalable and sustainable mechanisms to enable our cities and countries to benefit from the internet age. We are not addressing the challenges we face, from housing to healthcare, from taxation to climate change, in line with the pace of change.”

I urge you to read it, but again, my point is while the activity of using data in innovation and discovery in urban settings is not the sole preserve of public institutions and services, it is not an abstract as some consider it to be and is worthy of deeper consideration.



London is a world leader in the open data movement.  I argue, however, that the moment for more open data has passed.  My metaphor is the washing line.  The question it provokes, how much more is to be gained from hoisting spreadsheets up onto it so that they can flap around in the breeze (until the market chooses what is useful and what is not)?  There is too much built in redundancy in this exercise.  And to pursue an open data strategy alone puts us firmly in the domain of mild improvement by gentle increment.

It is city government’s role to be much more deterministic about the uses to which it wants data ‘the asset’ to be put – energy management; environmental improvement; labour market access; numerous other examples.  In doing this, we promote the use of ‘city data’ – data not just from government, but from a variety of sources like utilities, households, telecoms companies.

This is data that is shared – not necessarily made open, but certainly put to use, by the host of data aggregators, integrators and enrichers operating in this city.



And so to the new City Data model, to be fleshed out across the six themes of our forthcoming City Data Strategy.  I offer just four headlines for today:

  • First, a definition of city data that is widely accepted and understood;
  • Second, a marketplace in which an assured supply of city data can be ‘traded’;
  • Third, an attempt to place a monetised value on city data; and
  • Fourth, reliable city data infrastructure (created through the aggregation and standardisation of traditional IT and moves towards cloud storage and computing.

Together, these can help overcome organisational inertia and disconnections.  They are the collaborative building blocks needed to create a meaningful city data economy to support service innovation and world class analytics.  Without them, leadership in this area is difficult to the point of being practically pointless.

So what price failure?

  • 21st century government services in our world city will fail to materialise. Sure, talked about in rooms like these, but out there, cross-sector integration of services like and social care will fail.
  • The Internet of Things will pass public services by.
  • The analytical might making these things possible and value-giving for citizens and businesses, and the building of city resilience and capacity, will be hobbled.

What a waste this would be.



Despite this rather depressing note, I am fully convinced of the potential of data science.  And committed to it.  I hope I have made clear the key messages in this presentation in this city.  This city’s ability to capitalise on city data analytics is not a question of technology – rather, we need to get culture and co-ordination between public services right.

This is why I want to propose to the next Mayor a Data Innovation Lab for London – an environment in which politically important issues are turned over to meaningful data science experiments, potentially playing in the science of behavioural insights and service design expertise.

We all need to travel on the issue of data science.  The data scientists need to get better at the politics and move closer to the issues of real political relevance in this city.  The politicians need to get better at understanding what data science can do.

So I implore you – bring your algorithms, your heuristics, your machine learning and general computational might.  I promise only to bring you problems and challenges we can meet and at least start to solve together.