The London Datastore is piloting a new job postings dataset. This blog introduces the dataset and answers potential questions relating to its source, scope and quality. The data provides detailed information on the skills employers are looking for, the roles they’re seeking to fill, and the employment package they have on offer. We hope it will be useful for a variety of users involved in skills and careers in London. Please contact us with any feedback on the dataset or if you have questions not addressed below.

1. What is the online job postings data pilot for London?
London job postings data is a pilot online data service that provides access to new data from job postings to ‘skills market participants’, such as the LEAP, commissioners, providers, employers and careers services. We want this data to be used, in conjunction with traditional labour market indicators, to help London develop the skills that its economy needs, and thereby empower Londoners to make good careers decisions.

The Greater London Authority (GLA) is providing access to near, real-time labour market data on a pilot basis from the beginning of January to the end of June 2017. Access is free of charge and open to all.

2. Where do the data come from?
The data are being sourced from Burning Glass Technologies, a leading supplier of real-time labour market intelligence. Burning Glass Technologies (BGT) is a US-based company with a strong presence in the UK.

Burning Glass’ data are compiled from thousands of sources of job postings in the UK, including online job boards and company websites. To collect the data, BGT uses thousands of “robots” or “spiders” which collect job vacancy postings on a rolling basis. The information thus retrieved then goes through a sophisticated parsing process that standardises and classifies vacancy postings according to a number of data elements including location, industry and occupation classification, minimum qualifications, job hours, contract type, etc.

3. What data will you be making available?
We are providing access to quarterly counts of London online job postings, classified by occupation, job type (permanent, temporary, apprenticeship, internship), full-time/part-time status, advertised average salary, the types of skills sought, and location (local authority), where this information is available.

The counts relate to job postings made in each quarter from Q1 2012 to the latest period, and the data will be updated on a rolling basis each quarter to ensure timeliness.

By selecting the categories and time period of interest in the online data tool, users can aggregate, filter and export the online job postings data in line with their own needs.

4. Why are you providing access to this dataset?
Real-time job postings have enormous potential in the field of labour market intelligence, as an ever-increasing share of job openings are posted on the internet. We want to test the benefits of widening access to this kind of detailed, real-time information to help develop the skills that our economy needs.

We believe that we can achieve a significant public good by making the data widely and freely available, particularly since there are currently no official statistics on job vacancies available for London, or information on the skills that employers are seeking from new recruits.

We will use the learning from our pilot, and are seeking collaboration from partners, to determine whether to roll-out the job postings data via London Datastore on an ongoing basis.

5. Why is this dataset useful?
For commissioners, providers, careers services, and labour market analysts the online data tool can provide a rich source of detailed information on the skills that employers are looking for, the roles they’re seeking to fill, and the employment package they have on offer. This can complement more traditional labour market indicators available on the London Datastore, to allow for a near real-time, more granular view of the labour market of particular interest to skills policy practitioners.

For example, when making a choice about the right combination of skills to develop or career path to pursue, it can be helpful for an individual to get an impression of the number of opportunities that are currently available for jobs that they are interested in. Is there demand for people who are qualified to do the job? Are there sufficient opportunities for them to pursue a particular career in their local area? What types of skills do employers typically seek for recruits into that type of role?

6. How can I be sure of the quality of the data?
The websites visited by BGT software include Government pages such as Universal Jobmatch and the Apprenticeship Service, employers’ own websites, agency postings and postings on job boards. Each individual website accounts for no more than a few percent of the total aggregated data. The data parsing procedures used for UK postings have been developed locally and are updated regularly. Sites are vetted in advance to ensure they are a valid source of job opportunities.

Robust mechanisms have been put in place to de-duplicate job advertisements to ensure that the values presented are based on unique opportunities rather than an aggregation of all posting activity undertaken by recruiting and staffing agencies and other firms.

Within job postings dozens of data elements are parsed, extracted and coded. Burning Glass, for example, uses a curated set of tens of thousands of business roles to appropriately assign a job to an occupation.

Information gathering can only parse what is actually present in any given posting. For example, many of the job postings in London do not include further geographical information to identify a specific local authority. The online job postings data pilot presents users with a breakdown of how many job postings have not been classified, for each inquiry.

7. We are interested in other job postings data such as the employers, job titles, or monthly series. Why is this not being made available?
The range of data that we can make available through the trial is limited by budget constraints. If you have an opinion about the data that is most valuable for informing training provision and careers guidance we would like to hear from you.

Users in need of more detailed information may also consider accessing Burning Glass Technologies’ real-time labour market information tool Labour Insight, available via a subscription service direct with the supplier.

8. Are there any restrictions on how we can use the data?
There are additional restrictions placed on the use of this dataset, beyond what is already contained in our standard terms and conditions for London Datastore.

In line with the overall aim of the Datastore, developers may use the data as an input to applications and websites that are designed to support those individuals or organisations making choices about careers and learning options. The data may not be used for other commercial purposes, e.g. you may not download the data in bulk for resale. All outputs from the data should be cited as Burning Glass Technologies.

9. How long does the pilot run for and what happens afterwards?
Our current plan is to run a short pilot from the beginning of January 2017 to end June 2017 to gauge demand from local authorities, careers services, providers, data analysts, and other end users.

Depending on the level of demand and interest, we may seek to offer data of this kind on a longer term basis, or to extend the duration of the pilot, with support from our users.

In any event, we will not cut off access to the data on the 30 June; it will continue to be available afterwards but it may no longer be updated.

For further queries or to provide feedback, please contact us on: datastore@london.gov.uk