Confirmed updates to API, CSV files and documentation
Hi everyone,
Thank you for your useful comments, suggestions and feedback about the police.uk data during the past couple of months, on this forum, by email, and in person.
I'm pleased to say that we're able to take quite a few of improvements forward, hopefully resulting in an open data offering that is more consistent, easy-to-use, and reliable in the long term.
Firstly, all police.uk open data related things (except the application showcase) will be migrating across to the data.police.uk website, which we're currently revamping in line with the Open Data Institute best practices.
From early July, we'll be providing much better information about data provenance, quality issues, our assurance and anonymisation processes, and tools for working with the data. We'll also be making it much easier to get in touch with the Home Office about any concerns, issues or suggestions, as well as launching a Data User Group so it's easier to for you to influence future policy decisions.
In terms of the data itself, we'll be launching a clearer change log and providing single-click, permalinked archives of the full crime and outcomes database each month in CSV format.
Based mainly on the feedback from this forum, we'll also be providing a 'custom download' feature so you can bulk download data in CSV format for a specific time period and set of forces. A picture of the in-development site explains it better than I probably can : )
https://dl.dropboxusercontent.com/u/22172657/data_custom-downloads.png
The CSV files contained in the download will be very similar to those currently provided on police.uk/data, with the following changes:
- We won't be providing an aggregation of the data by Neighbourhood Team beyond the end of August. It will still be possible to get this data using the new 'poly' parameter in the API, and we'll make sure to provide help and guidance to anyone who needs to transition over to it.
- The 2011 Lower Super Output Area code and name will be listed alongside all crimes.
- The last known 'status' for each crime (e.g. under investigation, offender cautioned) will be included alongside the rest of the crime information, in addition to the current outcomes files.
- They will use WGS84 latitude and longitude, rather than eastings and northings.
I'll post a full example file to this thread as soon as I have one.
In terms of boundary files, we'll be making the KML files for police forces boundaries much easier to download, and well as a continuing to publish current and historical archives for the (regually changing) neighbourhood team boundaries.
A 'poly' parameter will also be added to the street-level crimes and street-level outcome methods in the API, allowing you to query for custom shapes (like wards, neighbourhood watch areas, councils) instead of just a set radius around a single lat long. The format will be:
poly=[lat],[lng]:[lat],[lng]:[lat],[lng]
The last point does not have to be the same as the first, as the system will join them up automatically. For example:
/api/crimes-street/all-crime?date=2013-01&poly=52.268,0.543:52.794,0.238:52.130,0.478
We're also going to be making a few additions to the API documentation in general over the coming months, based on comments and questions that users have been kind enough to send us.
We'll send out a message to the police.uk mailing list and post on this forum when the new data.police.uk site is ready (all things being well, early next week). All data will also continue to be available in their current formats until the end of August.
Thanks
Alex