jacksonj04 / leeds-city-council-members

README.md

Leeds City Council Members Scraper

A scraper which looks at the table of councillors on the Leeds City Council website, then at each individual councillor's page in turn to figure out their elected terms.

This scraper runs on Morph on a daily basis.

Maintainability

Why?

The data is used to inform two comparisons with the data in Wikidata, with the aim of keeping Wikidata up to date. These are:

In theory, neither of these pages should highlight any differences.

Mapping IDs

Leeds City Council IDs need mapping to Wikidata IDs - this is done in lcc_id_map.py. Missing IDs are flagged at the end of a scraper run.

All IDs in this list are scraped, even if they are not current councillors. This lets us keep a historic comparison.

Contributors jacksonj04

Last run failed with status code 999.

Console output of last run

Please install the Morph Github App on jacksonj04 so that Morph can access this repository on GitHub. Please go to https://github.com/apps/morph-io/installations/new/permissions?repository_ids%5B%5D=107424752&repository_ids%5B%5D=140699136&repository_ids%5B%5D=120324758&suggested_target_id=619082 Why? See https://morph.io/documentation/github_app

Data

Downloaded 2545 times by everypolitician jacksonj04

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (94 KB) Use the API

rows 10 / 380

ward url honorific_string party_id name party id ward_id wikidata_id current start lcc_id end
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=109
Councillor
Andrew Carter CBE
109-2012-05-03T00:00:00Z
Q55469119
false
2012-05-03T00:00:00Z
109
2016-05-05T00:00:00Z
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=109
Councillor
Andrew Carter CBE
109-2016-05-05T00:00:00Z
Q55469119
false
2016-05-05T00:00:00Z
109
2018-05-03T00:00:00Z
Calverley and Farsley
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=109
Councillor
Q9626
Andrew Carter CBE
Conservative
109-2018-05-03T00:00:00Z
Q55466776
Q55469119
true
2018-05-03T00:00:00Z
109
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=7108
Councillor
Wyn Kidger
7108-2018-05-08T00:00:00Z
Q55469199
false
2018-05-08T00:00:00Z
7108
2019-05-06T00:00:00Z
Morley South
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=7108
Councillor
Q55465915
Wyn Kidger
Morley Borough Independents
7108-2019-05-07T00:00:00Z
Q55466782
Q55469199
true
2019-05-07T00:00:00Z
7108
Pudsey
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=7109
Councillor
Q9626
Mark Harrison
Conservative
7109-2018-05-08T00:00:00Z
Q55466813
Q55469188
true
2018-05-08T00:00:00Z
7109
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=7106
Councillor
Mohammed Shahzad
7106-2018-05-04T00:00:00Z
Q55469247
false
2018-05-04T00:00:00Z
7106
2019-05-06T00:00:00Z
Moortown
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=7106
Councillor
Q6467393
Mohammed Shahzad
Labour & Co-operative Party
7106-2019-05-07T00:00:00Z
Q55466778
Q55469247
true
2019-05-07T00:00:00Z
7106
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=7107
Councillor
Andy Hutchison
7107-2018-05-08T00:00:00Z
Q55469192
false
2018-05-08T00:00:00Z
7107
2020-05-07T00:00:00Z
https://democracy.leeds.gov.uk/mgUserInfo.aspx?UID=7107
Councillor
Andy Hutchison
7107-current
Q55469192
true
7107

Statistics

Average successful run time: 8 minutes

Total run time: 11 days

Total cpu time used: about 1 hour

Total disk space used: 187 KB

History

  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • ...
  • Created on morph.io

Show complete history