LoveMyData / onkaparinga

Onkaparinga SA Development Applications


City Of Onkaparinga Council Scraper

Note - This scraper will only scrape the primary DA and any subsequent DAs will not be scrape. eg. 683/2017/2 will not be scraped but 683/2017 will

  • Server - .NET - ePathway
  • Cookie tracking - Yes
  • Pagination - No
  • Javascript - Yes
  • Clearly defined data within a row - No but acceptable
  • Force scan one record at a time - Yes
  • User-Agent - Yes

Setup MORPH_PERIOD for data recovery, available options are

  • year value (default is this year)

Enjoy

Contributors emikulic henare LoveMyData

Last run failed with status code 999.

Console output of last run

Please install the Morph Github App on LoveMyData so that Morph can access this repository on GitHub. Please go to https://github.com/apps/morph-io/installations/new/permissions?repository_ids%5B%5D=49949842&repository_ids%5B%5D=134056878&repository_ids%5B%5D=47324812&repository_ids%5B%5D=88161914&repository_ids%5B%5D=68444421&repository_ids%5B%5D=47314642&repository_ids%5B%5D=93590295&repository_ids%5B%5D=68878229&repository_ids%5B%5D=72811725&repository_ids%5B%5D=56903014&repository_ids%5B%5D=170608803&repository_ids%5B%5D=108057954&repository_ids%5B%5D=92110889&repository_ids%5B%5D=67974508&repository_ids%5B%5D=87934825&repository_ids%5B%5D=145681800&repository_ids%5B%5D=47260725&repository_ids%5B%5D=88051332&repository_ids%5B%5D=91232291&repository_ids%5B%5D=47496147&repository_ids%5B%5D=68881868&repository_ids%5B%5D=47331141&repository_ids%5B%5D=87817830&repository_ids%5B%5D=177515286&repository_ids%5B%5D=46924700&repository_ids%5B%5D=106624312&repository_ids%5B%5D=46796215&repository_ids%5B%5D=91168909&repository_ids%5B%5D=67973091&repository_ids%5B%5D=107615216&repository_ids%5B%5D=85133473&repository_ids%5B%5D=91752802&repository_ids%5B%5D=47815673&repository_ids%5B%5D=99768398&repository_ids%5B%5D=67903220&repository_ids%5B%5D=104952368&repository_ids%5B%5D=51361943&repository_ids%5B%5D=47970061&repository_ids%5B%5D=67978193&repository_ids%5B%5D=104017335&repository_ids%5B%5D=47314661&repository_ids%5B%5D=46924659&repository_ids%5B%5D=91053021&repository_ids%5B%5D=156531533&repository_ids%5B%5D=96767060&repository_ids%5B%5D=57299199&repository_ids%5B%5D=156799498&repository_ids%5B%5D=159443632&repository_ids%5B%5D=47876278&repository_ids%5B%5D=159422813&repository_ids%5B%5D=47313800&repository_ids%5B%5D=107520757&repository_ids%5B%5D=88119130&repository_ids%5B%5D=95849825&repository_ids%5B%5D=68373869&repository_ids%5B%5D=46966464&repository_ids%5B%5D=102549557&repository_ids%5B%5D=152169956&repository_ids%5B%5D=69199374&repository_ids%5B%5D=47871639&repository_ids%5B%5D=108206347&repository_ids%5B%5D=92701523&repository_ids%5B%5D=124995950&repository_ids%5B%5D=98845885&repository_ids%5B%5D=68511763&repository_ids%5B%5D=85155894&repository_ids%5B%5D=47400030&repository_ids%5B%5D=105615080&repository_ids%5B%5D=90943444&repository_ids%5B%5D=97595478&repository_ids%5B%5D=68342363&repository_ids%5B%5D=46921298&repository_ids%5B%5D=68098781&repository_ids%5B%5D=47917197&repository_ids%5B%5D=174301699&repository_ids%5B%5D=88171467&repository_ids%5B%5D=92243788&repository_ids%5B%5D=103522127&repository_ids%5B%5D=97825255&repository_ids%5B%5D=57381657&repository_ids%5B%5D=107633285&repository_ids%5B%5D=58468748&repository_ids%5B%5D=107853888&repository_ids%5B%5D=107861107&repository_ids%5B%5D=91777660&repository_ids%5B%5D=107852602&repository_ids%5B%5D=49410115&repository_ids%5B%5D=47873122&repository_ids%5B%5D=47874423&repository_ids%5B%5D=100552305&repository_ids%5B%5D=92278881&repository_ids%5B%5D=96265559&suggested_target_id=15683093 Why? See https://morph.io/documentation/github_app

Data

Downloaded 2056 times by LoveMyData dcalde equivalentideas

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (20.2 MB) Use the API

rows 10 / 60635

council_reference address description info_url comment_url date_scraped date_received
1/2017
11 Glenloth Drive, HAPPY VALLEY SA 5159
Verandah
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-02
2/2017
5 Tiller Drive, SEAFORD SA 5169
Single storey dwelling (dwelling 2)
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-03
3/2017
13 Kingsbridge Drive, MORPHETT VALE SA 5162
Single storey dwelling
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-03
4/2017
1/1927 Main South Road, HACKHAM SA 5163
Demolition of dwelling
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-03
5/2017
12 Semillon Circuit, REYNELLA SA 5161
Verandah
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-03
6/2017
28 Angle Road, SEAFORD MEADOWS SA 5169
Verandah
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-03
7/2017
13 Fabian Crescent, MORPHETT VALE SA 5162
Shed
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-03
8/2017
6A Sturt Lane, HACKHAM SA 5163
Shed
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-03
9/2017
30 Meadowvale Road, COROMANDEL VALLEY SA 5051
Retaining wall
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-03
10/2017
20 Daintree Drive, SEAFORD RISE SA 5169
Carport
mailto:mail@onkaparinga.sa.gov.au
2017-05-19
2017-01-04

Statistics

Average successful run time: 4 minutes

Total run time: 5 days

Total cpu time used: about 2 hours

Total disk space used: 20.2 MB

History

  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

onkaparinga / scraper.rb