planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Saving record 25017799, 62 COOMBE RD LANGHORNE CREEK SA 5255 Saving record 25017803, 51 ECKERT RD BELVIDERE SA 5255 Saving record 25020517, 360 DEEP CREEK RD CURRENCY CREEK SA 5214 Saving record 25010556, UNIT 8 19 TROON DR NORMANVILLE SA 5204 Saving record 25020831, 1 DERWENT ST CUMBERLAND PARK SA 5041 Saving record 25021684, 17 RAMAGE ST UNLEY SA 5061 Saving record 25013201, 25A KNIGHTSBRIDGE RD LEABROOK SA 5068 Saving record 25017498, 4 ROACH ST KADINA SA 5554 Saving record 25014332, UNIT 4 17 MONTEREY DR PORT LINCOLN SA 5606 Saving record 25017305, 10 JOHNSTON ST STIRLING SA 5152 Saving record 25019541, 43 GLADSTONE RD PROSPECT SA 5082 Saving record 25014972, 33 MAWSON RD MEADOWS SA 5201 Saving record 25017796, 299 CLEMENTS RD LANGHORNE CREEK SA 5255 Saving record 25017798, 299 CLEMENTS RD LANGHORNE CREEK SA 5255 Saving record 25020927, 54 -70 LESLIE PL PORT ADELAIDE SA 5015 Saving record 25021100, LOT 903 THE GOLDEN WAY GOLDEN GROVE SA 5125 Saving record 25017806, 51 ECKERT RD BELVIDERE SA 5255 /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:329:in `fetch': 429 => Net::HTTPTooManyRequests for https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticedetail -- unhandled response (Mechanize::ResponseCodeError) from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:998:in `response_redirect' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:321:in `fetch' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:1323:in `post_form' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:536:in `post' from scraper.rb:25:in `block in <main>' from scraper.rb:10:in `each' from scraper.rb:10:in `<main>'

Data

Downloaded 3934 times by openaustralia mlandauer dcalde jesse-a-reilly repose-lecture DevAppProject softgrow odnagroup ianheggie-oaf eucodepy tobiasoberrauch Jedwards157 MichaelBone 93399339

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (25.1 MB) Use the API

rows 10 / 73316

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 3 days

Total cpu time used: about 1 hour

Total disk space used: 25.2 MB

History

  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    19 records added, 2 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    19 records added, 19 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 11 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb