planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Saving record 25018352, 112 TAPALIN ST RENMARK WEST SA 5341 Saving record 24040980, 180 DRY PLAINS RD STRATHALBYN SA 5255 Saving record 24039236, 741 SEPPELTSFIELD RD SEPPELTSFIELD SA 5355 Saving record 23036372, 5 GULLY VIEW CT SALISBURY HEIGHTS SA 5109 Saving record 25019513, 134 -144 HUTT ST ADELAIDE SA 5000 Saving record 25019949, 66 HAWSON AV NORTH PLYMPTON SA 5037 Saving record 25017134, 2 NORTHCOTE ST TORRENSVILLE SA 5031 Saving record 25013879, 75 PENONG AV CAMDEN PARK SA 5038 Saving record 25015903, 179 KYNOCH RD MCCALLUM SA 5267 Saving record 25019906, 37 -37A HUMPHRIES TCE KILKENNY SA 5009 Saving record 24041012, 8 DELBRIDGE CT BEAUMONT SA 5066 Saving record 25012928, 18 TORONTO ST OVINGHAM SA 5082 Saving record 25008580, 253 MAIN RD BLACKWOOD SA 5051 Saving record 25019071, 3 DEEDS RD CAMDEN PARK SA 5038 Saving record 25004286, 12 WILLCOX AV PROSPECT SA 5082 Saving record 25011662, 406 -412 FINDON RD KIDMAN PARK SA 5025 Saving record 25020710, 467 MILITARY RD GRANGE SA 5022 /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:329:in `fetch': 429 => Net::HTTPTooManyRequests for https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticedetail -- unhandled response (Mechanize::ResponseCodeError) from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:998:in `response_redirect' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize/http/agent.rb:321:in `fetch' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:1323:in `post_form' from /app/vendor/bundle/ruby/2.5.0/gems/mechanize-2.7.6/lib/mechanize.rb:536:in `post' from scraper.rb:25:in `block in <main>' from scraper.rb:10:in `each' from scraper.rb:10:in `<main>'

Data

Downloaded 3954 times by openaustralia mlandauer dcalde jesse-a-reilly repose-lecture DevAppProject softgrow odnagroup ianheggie-oaf tobiasoberrauch Jedwards157 MichaelBone 93399339 eucodepy

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (25.1 MB) Use the API

rows 10 / 73427

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 3 days

Total cpu time used: about 1 hour

Total disk space used: 25.2 MB

History

  • Auto ran revision 9fd84356 and failed .
    17 records added, 7 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    18 records added, 18 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 9 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 17 records removed, 1 record updated in the database
  • Auto ran revision 9fd84356 and failed .
    17 records added, 10 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb