planningalerts-scrapers / saplanningportal

South Australia Development Applications - SA Planning Portal

README.md

South Australia (SA) Planning Portal scraper

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

  • Server - .NET
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Ajax - Yes
  • Clearly defined data within a record - Yes
  • Scrape detail page - No

Enjoy

Contributors LoveMyData MichaelBone mlandauer benrfairless ianheggie-oaf Br3nda

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Using user agent: Ruby/3.2.2 PlanningAlerts scraper for SA Planning Portal/2.0 (https://www.planningalerts.org.au/about) Visiting html page to set cookies: https://plan.sa.gov.au/have_your_say/notified_developments/current_notified_developments ... Delaying for 24.45s to avoid 429 Too Many Requests errors /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Got 200 response from https://cdn.plan.sa.gov.au/public-notifications/getpublicnoticessummary Found 102 applications to process in random order with 15.0...25.0 seconds between requests. Reusing comment email and authority from existing record: 25033619 Saving record 25033619, UNIT 1 G 16 BAGOT ST NORTH ADELAIDE SA 5006 Reusing comment email and authority from existing record: 26017459 Saving record 26017459, 30 DEEMSTER AV CHRISTIES BEACH SA 5165 Reusing comment email and authority from existing record: 26019089 Saving record 26019089, 65 STONE JAR RD ANGASTON SA 5353 Reusing comment email and authority from existing record: 26017045 Saving record 26017045, 8 FULLER ST WALKERVILLE SA 5081 Reusing comment email and authority from existing record: 26009594 Saving record 26009594, 297 KENSINGTON RD KENSINGTON PARK SA 5068 Reusing comment email and authority from existing record: 26017039 Saving record 26017039, 1 -1A AUSTIN CR ST GEORGES SA 5064 Reusing comment email and authority from existing record: 26015121 Saving record 26015121, 139 GILLMAN RD OAKBANK SA 5243 Reusing comment email and authority from existing record: 26009816 Saving record 26009816, 37 BROWN ST WEST CROYDON SA 5008 Reusing comment email and authority from existing record: 26012093 Saving record 26012093, 27 INNOVATION DR TOTNESS SA 5250 Reusing comment email and authority from existing record: 26016976 Saving record 26016976, 164 -168 PORTRUSH RD TRINITY GARDENS SA 5068 Reusing comment email and authority from existing record: 26015334 Saving record 26015334, 33 BYARD TCE MITCHELL PARK SA 5043 Reusing comment email and authority from existing record: 26019367 Saving record 26019367, 120 MEAD ST PETERHEAD SA 5016 Reusing comment email and authority from existing record: 25032526 Saving record 25032526, 11A FLINDERS ST KENT TOWN SA 5067 Reusing comment email and authority from existing record: 26014227 Saving record 26014227, LOT 604 MILLSWOOD CR MILLSWOOD SA 5034 Reusing comment email and authority from existing record: 25010738 Saving record 25010738, LOT 309 AVERIS RD DUCK PONDS SA 5607 Reusing comment email and authority from existing record: 26017246 Saving record 26017246, LOT 21 PORT RD BEVERLEY SA 5009 Reusing comment email and authority from existing record: 25035193 Saving record 25035193, 373 DIAGONAL RD STURT SA 5047 Reusing comment email and authority from existing record: 26004775 Saving record 26004775, 24 HERMIT ST ROXBY DOWNS SA 5725 Reusing comment email and authority from existing record: 26004875 Saving record 26004875, 17 SEAVIEW AV MIDDLETON SA 5213 Reusing comment email and authority from existing record: 26018106 Saving record 26018106, 6 -8 COMMERCIAL RD SHEIDOW PARK SA 5158 Reusing comment email and authority from existing record: 26019464 Saving record 26019464, 19 CLIFTON ST MILLSWOOD SA 5034 Reusing comment email and authority from existing record: 25024461 Saving record 25024461, 7 MERCOWIE RD CRYSTAL BROOK SA 5523 Reusing comment email and authority from existing record: 26016615 Saving record 26016615, 1357 MAIL RD SOUTH HUMMOCKS SA 5550 Reusing comment email and authority from existing record: 26018333 Saving record 26018333, 55 ELIZABETH ST NORWOOD SA 5067 Reusing comment email and authority from existing record: 26015317 Saving record 26015317, 193 WALKERVILLE TCE WALKERVILLE SA 5081 Reusing comment email and authority from existing record: 26012258 Saving record 26012258, 10 CYPRESS DR MORPHETT VALE SA 5162 Reusing comment email and authority from existing record: 26018008 Saving record 26018008, 28 LOWBANK RD KANNI SA 5330 Reusing comment email and authority from existing record: 26017172 Saving record 26017172, 10 JOHNSTON ST STIRLING SA 5152 Reusing comment email and authority from existing record: 26004571 Saving record 26004571, 98 FREDERICK ST WELLAND SA 5007 Reusing comment email and authority from existing record: 26016585 Saving record 26016585, 33 45 54 06 S 136 53 39 71 E 33 45 39 21 S 136 53 18 39 E Reusing comment email and authority from existing record: 26009234 Saving record 26009234, 29 SALTASH AV CHRISTIES BEACH SA 5165 Reusing comment email and authority from existing record: 24010498 Saving record 24010498, 77 FREDERICK ST WELLAND SA 5007 Reusing comment email and authority from existing record: 26011376 Saving record 26011376, 57 MARINE PDE KINGSTON SE SA 5275 Reusing comment email and authority from existing record: 26015315 Saving record 26015315, 496 NELSON RD GULFVIEW HEIGHTS SA 5096 Reusing comment email and authority from existing record: 25019950 Saving record 25019950, 121 OLD BETHEL RD WILLIAMSTOWN SA 5351 Reusing comment email and authority from existing record: 26011790 Saving record 26011790, 33 GATES RD HACKHAM SA 5163 Reusing comment email and authority from existing record: 26019048 Saving record 26019048, 34 CRAMOND PWY OAKDEN SA 5086 Reusing comment email and authority from existing record: 26011951 Saving record 26011951, 2 CALCA ST SCEALE BAY SA 5680 Reusing comment email and authority from existing record: 26018637 Saving record 26018637, 158 COWRA ST RENMARK SA 5341 Reusing comment email and authority from existing record: 26004226 Saving record 26004226, 7 SPROD AV TOORAK GARDENS SA 5065 Reusing comment email and authority from existing record: 26016910 Saving record 26016910, 2 CAPPERS DR BROMPTON SA 5007 Reusing comment email and authority from existing record: 26010724 Saving record 26010724, 3 SANCTUARY AV SHEIDOW PARK SA 5158 Reusing comment email and authority from existing record: 25032087 Saving record 25032087, 36 SURREY RD KESWICK SA 5035 Reusing comment email and authority from existing record: 26007948 Saving record 26007948, 17 WILSON ST ADELAIDE SA 5000 Reusing comment email and authority from existing record: 26010572 Saving record 26010572, 81A RIDGE RD SUMMERTOWN SA 5141 Reusing comment email and authority from existing record: 26019800 Saving record 26019800, 37 POWELL ST MOUNT GAMBIER SA 5290 Reusing comment email and authority from existing record: 25030284 Saving record 25030284, 167 LANGHORNE CREEK RD STRATHALBYN SA 5255 Reusing comment email and authority from existing record: 26014013 Saving record 26014013, 12 EAST TCE HAWTHORNDENE SA 5051 Reusing comment email and authority from existing record: 25024405 Saving record 25024405, 86A -86 PROSPECT RD PROSPECT SA 5082 Reusing comment email and authority from existing record: 25021414 Saving record 25021414, LOT 462 EDEN VALLEY RD FLAXMAN VALLEY SA 5235 Reusing comment email and authority from existing record: 26019468 Saving record 26019468, 550 STOCKWELL RD LIGHT PASS SA 5355 Reusing comment email and authority from existing record: 26013760 Saving record 26013760, 11 GOVER ST NORTH ADELAIDE SA 5006 Reusing comment email and authority from existing record: 26005122 Saving record 26005122, 28 -45 BROUGHAM PL NORTH ADELAIDE SA 5006 Reusing comment email and authority from existing record: 25036289 Saving record 25036289, LOT 3 STROUT RD MCLAREN VALE SA 5171 Reusing comment email and authority from existing record: 26004891 Saving record 26004891, LOT 103 ESPLANADE SEMAPHORE SA 5019 Reusing comment email and authority from existing record: 26017458 Saving record 26017458, 15 TORRENS ST COLLEGE PARK SA 5069 Reusing comment email and authority from existing record: 26013618 Saving record 26013618, 10 GRACE HARWAR DR PORT BROUGHTON SA 5522 Reusing comment email and authority from existing record: 26019683 Saving record 26019683, LOT 21 GOOCH RD COOKE PLAINS SA 5261 Reusing comment email and authority from existing record: 25036047 Saving record 25036047, 6 ANDERSON ST STRATHALBYN SA 5255 Reusing comment email and authority from existing record: 26019541 Saving record 26019541, 7 WHALEBOAT CT VICTOR HARBOR SA 5211 Reusing comment email and authority from existing record: 25039789 Saving record 25039789, 44A COOMURRA DR SALISBURY HEIGHTS SA 5109 Reusing comment email and authority from existing record: 26019388 Saving record 26019388, 16A MILLICENT ST ATHOL PARK SA 5012 Reusing comment email and authority from existing record: 24018868 Saving record 24018868, LOT 153 STATION RD SOUTH HUMMOCKS SA 5550 Reusing comment email and authority from existing record: 26012171 Saving record 26012171, 37 -39 WEST ST SEACLIFF PARK SA 5049 Reusing comment email and authority from existing record: 26009619 Saving record 26009619, 43 BARBARA ST PORT ELLIOT SA 5212 Reusing comment email and authority from existing record: 26017688 Saving record 26017688, 17 FORT ST GRANGE SA 5022 Reusing comment email and authority from existing record: 26011206 Saving record 26011206, 2 THIRTEENTH ST GAWLER SOUTH SA 5118 Reusing comment email and authority from existing record: 26016765 Saving record 26016765, 8 PROVIDENT AV GLYNDE SA 5070 Reusing comment email and authority from existing record: 26010520 Saving record 26010520, LOT 30 SOUTH RD CLARENCE GARDENS SA 5039 Reusing comment email and authority from existing record: 26013452 Saving record 26013452, 13 SEVENTH AV ST MORRIS SA 5068 Reusing comment email and authority from existing record: 26016152 Saving record 26016152, 53 -55 WEST PWY COLONEL LIGHT GARDENS SA 5041 Reusing comment email and authority from existing record: 26020920 Saving record 26020920, 16 BLANCHE AV MAGILL SA 5072 Reusing comment email and authority from existing record: 26009660 Saving record 26009660, 10 EMU ST SEMAPHORE SA 5019 Reusing comment email and authority from existing record: 26013312 Saving record 26013312, 3284 MAIN NORTH RD EVANSTON SOUTH SA 5116 Reusing comment email and authority from existing record: 26019346 Saving record 26019346, 143 -173 BAINS RD MORPHETT VALE SA 5162 Reusing comment email and authority from existing record: 26009466 Saving record 26009466, 360 HILLIER RD HILLIER SA 5116 Reusing comment email and authority from existing record: 26016980 Saving record 26016980, LOT 12 LINCOLN HWY POONINDIE SA 5607 Reusing comment email and authority from existing record: 26004599 Saving record 26004599, 11 HAYNES AV RICHMOND SA 5033 Reusing comment email and authority from existing record: 26004581 Saving record 26004581, 10 CHARTERIS ST PORT ELLIOT SA 5212 Reusing comment email and authority from existing record: 26013842 Saving record 26013842, 46 KENNAWAY ST TUSMORE SA 5065 Reusing comment email and authority from existing record: 26012532 Saving record 26012532, 191 TEMORA WAY PALLAMANA SA 5254 Reusing comment email and authority from existing record: 26012176 Saving record 26012176, 52 MARINE PDE KINGSTON SE SA 5275 Reusing comment email and authority from existing record: 26018127 Saving record 26018127, 30 LYONS RD HOLDEN HILL SA 5088 Reusing comment email and authority from existing record: 26010064 Saving record 26010064, 32 SANDPIPER TCE HALLETT COVE SA 5158 Reusing comment email and authority from existing record: 26020626 Saving record 26020626, 15 ANTIGUA GR WEST LAKES SA 5021 Reusing comment email and authority from existing record: 26014279 Saving record 26014279, 22A HUMPHRIES TCE KILKENNY SA 5009 Reusing comment email and authority from existing record: 26012275 Saving record 26012275, LOT 100 GEORGE ST PORT WAKEFIELD SA 5550 Reusing comment email and authority from existing record: 26002805 Saving record 26002805, 1A VIEW RD BLACKWOOD SA 5051 Reusing comment email and authority from existing record: 26010051 Saving record 26010051, 29 CAMPANIA RD N KUDLA SA 5115 Reusing comment email and authority from existing record: 26014102 Saving record 26014102, 47 BURTON RD ATHELSTONE SA 5076 Reusing comment email and authority from existing record: 26014536 Saving record 26014536, 1A SEAGULL AV CHITON SA 5211 Reusing comment email and authority from existing record: 26016921 Saving record 26016921, 2 PULLEINE RD NAIRNE SA 5252 Reusing comment email and authority from existing record: 26013635 Saving record 26013635, 526 FULLARTON RD NETHERBY SA 5062 Reusing comment email and authority from existing record: 24019166 Saving record 24019166, 255 MAGOR RD COONAMIA SA 5540 Reusing comment email and authority from existing record: 26015179 Saving record 26015179, 77 WIGLEY FLAT RD WIGLEY FLAT SA 5330 Reusing comment email and authority from existing record: 26016722 Saving record 26016722, 9 EGMONT TCE ERINDALE SA 5066 Reusing comment email and authority from existing record: 26020134 Saving record 26020134, 3 -4 SHEOAK GR LUCINDALE SA 5272 Reusing comment email and authority from existing record: 26015678 Saving record 26015678, UNIT 9 238 CHANDLERS HILL RD HAPPY VALLEY SA 5159 Reusing comment email and authority from existing record: 26011906 Saving record 26011906, LOT 51 HARTWIG RD TAYLORVILLE SA 5330 Reusing comment email and authority from existing record: 26014332 Saving record 26014332, 10 PANORAMA CT PARINGA SA 5340 Reusing comment email and authority from existing record: 26017595 Saving record 26017595, 143 YOUNG ST PARKSIDE SA 5063 Reusing comment email and authority from existing record: 26007501 Saving record 26007501, 123 -125 LEICESTER ST PARKSIDE SA 5063 First Application data retrieved: {"publicNotificationID"=>13569, "applicationID"=>25024461, "applicationDocumentStorageMetadataID"=>0, "propertyAddress"=>"7 MERCOWIE RD CRYSTAL BROOK SA 5523", "developmentDescription"=>"Domestic storage shed (outbuilding), inclusive of storage of a tipper truck of 4770 kg", "closingDate"=>"7/29/2026 12:00:00 AM", "decisionAuthority"=>"Upper Spencer Gulf Regional Assessment Panel", "relevantAuthority"=>"Port Pirie Regional Council", "locationCouncil"=>"PORT PIRIE REGIONAL COUNCIL", "notificationChannel"=>"DAP", "publicConsultationUrl"=>"", "applicants"=>[{"interestedPartyId"=>960439, "applicationId"=>25024461, "contactName"=>"Steven Marks ", "address"=>"42 CRAWFORD CRESCENT MANNUM 5238"}]} Found 102 applications that were already in the database, and added 0 new applications. Added 24.45 seconds total delay over 1 requests so server doesn't complain! Count Authority ----- ------------------ 3 Adelaide Hills Council 6 Alexandrina Council 3 Barunga West Council 4 City of Adelaide 5 City of Burnside 2 City of Campbelltown 9 City of Charles Sturt 6 City of Marion 5 City of Mitcham 1 City of Mt Gambier 6 City of Norwood, Payneham and St. Peters 7 City of Onkaparinga 5 City of Port Adelaide Enfield 1 City of Prospect 2 City of Salisbury 4 City of Unley 1 City of Victor Harbor 2 City of West Torrens 1 District Council of Franklin Harbour 2 District Council of Kingston 2 District Council of Loxton Waikerie 1 District Council of Streaky Bay 2 Lower Eyre Council 2 Mount Barker District Council 1 Municipal Council of Roxby Downs 1 Naracoorte Lucindale Council 2 Port Pirie Regional Council 1 Renmark Paringa Council 1 Rural City of Murray Bridge 2 State Planning Commission 4 The Barossa Council 1 The Coorong District Council 2 The Corporation of the Town of Walkerville 4 Town of Gawler 1 Wakefield Regional Council

Data

Downloaded 4878 times by openaustralia mlandauer dcalde jesse-a-reilly Manasaanand tkrikke repose-lecture AthleticoPrawn DevAppProject softgrow qs4v8thvrq-boop odnagroup ianheggie-oaf 538digital MichaelBone 93399339 eucodepy seegolovemee tobiasoberrauch Jedwards157

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (26 MB) Use the API

rows 10 / 74780

id council_reference address description info_url comment_url date_scraped date_received on_notice_to comment_email comment_authority on_notice_from
1
252/D341/17
Wattle Street Brompton, SA
Boundary Realignment. Torrens Title division. 2 semi-detached dwellings. 1 detached dwelling.
http://www.saplanningportal.sa.gov.au/public_register#view-59720-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-15
2
040/D295/17
17-23 Ashburton Street Blair Athol, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59714-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
3
145/D202/17
Valley View Drive McLaren Vale, SA
1 in to 2 land division
http://www.saplanningportal.sa.gov.au/public_register#view-59712-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
4
170/D195/17
Andrea Avenue Newton, SA
Torrens Title land division. One allotment into three. Existing dwelling to be demolished.
http://www.saplanningportal.sa.gov.au/public_register#view-59628-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
5
170/D196/17
8-10 Gilbert Street NEWTON, SA
1 into 4 Torrens division
http://www.saplanningportal.sa.gov.au/public_register#view-59717-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
6
211/D172/17
37 and 39 Malurus Avenue Lockleys, SA
division of land to create 3 new allotments from the existing two.
http://www.saplanningportal.sa.gov.au/public_register#view-59713-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
7
361/D188/17
19 Uranbo Street Salisbury North, SA
Torrens Land Division. One Allotment into Two Allotments.
http://www.saplanningportal.sa.gov.au/public_register#view-59715-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
8
361/D189/17
7 Gilbert Street Ingle Farm, SA
Land Division
http://www.saplanningportal.sa.gov.au/public_register#view-59719-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-13
9
100/D241/17
9 Rosedale Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59699-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12
10
100/D242/17
19 Lawson Avenue Morphettville, SA
Division of land to create two new Torrens title allotments
http://www.saplanningportal.sa.gov.au/public_register#view-59700-LDE
http://www.saplanningportal.sa.gov.au/public_register
2017-10-17
2017-10-12

Statistics

Average successful run time: 3 minutes

Total run time: 4 days

Total cpu time used: about 1 hour

Total disk space used: 26.1 MB

History

  • Auto ran revision b38f71b9 and completed successfully .
    102 records added, 102 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    102 records added, 94 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    107 records added, 106 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    111 records added, 98 records removed, 1 record updated in the database
  • Auto ran revision b38f71b9 and completed successfully .
    106 records added, 106 records removed, 1 record updated in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

saplanningportal / scraper.rb