ianheggie-oaf / banyule

Banyule City Council Development Applications


Banyule City Council Scraper

  • Server - Unknown
  • Cookie tracking - No
  • Pagnation - No
  • Javascript - No
  • Clearly defined data within a row - Yes
  • Scrape detail page - Yes

Enjoy

This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

Set MORPH_AUSTRALIAN_PROXY to the url for an Australian proxy

Error handling

The scraper will * warn if it was unable to extract the council reference and skip to the next page * warn if it was unable to extract the address from the details page and skip to the next entry on the same page * stop retrieving pages if no records where found or no next page link is present * Return a non-zero status at the end of run if any warnings occurred during the run so morph can alert the scaper owner

Expected output

Saving record P820/2025 - 3 Hebden Street, Greensborough 3088 VIC
Saving record P1288/2015 PT3 - 4/37 The Concord, Bundoora 3083 VIC
(etc)
Saving record P703/2025 - 17 Bruce Street, Greensborough 3088 VIC
Saving record P968/2025 - 47 Greville Road, Rosanna 3084 VIC

Continuing to page 2
Saving record P676/2025 pt1 - 29 Timor Parade, Heidelberg West 3081 VIC
Saving record P316/2025 - 18-28 Irvine Road, Ivanhoe 3079 VIC
(etc)
Saving record P497/2025 - 276 Oriel Road, Heidelberg West 3081 VIC
Saving record P851/2025 - 201 Waiora Road, Heidelberg Heights 3081 VIC

Continuing to page 3
Saving record P382/2025 - 3 Carisbrook Crescent, Lower Plenty 3093 VIC
(etc)

Execution time under a minute

To run style and coding checks

bundle exec rubocop

To check for security updates

gem install bundler-audit
bundle-audit

Contributors mlandauer LoveMyData AndreM101 jamezpolley ianheggie-oaf benrfairless simonzippy

Last run failed with status code 3.

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Ruby app detected [1G-----> Installing bundler 2.3.10 [1G-----> Removing BUNDLED WITH version in the Gemfile.lock [1G-----> Compiling Ruby [1G-----> Using Ruby version: ruby-3.2.2 [1G-----> Installing dependencies using bundler 2.3.10 [1G Running: BUNDLE_WITHOUT='development:test' BUNDLE_PATH=vendor/bundle BUNDLE_BIN=vendor/bundle/bin BUNDLE_DEPLOYMENT=1 bundle install -j4 [1G Fetching gem metadata from https://rubygems.org/......... [1G Fetching https://github.com/openaustralia/scraperwiki-ruby.git [1G Fetching rake 12.3.3 [1G Installing rake 12.3.3 [1G Fetching public_suffix 6.0.2 [1G Fetching ansi 1.5.0 [1G Fetching ast 2.4.3 [1G Fetching base64 0.3.0 [1G Installing ast 2.4.3 [1G Installing base64 0.3.0 [1G Installing ansi 1.5.0 [1G Installing public_suffix 6.0.2 [1G Using bundler 2.4.10 [1G Fetching connection_pool 2.5.3 [1G Fetching diff-lcs 1.6.2 [1G Fetching docile 1.4.1 [1G Installing connection_pool 2.5.3 [1G Installing diff-lcs 1.6.2 [1G Fetching domain_name 0.6.20240107 [1G Installing docile 1.4.1 [1G Fetching mutex_m 0.3.0 [1G Fetching json 2.17.1 [1G Installing domain_name 0.6.20240107 [1G Installing mutex_m 0.3.0 [1G Fetching language_server-protocol 3.17.0.5 [1G Installing json 2.17.1 with native extensions [1G Fetching lint_roller 1.1.0 [1G Installing language_server-protocol 3.17.0.5 [1G Installing lint_roller 1.1.0 [1G Fetching logger 1.7.0 [1G Installing logger 1.7.0 [1G Fetching mime-types-data 3.2025.0603 [1G Fetching net-http-digest_auth 1.4.1 [1G Installing net-http-digest_auth 1.4.1 [1G Installing mime-types-data 3.2025.0603 [1G Fetching racc 1.8.1 [1G Installing racc 1.8.1 with native extensions [1G Fetching webrick 1.9.1 [1G Installing webrick 1.9.1 [1G Fetching webrobots 0.1.2 [1G Fetching parallel 1.27.0 [1G Installing webrobots 0.1.2 [1G Installing parallel 1.27.0 [1G Fetching rainbow 3.1.1 [1G Fetching prism 1.6.0 [1G Installing rainbow 3.1.1 [1G Fetching regexp_parser 2.11.3 [1G Installing regexp_parser 2.11.3 [1G Installing prism 1.6.0 with native extensions [1G Fetching rspec-support 3.13.4 [1G Installing rspec-support 3.13.4 [1G Fetching ruby-progressbar 1.13.0 [1G Installing ruby-progressbar 1.13.0 [1G Fetching unicode-emoji 4.0.4 [1G Installing unicode-emoji 4.0.4 [1G Fetching sqlite3 1.6.9 (x86_64-linux) [1G Fetching simplecov-html 0.13.1 [1G Installing simplecov-html 0.13.1 [1G Fetching rubyntlm 0.6.5 [1G Installing rubyntlm 0.6.5 [1G Fetching addressable 2.8.7 [1G Installing addressable 2.8.7 [1G Fetching net-http-persistent 4.0.6 [1G Installing net-http-persistent 4.0.6 [1G Installing sqlite3 1.6.9 (x86_64-linux) [1G Fetching httpclient 2.9.0 [1G Installing httpclient 2.9.0 [1G Fetching http-cookie 1.0.8 [1G Installing http-cookie 1.0.8 [1G Fetching mime-types 3.7.0 [1G Installing mime-types 3.7.0 [1G Fetching rspec-core 3.13.4 [1G Fetching rspec-expectations 3.13.5 [1G Installing rspec-core 3.13.4 [1G Installing rspec-expectations 3.13.5 [1G Fetching rspec-mocks 3.13.5 [1G Fetching nokogiri 1.16.8 (x86_64-linux) [1G Installing rspec-mocks 3.13.5 [1G Fetching parser 3.3.10.0 [1G Installing parser 3.3.10.0 [1G Installing nokogiri 1.16.8 (x86_64-linux) [1G Fetching unicode-display_width 3.1.4 [1G Installing unicode-display_width 3.1.4 [1G Fetching simplecov 0.18.5 [1G Installing simplecov 0.18.5 [1G Fetching sqlite_magic 0.0.6 [1G Installing sqlite_magic 0.0.6 [1G Fetching rspec 3.13.1 [1G Installing rspec 3.13.1 [1G Fetching terminal-table 4.0.0 [1G Installing terminal-table 4.0.0 [1G Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176) [1G Fetching simplecov-console 0.9.3 [1G Installing simplecov-console 0.9.3 [1G Fetching mechanize 2.8.5 [1G Installing mechanize 2.8.5 [1G Fetching rubocop-ast 1.48.0 [1G Installing rubocop-ast 1.48.0 [1G Fetching rubocop 1.81.7 [1G Installing rubocop 1.81.7 [1G Bundle complete! 9 Gemfile dependencies, 50 gems now installed. [1G Gems in the groups 'development' and 'test' were not installed. [1G Bundled gems are installed into `./vendor/bundle` [1G Bundle completed (20.57s) [1G Cleaning up the bundler cache. [1G Removing bundler (2.3.10) [1G-----> Detecting rake tasks [1G [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... Using Australian proxy... Saving record P820/2025 - 3 Hebden Street, Greensborough 3088 VIC Saving record P1288/2015 PT3 - 4/37 The Concord, Bundoora 3083 VIC Saving record P714/2025 - 122 Porter Road, Heidelberg Heights 3081 VIC Saving record P681/2025 - 36 Hardy Terrace, Ivanhoe East 3079 VIC Saving record P807/2025 - 11 Rand Street, Greensborough 3088 VIC Saving record P1155/2022 pt4 - 121-125 Northern Road, Heidelberg West 3081 VIC Saving record P100/2025 - 229-231 Lower Heidelberg Road, Ivanhoe East 3079 VIC Saving record P633/2025 - 203 Southern Road, Heidelberg West 3081 VIC Saving record P703/2025 - 17 Bruce Street, Greensborough 3088 VIC Saving record P968/2025 - 47 Greville Road, Rosanna 3084 VIC Continuing to page 2 Saving record P676/2025 pt1 - 29 Timor Parade, Heidelberg West 3081 VIC Saving record P316/2025 - 18-28 Irvine Road, Ivanhoe 3079 VIC Saving record P861/2013 Part 2 - 23 & 25 Kolora Road, Heidelberg West 3081 VIC Saving record P825/2025 - 8 Boyd Crescent, Heidelberg West 3081 VIC Saving record P1344/2022 Part 4 - 598 & 600 Morwell Avenue, Bundoora 3083 VIC Saving record P628/2025 - 129 Old Eltham Road, Lower Plenty 3093 VIC Saving record P1031/2024 - 83 Karingal Drive, Briar Hill 3088 VIC Saving record P797/2025 - 102 St James Road, Heidelberg 3084 VIC Saving record P497/2025 - 276 Oriel Road, Heidelberg West 3081 VIC Saving record P851/2025 - 201 Waiora Road, Heidelberg Heights 3081 VIC Continuing to page 3 Saving record P382/2025 - 3 Carisbrook Crescent, Lower Plenty 3093 VIC Saving record P673/2025 - 6 The Panorama, Eaglemont 3084 VIC WARNING: Unable to extract address from detail page (no map view?) [Skipping to next record] Saving record P768/2025 - 205 Lower Heidelberg Road, Ivanhoe East 3079 VIC Saving record P694/2025 - 9 Buna Street, Heidelberg West 3081 VIC Saving record P315/2025 - 21 Outlook Drive, Eaglemont 3084 VIC Saving record P1000/2025 - 41 Mountain View Parade, Rosanna 3084 VIC Saving record P746/2017 pt3 - 1/63 Main Road, Lower Plenty 3093 VIC Saving record P833/2025 - 1/144, 2/144 and 144 Porter Road, Heidelberg Heights 3081 VIC Saving record P1535/2021 pt3 - 7 Ryrie Court, Montmorency 3094 VIC Continuing to page 4 Saving record P912/2025 - 27 Greenwood Street, Briar Hill 3088 VIC Saving record P572/2025 - 8 Airlie Road, Montmorency 3094 VIC Saving record P47/2020 pt6 - 45 Cleveland Avenue, Lower Plenty 3093 VIC Saving record P601/2025 - 25 Skeffington Street, Bellfield 3081 VIC Saving record P766/2025 - 29 Kenmare Street, Watsonia 3087 VIC Saving record P638/2025 - 23 Lilicur Road, Montmorency 3094 VIC Saving record P803/2025 - 68 Cape Street, Heidelberg 3084 VIC Saving record P1075/2024 - 126 Nell Street, Greensborough 3088 VIC Saving record P594/2025 - 46 Dunvegan Crescent, Macleod 3085 VIC Saving record P895/2025 - 171 Upper Heidelberg Road, Ivanhoe 3079 VIC Continuing to page 5 Saving record P156/2025 - 77 Altona Street, Heidelberg Heights 3081 VIC Saving record P491/2025 - 22 Beleura Grove, Lower Plenty 3093 VIC Saving record P827/2025 - 152 The Boulevard, Ivanhoe East 3079 VIC Saving record P624/2025 - 10 Setani Crescent, Heidelberg West 3081 VIC Exiting on last page (no Next button) Finished with errors

Data

Downloaded 20122 times by ianheggie-oaf dcalde tkrikke

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (100 KB) Use the API

rows 10 / 214

council_reference address description info_url comment_url date_scraped on_notice_to
P157/2025
54 Alandale Road, Eaglemont 3084 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-14
2025-04-18
P926/2024 pt1
343-345 The Boulevard, Ivanhoe East 3079 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-14
2025-04-18
P1188/2024
25 Edward Street, Macleod 3085 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-14
2025-04-18
P1243/2024
19 Blackwood Parade, Heidelberg West 3081 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-14
2025-04-18
P1165/2024
582 Morwell Avenue, Bundoora 3083 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-14
2025-04-18
P175/2025
18 Harborne Street, Macleod 3085 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-14
2025-04-18
P961/2024
9 Waldemar Road, Eaglemont 3084 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-14
2025-04-18
P929/2024
182 Cape Street, Heidelberg 3084 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-19
2025-04-22
P111/2025
13 Marshall Street, Ivanhoe 3079 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-19
2025-04-22
P988/2024
25 Beauview Parade, Ivanhoe East 3079 VIC
mailto:enquiries@banyule.vic.gov.au
2025-05-19
2025-04-22

Statistics

Average successful run time: 2 minutes

Total run time: about 9 hours

Total cpu time used: 6 minutes

Total disk space used: 162 KB

History

  • Manually ran revision 1829c36c and failed .
    43 records added, 39 records removed in the database
  • Auto ran revision 06d80d3e and failed .
    22 records added, 17 records removed in the database
  • Auto ran revision 06d80d3e and failed .
    17 records added, 17 records removed in the database
  • Auto ran revision 06d80d3e and failed .
    17 records added, 17 records removed in the database
  • Auto ran revision 06d80d3e and failed .
    17 records added, 17 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

banyule / scraper.rb