planningalerts-scrapers / rockingham

City of Rockingham Town planning advertising and submissions Scraper

README.md

City of Rockingham - Town planning advertising and submissions

  • Cookie tracking - No
  • Pagnation - none obvious
  • JavaScript - No
  • Clearly defined data within a row - Clear enough in HTML dom, with additional address details in free text in details page
  • System - custom

This is a scraper that runs on Morph. To get started see the documentation

Add any issues to https://github.com/planningalerts-scrapers/issues/issues

To run the scraper

bundle exec ruby scraper.rb

Expected output

Getting listing page
  Pausing 4.057s
  Fetching detail page: https://rockingham.wa.gov.au/planning-and-building/local-planning/town-planning-advertising-and-submissions/proposed-oilseed-processing-plant-patterson-road,-
Saving record Proposed Oilseed Processing Plant Patterson Road - - Lot 9008 Patterson Road, East Rockingham, WA
...
  Pausing 3.815s
  Fetching detail page: https://rockingham.wa.gov.au/planning-and-building/local-planning/town-planning-advertising-and-submissions/proposed-holiday-house-emerald-court,-singleton
Saving record Proposed Holiday House Emerald Court Singleton - 38 Emerald Court, Singleton, WA
Deleting 0 applications scraped between  and 2025-12-28
  Running VACUUM to reclaim space...
Finished! Added 3 applications, and skipped 0 unprocessable applications.

Execution time: ~ 30 seconds

To run style and coding checks

bundle exec rubocop

To check for security updates

gem install bundler-audit
bundle-audit

Contributors ianheggie-oaf

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Getting listing page /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch': 404 => Net::HTTPNotFound for https://rockingham.wa.gov.au/town-planning-advertising-and-submissions -- unhandled response (Mechanize::ResponseCodeError) from /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:1012:in `response_redirect' from /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:325:in `fetch' from /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' from scraper.rb:133:in `block in run' from scraper.rb:30:in `throttle_block' from scraper.rb:131:in `run' from scraper.rb:195:in `<main>'

Data

Downloaded 247 times by dcalde mlandauer Manasaanand guptara123

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (12 KB) Use the API

rows 0 / 0

Statistics

Average successful run time: half a minute

Total run time: 29 minutes

Total cpu time used: 1 minute

Total disk space used: 41.8 KB

History

  • Auto ran revision 9e6f9e4a and failed .
    nothing changed in the database
  • Auto ran revision 9e6f9e4a and failed .
    nothing changed in the database
  • Auto ran revision 9e6f9e4a and failed .
    nothing changed in the database
  • Auto ran revision 9e6f9e4a and failed .
    nothing changed in the database
  • Auto ran revision 9e6f9e4a and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

rockingham / scraper.rb