StashProperty / multiple_masterview

Scrapes multiple authorities that have a masterview system


This is a scraper that runs on Morph. To get started see the documentation

To run the scraper

bundle exec ruby scraper.rb

To run the tests

bundle exec rake

To run style and coding checks

bundle exec rubocop

What a Masterview website looks like

Sign up

(To update this screenshot run bundle exec rake screenshots)

Contributors mlandauer njenkins ianheggie-oaf dcalde

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... Injecting scraper and running... Default EXPECT_BAD: --- :bogan: https://github.com/planningalerts-scrapers/issues/issues/935 - Council website bad, does not publish Scraping authorities: albury, ballina, bega_valley, bogan, broken_hill, bundaberg, burwood, byron, cessnock, dubbo, fairfield, griffith, gunnedah, gympie, lismore, maranoa, muswellbrook, port_macquarie_hastings, port_stephens, shoalhaven, singleton, strathfield, upper_hunter Collecting feed data for albury, attempt: 1... Collecting feed data for ballina, attempt: 1... albury: ERROR: Failed to open TCP connection to eservice.alburycity.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record ballina - 2 Hibiscus Ave, Ballina NSW 2478 Saving record ballina - 3/87 Crane St, Ballina NSW 2478 Saving record ballina - 221 Hutley Dr, Lennox Head NSW 2478 Saving record ballina - 186 Broken Head Rd, Newrybar NSW 2479 Saving record ballina - 78 Teakwood Dr, Alstonville NSW 2477 Saving record ballina - 19 Gibbon St, Lennox Head NSW 2478 Saving record ballina - 1/69 Norton St, Ballina NSW 2478 Saving record ballina - 19 Cedar St, Wardell NSW 2477 Saving record ballina - 20 Wave Break Cct, Skennars Head NSW 2478 Saving record ballina - 158 Ross La, Kinvara NSW 2478 Saving record ballina - 1/8 Sunray Cr, Lennox Head NSW 2478 Saving record ballina - 11 Mosman Ch, Lennox Head NSW 2478 Saving record ballina - 16 Parkland Dr, Alstonville NSW 2477 Saving record ballina - 709 Fernleigh Rd, Brooklet NSW 2479 Saving record ballina - 49 Stoneyhurst Dr, Lennox Head NSW 2478 Saving record ballina - 20 Cedar Cr, East Ballina NSW 2478 Saving record ballina - 3 Albert Pl, Alstonville NSW 2477 Saving record ballina - 1 Mahogany Pl, East Ballina NSW 2478 Saving record ballina - 31 Lynette Cct, Wollongbar NSW 2477 Saving record ballina - 6 Short St, East Ballina NSW 2478 Saving record ballina - 66 Hillcrest Dr, Tintenbar NSW 2478 Saving record ballina - 43 Tuckombil La, Tuckombil NSW 2477 Saving record ballina - 131 Moylans La, Empire Vale NSW 2478 Saving record ballina - 3 Shoalhaven St, Alstonville NSW 2477 Saving record ballina - 35 Carroll Ave, Skennars Head NSW 2478 Saving record ballina - 1 Bentinck St, Ballina NSW 2478 Saving record ballina - 90 Ballina St, Lennox Head NSW 2478 Saving record ballina - 28 Windrow Ave, Cumbalum NSW 2478 Saving record ballina - 164 North Creek Rd, Lennox Head NSW 2478 Saving record ballina - 46 Amber Dr, Lennox Head NSW 2478 Saving record ballina - 34 Fig Tree Hill Dr, Lennox Head NSW 2478 Saving record ballina - 1 Seabreeze Pl, Lennox Head NSW 2478 Saving record ballina - 62 Forest Rd, Uralba NSW 2477 Saving record ballina - 39 Avalon Ave, Wollongbar NSW 2477 Saving record ballina - 72 Dolphin Dr, West Ballina NSW 2478 Saving record ballina - 50 Seaswell Cr, Lennox Head NSW 2478 Saving record ballina - 41 Bruxner Hwy, West Ballina NSW 2478 Saving record ballina - 104 Riverside Dr, West Ballina NSW 2478 Saving record ballina - 50 Seaswell Cr, Lennox Head NSW 2478 Saving record ballina - 6/33 Redford Dr, Skennars Head NSW 2478 Saving record ballina - 1/112 Eyles Dr, East Ballina NSW 2478 Saving record ballina - 43 Snapper Dr, Lennox Head NSW 2478 Saving record ballina - 3 Brunswick St, Ballina NSW 2478 Saving record ballina - 117 Old Byron Bay Rd, Newrybar NSW 2479 Saving record ballina - 2/81 Tamar St, Ballina NSW 2478 Saving record ballina - 69 Southern Cross Dr, Ballina NSW 2478 Saving record ballina - 3 Admiral Vsta, Lennox Head NSW 2478 Saving record ballina - 195 Rifle Range Rd, Alstonville NSW 2477 Saving record ballina - 3 Bugden Ave, Alstonville NSW 2477 Saving record ballina - 1/75A Habitat Wy, Lennox Head NSW 2478 Saving record ballina - 1/24 Barlows Rd, West Ballina NSW 2478 Saving record ballina - 2 Brooklet Rd, Newrybar NSW 2479 Saving record ballina - 75 Saratoga Dr, Lennox Head NSW 2478 Saving record ballina - 46 Gibbon St, Lennox Head NSW 2478 Saving record ballina - 67 North Creek Rd, Lennox Head NSW 2478 Saving record ballina - 239 Teven Rd, Teven NSW 2478 Saving record ballina - 21 Mcleish Rd, Kinvara NSW 2478 Saving record ballina - Ascot Rd, Ballina NSW 2478 Saving record ballina - 11 Stewart St, Lennox Head NSW 2478 Saving record ballina - 27 Hickey St, Ballina NSW 2478 Saving record ballina - 6 Hyde Park Rd, Wollongbar NSW 2477 Saving record ballina - 21 Adele St, Alstonville NSW 2477 Saving record ballina - 177 Humpty Back Rd, Mcleans Ridges NSW 2480 Saving record ballina - 14 Seaside Ave, Skennars Head NSW 2478 Saving record ballina - 18 Montwood Dr, Lennox Head NSW 2478 Saving record ballina - 33 Avalon Ave, Wollongbar NSW 2477 Saving record ballina - 18 Curlew Ct, East Ballina NSW 2478 Saving record ballina - 12-16 Fig Tree Hill Dr, Lennox Head NSW 2478 Saving record ballina - 93 Weis La, Rous NSW 2477 Saving record ballina - 2/19 Beryl Pl, Lennox Head NSW 2478 Saving record ballina - 10 Henry Philp Ave, Ballina NSW 2478 Saving record ballina - 426 Old Byron Bay Rd, Newrybar NSW 2479 Saving record ballina - 13 Admiral Vsta, Lennox Head NSW 2478 Saving record ballina - 14 Anchorage Ave, Lennox Head NSW 2478 Saving record ballina - 60 Gahans Rd, Meerschaum Vale NSW 2477 Saving record ballina - 2/8 Sunrise Cr, Lennox Head NSW 2478 Saving record ballina - 21 Lynette Cct, Wollongbar NSW 2477 Saving record ballina - 5 Racecourse Rd, Ballina NSW 2478 Saving record ballina - 84 Kerr St, Ballina NSW 2478 Saving record ballina - 1/8 Fawcett St, Ballina NSW 2478 Saving record ballina - 100 Fredericks La, Tintenbar NSW 2478 Saving record ballina - 1/23 Diversity Ave, Cumbalum NSW 2478 Saving record ballina - 1/14 Ross St, Ballina NSW 2478 Saving record ballina - 102 Riverside Dr, West Ballina NSW 2478 Saving record ballina - 58 Barrett Dr, Lennox Head NSW 2478 Saving record ballina - 32 Boeing Ave, Ballina NSW 2478 Saving record ballina - 32 Boeing Ave, Ballina NSW 2478 Saving record ballina - 32 Boeing Ave, Ballina NSW 2478 Saving record ballina - 5 Ingrams Rd, Cumbalum NSW 2478 Saving record ballina - 356 Old Byron Bay Rd, Newrybar NSW 2479 Saving record ballina - 44 Bangalow Rd, Ballina NSW 2478 Saving record ballina - 781 Fernleigh Rd, Brooklet NSW 2479 Saving record ballina - 18 Habitat Wy, Lennox Head NSW 2478 Saving record ballina - 26 Acreage St, Cumbalum NSW 2478 Saving record ballina - 28 Windrow Ave, Cumbalum NSW 2478 Saving record ballina - 35 Fitzroy St, Wardell NSW 2477 Saving record ballina - 30 Rifle Range Rd, Wollongbar NSW 2477 Saving record ballina - 32 Delva Rd, Wollongbar NSW 2477 Saving record ballina - 26 Batrel Rd, Mcleans Ridges NSW 2480 Saving record ballina - 29 Snapper Dr, Lennox Head NSW 2478 /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record ballina - 1 Crane St, Ballina NSW 2478 Saving record ballina - 4 Blossom Pl, Wardell NSW 2477 Saving record ballina - 3/19 Cooloola Ave, Lennox Head NSW 2478 Saving record ballina - 58 Pacific Heights Dr, Cumbalum NSW 2478 Saving record ballina - 31 Delva Rd, Wollongbar NSW 2477 Saving record ballina - 12 Delva Rd, Wollongbar NSW 2477 Saving record ballina - 4 Blossom Pl, Wardell NSW 2477 Saving record ballina - 298 Rous Rd, Rous NSW 2477 Saving record ballina - 15 Delva Rd, Wollongbar NSW 2477 Saving record ballina - 5 Wejuba Gdns, Ballina NSW 2478 Saving record ballina - 47 Avalon Ave, Wollongbar NSW 2477 Saving record ballina - 5 Cove Ave, Skennars Head NSW 2478 Collecting feed data for bega_valley, attempt: 1... Collecting feed data for bogan, attempt: 1... bega_valley: ERROR: Failed to open TCP connection to datracker.begavalley.nsw.gov.au:443 (getaddrinfo: Name or service not known) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' Collecting feed data for broken_hill, attempt: 1... bogan: ERROR: Failed to open TCP connection to datracker.bogan.nsw.gov.au:81 (getaddrinfo: Name or service not known) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record broken_hill - 105 Nicholls St, Broken Hill NSW 2880 Saving record broken_hill - 408 Williams St, Broken Hill NSW 2880 Saving record broken_hill - 105 Nicholls St, Broken Hill NSW 2880 Saving record broken_hill - 74 Wickes St, Broken Hill NSW 2880 Saving record broken_hill - 131 Wills St, Broken Hill NSW 2880 Saving record broken_hill - 235 Hebbard St, Broken Hill NSW 2880 Saving record broken_hill - 8 Crystal St, Broken Hill NSW 2880 Saving record broken_hill - 560 Argent St, Broken Hill NSW 2880 Saving record broken_hill - 217 Oxide St, Broken Hill NSW 2880 Saving record broken_hill - 40 Cobalt St, Broken Hill NSW 2880 Saving record broken_hill - 26 Allendale St, Broken Hill NSW 2880 Saving record broken_hill - 224 Duff St, Broken Hill NSW 2880 Saving record broken_hill - 58-70 Oxide St, Broken Hill NSW 2880 Saving record broken_hill - 154-156 Patton St, Broken Hill NSW 2880 Collecting feed data for bundaberg, attempt: 1... Collecting feed data for burwood, attempt: 1... bundaberg: ERROR: 403 => Net::HTTPForbidden for https://da.bundaberg.qld.gov.au/ -- unhandled response /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' Collecting feed data for byron, attempt: 1... burwood: ERROR: Failed to open TCP connection to datracker.burwood.nsw.gov.au:443 (getaddrinfo: Name or service not known) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 18 Armstrong Ln, Broken Head 2481 NSW Saving record byron - 1 Gittoes Ln, Possum Creek 2479 NSW Saving record byron - 3 Raftons Rd, Bangalow 2479 NSW Saving record byron - 104 Alcorn St, Suffolk Park 2481 NSW Saving record byron - 239 Tyagarah Rd, Myocum 2481 NSW Saving record byron - 41 Kingsley Ln, Byron Bay 2481 NSW Saving record byron - 1 Brownell Drv, Byron Bay 2481 NSW Saving record byron - 39 Picadilly Hill Rd, Coopers Shoot 2481 NSW Saving record byron - 67 Argyle St, Mullumbimby 2482 NSW Saving record byron - 42 Tuckeroo Ave, Mullumbimby 2482 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 124 Alcorn St, Suffolk Park 2481 NSW Saving record byron - 15 Clifford St, Suffolk Park 2481 NSW Saving record byron - 101 Bangalow Rd, Byron Bay 2481 NSW Saving record byron - 90 Lacebark Ln, Eureka 2480 NSW Saving record byron - 6 Coral Crt, Byron Bay 2481 NSW Saving record byron - 142 Bangalow Rd, Byron Bay 2481 NSW Saving record byron - 17/111 Fowlers Ln, Bangalow 2479 NSW Saving record byron - 140 Quarry Ln, Ewingsdale 2481 NSW Saving record byron - 44 Benloro Ln, Myocum 2481 NSW Saving record byron - 12 Kallaroo Cct, Ocean Shores 2483 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 464 Friday Hut Rd, Possum Creek 2479 NSW Saving record byron - 62 Stuart St, Mullumbimby 2482 NSW Saving record byron - 18 Jarrah Crs, Ocean Shores 2483 NSW Saving record byron - 1 Omega Cct, Brunswick Heads 2483 NSW Saving record byron - 51 Coogera Cct, Suffolk Park 2481 NSW Saving record byron - 216 Friday Hut Rd, Possum Creek 2479 NSW Saving record byron - 325A Eureka Rd, Eureka 2480 NSW Saving record byron - 1/164 Alcorn St, Suffolk Park 2481 NSW Saving record byron - 15 Grevillea St, Byron Bay 2481 NSW Saving record byron - 124 Mcgoughans Ln, Mullumbimby 2482 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 3 Cedarvale Rd, Bangalow 2479 NSW Saving record byron - 8 Meadows Cl, Bangalow 2479 NSW Saving record byron - 67 Bougainvillea Drv, Federal 2480 NSW Saving record byron - 1 Cassia Crt, Mullumbimby 2482 NSW Saving record byron - 7 Julian Rocks Drv, Byron Bay 2481 NSW Saving record byron - 28 Mia Crt, Ocean Shores 2483 NSW Saving record byron - 87 Myrtle Rd, Coorabell 2479 NSW Saving record byron - 25 Childe St, Byron Bay 2481 NSW Saving record byron - 8 Goshawk Ln, Mullumbimby 2482 NSW Saving record byron - 3 Cedarvale Rd, Bangalow 2479 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 25 New City Rd, Mullumbimby 2482 NSW Saving record byron - 10 Royal Ave, South Golden Beach 2483 NSW Saving record byron - 26 Baileys Rd, Coopers Shoot 2481 NSW Saving record byron - 119 Jonson St, Byron Bay 2481 NSW Saving record byron - 38 Tweed St, Brunswick Heads 2483 NSW Saving record byron - 29 Shirley St, Byron Bay 2481 NSW Saving record byron - 8 Blueberry Crt, Byron Bay 2481 NSW Saving record byron - 4 Strand Ave, New Brighton 2483 NSW Saving record byron - 6 Byron Bay Rd, Bangalow 2479 NSW Saving record byron - 4 Browning St, Byron Bay 2481 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 20 Azalea St, Mullumbimby 2482 NSW Saving record byron - 11 Terrace St, New Brighton 2483 NSW Saving record byron - 110 Jonson St, Byron Bay 2481 NSW Saving record byron - 1/77A Bangalow Rd, Byron Bay 2481 NSW Saving record byron - 102 Jonson St, Byron Bay 2481 NSW Saving record byron - 8 Newberry Pde, Brunswick Heads 2483 NSW Saving record byron - 5 Tasha Cl, Ewingsdale 2481 NSW Saving record byron - 6 Boronia Pl, Byron Bay 2481 NSW Saving record byron - 15 Grevillea St, Byron Bay 2481 NSW Saving record byron - 15 Beachcomber Drv, Byron Bay 2481 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 119 Jonson St, Byron Bay 2481 NSW Saving record byron - 3 Argyle St, Mullumbimby 2482 NSW Saving record byron - 43 Harbour Way, Brunswick Heads 2483 NSW Saving record byron - 25 Massinger St, Byron Bay 2481 NSW Saving record byron - 124 Lacebark Ln, Eureka 2480 NSW Saving record byron - 15 Mcgettigans Ln, Ewingsdale 2481 NSW Saving record byron - 9/231 Fowlers Ln, Bangalow 2479 NSW Saving record byron - 501A Booyong Rd, Booyong 2480 NSW Saving record byron - 58 Ann St, Mullumbimby 2482 NSW Saving record byron - 13 Jullum Cct, Suffolk Park 2481 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 2 Kulgun Crt, Ocean Shores 2483 NSW Saving record byron - 65 Hyrama Crs, Brunswick Heads 2483 NSW Saving record byron - 82 Cooks Rd, Nashua 2479 NSW Saving record byron - 6 Border St, Byron Bay 2481 NSW Saving record byron - 174 Alcorn St, Suffolk Park 2481 NSW Saving record byron - 11 Harvest Bvd, Byron Bay 2481 NSW Saving record byron - 16 Jacaranda Drv, Byron Bay 2481 NSW Saving record byron - 50 Corkwood Crs, Suffolk Park 2481 NSW Saving record byron - 9 Marvell St, Byron Bay 2481 NSW Saving record byron - 9 Pacific St, New Brighton 2483 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 18 Jonson St, Byron Bay 2481 NSW Saving record byron - 2/140-142 Jonson St, Byron Bay 2481 NSW Saving record byron - 7 Burns St, Byron Bay 2481 NSW Saving record byron - 417 Middle Pocket Rd, Middle Pocket 2483 NSW Saving record byron - 305 Coolamon Scenic Drv, Coorabell 2479 NSW Saving record byron - 2/4 Jonson St, Byron Bay 2481 NSW Saving record byron - 5 Raftons Rd, Bangalow 2479 NSW Saving record byron - 5 Station St, Mullumbimby 2482 NSW Saving record byron - 56 Shirley Ln, Byron Bay 2481 NSW Saving record byron - 46 Kallaroo Cct, Ocean Shores 2483 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 17 Stuart St, Mullumbimby 2482 NSW Saving record byron - 84 Harbour Way, Brunswick Heads 2483 NSW Saving record byron - 152 Alcorn St, Suffolk Park 2481 NSW Saving record byron - 53 Massinger St, Byron Bay 2481 NSW Saving record byron - 50-54 Bayshore Drv, Byron Bay 2481 NSW Saving record byron - 93 Balemo Drv, Ocean Shores 2483 NSW Saving record byron - 18 Burns St, Byron Bay 2481 NSW Saving record byron - 14 Cooper St, Byron Bay 2481 NSW Saving record byron - 8 Angus Pl, Ewingsdale 2481 NSW Saving record byron - 37 Wiley Rd, Bangalow 2479 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 39 Armstrong St, Suffolk Park 2481 NSW Saving record byron - 45 Carlyle Ln, Byron Bay 2481 NSW Saving record byron - 119 Jonson St, Byron Bay 2481 NSW Saving record byron - 104 Tuckeroo Ave, Mullumbimby 2482 NSW Saving record byron - 1472 Hinterland Way, Bangalow 2479 NSW Saving record byron - 33 Parrot Tree Pl, Bangalow 2479 NSW Saving record byron - 88 Centennial Cct, Byron Bay 2481 NSW Saving record byron - 6 Grevillea Ave, Mullumbimby 2482 NSW Saving record byron - 13 Redgum Pl, Suffolk Park 2481 NSW Saving record byron - 199 Myocum Rd, Ewingsdale 2481 NSW /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record byron - 1025 Friday Hut Rd, Binna Burra 2479 NSW Saving record byron - 11 Fletcher St, Byron Bay 2481 NSW Saving record byron - 14 Jubilee Ave, Mullumbimby 2482 NSW Saving record byron - 39 Narooma Drv, Ocean Shores 2483 NSW Saving record byron - 2/24 River St, New Brighton 2483 NSW Collecting feed data for cessnock, attempt: 1... Collecting feed data for dubbo, attempt: 1... /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Collecting feed data for fairfield, attempt: 1... dubbo: ERROR: 403 => Net::HTTPForbidden for https://planning.dubbo.nsw.gov.au// -- unhandled response /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' Collecting feed data for griffith, attempt: 1... fairfield: ERROR: Failed to open TCP connection to openaccess.fairfieldcity.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:148:in `scrape_url' /app/lib/masterview_scraper.rb:55:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' Collecting feed data for gunnedah, attempt: 1... griffith: ERROR: Failed to open TCP connection to datracking.griffith.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record gunnedah - 7 Yarraandoo Cl, Gunnedah NSW 2380 Saving record gunnedah - 8679 Oxley Hwy, Gunnedah NSW 2380 Saving record gunnedah - 47 Wentworth St, Gunnedah NSW 2380 Saving record gunnedah - 22 Elm St, Gunnedah NSW 2380 Saving record gunnedah - 181-183 Conadilly St, Gunnedah NSW 2380 Saving record gunnedah - 27 Elm St, Gunnedah NSW 2380 Saving record gunnedah - 8 Mcdermott Pl, Gunnedah NSW 2380 Saving record gunnedah - 58 Kurrajong Rd, Gunnedah NSW 2380 Saving record gunnedah - 55 Beulah St, Gunnedah NSW 2380 Saving record gunnedah - 648 Wandobah Rd, Gunnedah NSW 2380 Saving record gunnedah - 25 Elm St, Gunnedah NSW 2380 Collecting feed data for gympie, attempt: 1... /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record gympie - 123 East Deep Creek Rd, Monkland 4570 QLD Saving record gympie - 42 Zerner Rd, Pie Creek 4570 QLD Saving record gympie - 1 Rosslyn Rd, Southside 4570 QLD Saving record gympie - 2 Clematis St, Gympie 4570 QLD Saving record gympie - 77 Little Widgee Rd, Widgee 4570 QLD Saving record gympie - 1371 Wide Bay Hwy, Lower Wonga 4570 QLD Saving record gympie - 440 Diamondfield Rd, Kandanga 4570 QLD Saving record gympie - 440 Diamondfield Rd, Kandanga 4570 QLD Saving record gympie - 2 Stanley St, Gympie 4570 QLD Saving record gympie - 12 Bream St, Tin Can Bay 4580 QLD Saving record gympie - 49 Wises Rd, Gympie 4570 QLD Saving record gympie - 5 Llewellyn St, Goomeri 4601 QLD Saving record gympie - 29 Mellor St, Gympie 4570 QLD Saving record gympie - 29 Mellor St, Gympie 4570 QLD Saving record gympie - 16 Sorrel St, Gympie 4570 QLD Saving record gympie - 219 Nash Rd, Tamaree 4570 QLD Saving record gympie - 104 Groundwater Rd, Southside 4570 QLD Saving record gympie - 383 Kandanga Creek Rd, Kandanga 4570 QLD Saving record gympie - 20 Lobwein Rd, Kybong 4570 QLD Saving record gympie - 2319 Kinbombi Rd, Manumbar 4601 QLD Saving record gympie - 146 Reilly Rd, Woolooga 4570 QLD Saving record gympie - 55 Gambling Rd, Veteran 4570 QLD Saving record gympie - 10A Tucker St, Gympie 4570 QLD Collecting feed data for lismore, attempt: 1... Collecting feed data for maranoa, attempt: 1... lismore: ERROR: 403 => Net::HTTPForbidden for https://www.lismore-nsw.altitudelg.com/e-services/ -- unhandled response /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:1012:in `response_redirect' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:325:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record maranoa - 31 Duke St, Roma 4455 QLD Saving record maranoa - Origin Lighthouse Access Rd, Durham Downs 4454 QLD Saving record maranoa - Industrial Estate Rd, Mitchell 4465 QLD Saving record maranoa - Industrial Estate Rd, Mitchell 4465 QLD Saving record maranoa - 91A Burrowes St, Surat 4417 QLD Saving record maranoa - 120 Winchester St, Mitchell 4465 QLD Saving record maranoa - Bollon Rd, Womalilla 4465 QLD Saving record maranoa - 450 Pickanjinnie North Rd, Pickanjinnie 4428 QLD PROBLEM LOADING PAGE http://pdonline.maranoa.qld.gov.au/Application/ApplicationDetails/022.2024.00021278.001 Saving record maranoa - 27-29 Lovell St, Roma 4455 QLD Saving record maranoa - Wormwell Dr, Roma 4455 QLD Saving record maranoa - 79 Mcdowall St, Roma 4455 QLD Saving record maranoa - 76 Bowen St, Roma 4455 QLD Saving record maranoa - 2859 Mt Saltbush Rd, Mooga 4455 QLD Saving record maranoa - 85 Taylor St, Roma 4455 QLD Saving record maranoa - 54 Howard St, Roma 4455 QLD Saving record maranoa - 63-65 Hawthorne St, Roma 4455 QLD Saving record maranoa - 40540 Warrego Hwy, Jackson 4426 QLD Saving record maranoa - 44670 Warrego Hwy, Roma 4455 QLD Saving record maranoa - 291 Donnybrook Rd, Mount Bindango 4455 QLD Saving record maranoa - Bindango Rd, Hodgson 4455 QLD Saving record maranoa - 8 Whip St, Roma 4455 QLD Saving record maranoa - 38 Hutton St, Injune 4454 QLD Saving record maranoa - 8 Whip St, Roma 4455 QLD Saving record maranoa - 31 Duke St, Roma 4455 QLD Saving record maranoa - 16-18 Denton Ct, Injune 4454 QLD Collecting feed data for muswellbrook, attempt: 1... Collecting feed data for port_macquarie_hastings, attempt: 1... /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Collecting feed data for port_stephens, attempt: 1... port_macquarie_hastings: ERROR: Failed to open TCP connection to datracker.pmhc.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' Collecting feed data for shoalhaven, attempt: 1... port_stephens: ERROR: Failed to open TCP connection to datracker.portstephens.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' Collecting feed data for singleton, attempt: 1... shoalhaven: ERROR: Couldn't find table element on https://www.shoalhaven.nsw.gov.au/Home /app/lib/masterview_scraper/pages/index.rb:41:in `scrape' /app/lib/masterview_scraper.rb:160:in `scrape_url' /app/lib/masterview_scraper.rb:55:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' Collecting feed data for strathfield, attempt: 1... singleton: ERROR: Failed to open TCP connection to datracker.singleton.nsw.gov.au:444 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/pluggable_parsers.rb:107:in `new': MIME::Type.MIME::Type.new when called with a String is deprecated. Saving record strathfield - 17 Torrington Rd, Strathfield 2135 NSW Saving record strathfield - 17 Water St, Strathfield South 2136 NSW Saving record strathfield - 31 Fitzgerald Cres, Strathfield 2135 NSW Saving record strathfield - 63 High St, Strathfield 2135 NSW Saving record strathfield - 28 Boden Ave, Strathfield 2135 NSW Saving record strathfield - 21 Parramatta Rd, Homebush 2140 NSW Saving record strathfield - 140 Barker Rd, Strathfield 2135 NSW Saving record strathfield - 75 Chalmers Rd, Strathfield 2135 NSW Saving record strathfield - 15 Albert Rd, Strathfield 2135 NSW Saving record strathfield - 90 Abbotsford Rd, Homebush 2140 NSW Saving record strathfield - 61-65 Roberts Rd, Greenacre 2190 NSW Saving record strathfield - 16 Alviston St, Strathfield 2135 NSW Saving record strathfield - 11/161 Arthur St, Homebush West 2140 NSW Saving record strathfield - 11-23 The Boulevarde, Strathfield 2135 NSW Saving record strathfield - 30 Hydebrae St, Strathfield 2135 NSW Saving record strathfield - 67 Churchill Ave, Strathfield 2135 NSW Saving record strathfield - 9 Morgan Pl, Strathfield 2135 NSW Saving record strathfield - 54 Cave Rd, Strathfield 2135 NSW Saving record strathfield - 5 Wakeford Rd, Strathfield 2135 NSW Saving record strathfield - 5 Victoria St, Strathfield 2135 NSW Saving record strathfield - 11 Merley Rd, Strathfield 2135 NSW Saving record strathfield - 4 Sunlea Cres, Strathfield South 2136 NSW Saving record strathfield - 201 Parramatta Rd, Homebush West 2140 NSW Saving record strathfield - 4 Marion St, Strathfield 2135 NSW Saving record strathfield - 53-57 Punchbowl Rd, Strathfield South 2136 NSW Saving record strathfield - 12 Gregory St, Strathfield South 2136 NSW Saving record strathfield - 28 Mackenzie St, Homebush 2140 NSW Saving record strathfield - 53 Cosgrove Rd, Strathfield South 2136 NSW Saving record strathfield - 19 Fitzgerald Cres, Strathfield 2135 NSW Saving record strathfield - 21 Hydebrae St, Strathfield 2135 NSW Saving record strathfield - 38 Rickard Rd, Strathfield 2135 NSW Saving record strathfield - 57 Wentworth St, Greenacre 2190 NSW Saving record strathfield - 40 Pemberton St, Strathfield 2135 NSW Saving record strathfield - 1 Macarthur Ave, Strathfield 2135 NSW Saving record strathfield - 65-67 Parramatta Rd, Homebush 2140 NSW Saving record strathfield - 103 Wallis Ave, Strathfield 2135 NSW Saving record strathfield - 91 Burlington Rd, Homebush 2140 NSW Saving record strathfield - 127 Madeline St, Strathfield South 2136 NSW Saving record strathfield - 179 Albert Rd, Strathfield 2135 NSW Collecting feed data for upper_hunter, attempt: 1... upper_hunter: ERROR: 404 => Net::HTTPNotFound for https://onlineservices.upperhunter.nsw.gov.au/Application/GetApplications -- unhandled response /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:1012:in `response_redirect' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:325:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:1324:in `post_form' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:537:in `post' /app/lib/masterview_scraper/get_applications_api.rb:45:in `scrape_page' /app/lib/masterview_scraper/get_applications_api.rb:18:in `scrape' /app/lib/masterview_scraper.rb:91:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:43:in `run' scraper.rb:83:in `<main>' *************************************************** Now retrying authorities which earlier had failures albury, bega_valley, bogan, bundaberg, burwood, dubbo, fairfield, griffith, lismore, port_macquarie_hastings, port_stephens, shoalhaven, singleton, upper_hunter *************************************************** Collecting feed data for albury, attempt: 2... Collecting feed data for bega_valley, attempt: 2... albury: ERROR: Failed to open TCP connection to eservice.alburycity.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for bogan, attempt: 2... bega_valley: ERROR: Failed to open TCP connection to datracker.begavalley.nsw.gov.au:443 (getaddrinfo: Name or service not known) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for bundaberg, attempt: 2... bogan: ERROR: Failed to open TCP connection to datracker.bogan.nsw.gov.au:81 (getaddrinfo: Name or service not known) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for burwood, attempt: 2... bundaberg: ERROR: 403 => Net::HTTPForbidden for https://da.bundaberg.qld.gov.au/ -- unhandled response /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for dubbo, attempt: 2... burwood: ERROR: Failed to open TCP connection to datracker.burwood.nsw.gov.au:443 (getaddrinfo: Name or service not known) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for fairfield, attempt: 2... dubbo: ERROR: 403 => Net::HTTPForbidden for https://planning.dubbo.nsw.gov.au// -- unhandled response /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for griffith, attempt: 2... fairfield: ERROR: Failed to open TCP connection to openaccess.fairfieldcity.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:148:in `scrape_url' /app/lib/masterview_scraper.rb:55:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for lismore, attempt: 2... griffith: ERROR: Failed to open TCP connection to datracking.griffith.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for port_macquarie_hastings, attempt: 2... lismore: ERROR: 403 => Net::HTTPForbidden for https://www.lismore-nsw.altitudelg.com/e-services/ -- unhandled response /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:1012:in `response_redirect' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:325:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for port_stephens, attempt: 2... port_macquarie_hastings: ERROR: Failed to open TCP connection to datracker.pmhc.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for shoalhaven, attempt: 2... port_stephens: ERROR: Failed to open TCP connection to datracker.portstephens.nsw.gov.au:443 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for singleton, attempt: 2... shoalhaven: ERROR: Couldn't find table element on https://www.shoalhaven.nsw.gov.au/Home /app/lib/masterview_scraper/pages/index.rb:41:in `scrape' /app/lib/masterview_scraper.rb:160:in `scrape_url' /app/lib/masterview_scraper.rb:55:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' Collecting feed data for upper_hunter, attempt: 2... singleton: ERROR: Failed to open TCP connection to datracker.singleton.nsw.gov.au:444 (execution expired) /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `block in connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/timeout.rb:196:in `timeout' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1269:in `connect' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1248:in `do_start' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1243:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:712:in `start' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:641:in `connection_for' /app/vendor/bundle/ruby/3.2.0/gems/net-http-persistent-4.0.6/lib/net/http/persistent.rb:942:in `request' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:284:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' upper_hunter: ERROR: 404 => Net::HTTPNotFound for https://onlineservices.upperhunter.nsw.gov.au/Application/GetApplications -- unhandled response /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:1012:in `response_redirect' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:325:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:1324:in `post_form' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:537:in `post' /app/lib/masterview_scraper/get_applications_api.rb:45:in `scrape_page' /app/lib/masterview_scraper/get_applications_api.rb:18:in `scrape' /app/lib/masterview_scraper.rb:91:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape' scraper.rb:17:in `each' scraper.rb:17:in `scrape' scraper.rb:57:in `run' scraper.rb:83:in `<main>' MORPH_EXPECT_BAD=bogan /app/vendor/bundle/ruby/3.2.0/gems/scraper_utils-0.8.2/lib/scraper_utils/log_utils.rb:192:in `report_on_results': ERROR: Unexpected errors in: albury,bega_valley,bundaberg,burwood,dubbo,fairfield,griffith,lismore,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter (Add to MORPH_EXPECT_BAD?) (RuntimeError) albury: Net::OpenTimeout - Failed to open TCP connection to eservice.alburycity.nsw.gov.au:443 (execution expired) Scraping Summary: Authority OK Bad Exception -------------------- ------ ------ -------------------------------------------------- albury 0 0 Net::OpenTimeout - Failed to open TCP connection to eservice.alburycity.nsw.gov.au:443 (execution expired) [lib/masterview_scraper.rb:85:in `scrape_api_period'] ballina 112 0 - bega_valley 0 0 SocketError - Failed to open TCP connection to datracker.begavalley.nsw.gov.au:443 (getaddrinfo: Name or service not known) [lib/masterview_scraper.rb:85:in `scrape_api_period'] bogan 0 0 [EXPECT BAD] SocketError - Failed to open TCP connection to datracker.bogan.nsw.gov.au:81 (getaddrinfo: Name or service not known) [lib/masterview_scraper.rb:85:in `scrape_api_period'] broken_hill 14 0 - bega_valley: SocketError - Failed to open TCP connection to datracker.begavalley.nsw.gov.au:443 (getaddrinfo: Name or service not known) bundaberg: Mechanize::ResponseCodeError - 403 => Net::HTTPForbidden for https://da.bundaberg.qld.gov.au/ -- unhandled response bundaberg 0 0 Mechanize::ResponseCodeError - 403 => Net::HTTPForbidden for https://da.bundaberg.qld.gov.au/ -- unhandled response [lib/masterview_scraper.rb:85:in `scrape_api_period'] burwood: SocketError - Failed to open TCP connection to datracker.burwood.nsw.gov.au:443 (getaddrinfo: Name or service not known) burwood 0 0 SocketError - Failed to open TCP connection to datracker.burwood.nsw.gov.au:443 (getaddrinfo: Name or service not known) [lib/masterview_scraper.rb:85:in `scrape_api_period'] dubbo: Mechanize::ResponseCodeError - 403 => Net::HTTPForbidden for https://planning.dubbo.nsw.gov.au// -- unhandled response byron 115 0 - cessnock 0 0 - fairfield: Net::OpenTimeout - Failed to open TCP connection to openaccess.fairfieldcity.nsw.gov.au:443 (execution expired) griffith: Net::OpenTimeout - Failed to open TCP connection to datracking.griffith.nsw.gov.au:443 (execution expired) dubbo 0 0 Mechanize::ResponseCodeError - 403 => Net::HTTPForbidden for https://planning.dubbo.nsw.gov.au// -- unhandled response [lib/masterview_scraper.rb:85:in `scrape_api_period'] lismore: Mechanize::ResponseCodeError - 403 => Net::HTTPForbidden for https://www.lismore-nsw.altitudelg.com/e-services/ -- unhandled response fairfield 0 0 Net::OpenTimeout - Failed to open TCP connection to openaccess.fairfieldcity.nsw.gov.au:443 (execution expired) [lib/masterview_scraper.rb:148:in `scrape_url'] port_macquarie_hastings: Net::OpenTimeout - Failed to open TCP connection to datracker.pmhc.nsw.gov.au:443 (execution expired) griffith 0 0 Net::OpenTimeout - Failed to open TCP connection to datracking.griffith.nsw.gov.au:443 (execution expired) [lib/masterview_scraper.rb:85:in `scrape_api_period'] port_stephens: Net::OpenTimeout - Failed to open TCP connection to datracker.portstephens.nsw.gov.au:443 (execution expired) gunnedah 11 0 - gympie 23 0 - shoalhaven: RuntimeError - Couldn't find table element on https://www.shoalhaven.nsw.gov.au/Home singleton: Net::OpenTimeout - Failed to open TCP connection to datracker.singleton.nsw.gov.au:444 (execution expired) lismore 0 0 Mechanize::ResponseCodeError - 403 => Net::HTTPForbidden for https://www.lismore-nsw.altitudelg.com/e-services/ -- unhandled response [lib/masterview_scraper.rb:85:in `scrape_api_period'] maranoa 25 0 - muswellbrook 0 0 - upper_hunter: Mechanize::ResponseCodeError - 404 => Net::HTTPNotFound for https://onlineservices.upperhunter.nsw.gov.au/Application/GetApplications -- unhandled response See earlier output for details from scraper.rb:67:in `run' from scraper.rb:83:in `<main>' port_macquarie_hastings 0 0 Net::OpenTimeout - Failed to open TCP connection to datracker.pmhc.nsw.gov.au:443 (execution expired) [lib/masterview_scraper.rb:85:in `scrape_api_period'] port_stephens 0 0 Net::OpenTimeout - Failed to open TCP connection to datracker.portstephens.nsw.gov.au:443 (execution expired) [lib/masterview_scraper.rb:85:in `scrape_api_period'] shoalhaven 0 0 RuntimeError - Couldn't find table element on https://www.shoalhaven.nsw.gov.au/Home [lib/masterview_scraper/pages/index.rb:41:in `scrape'] singleton 0 0 Net::OpenTimeout - Failed to open TCP connection to datracker.singleton.nsw.gov.au:444 (execution expired) [lib/masterview_scraper.rb:85:in `scrape_api_period'] strathfield 39 0 - upper_hunter 0 0 Mechanize::ResponseCodeError - 404 => Net::HTTPNotFound for https://onlineservices.upperhunter.nsw.gov.au/Application/GetApplications -- unhandled response [lib/masterview_scraper/get_applications_api.rb:45:in `scrape_page']

Data

Downloaded 169 times by dcalde tkrikke Joel-Tan_valocity chan0415 AndreM101 muzzie86 jamasalbertastash studiology repose-lecture

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (26 MB) Use the API

rows 10 / 64658

council_reference address description info_url date_scraped date_received authority_label
10-2021-16-1
2 Angus St, Forbes 2871 NSW
New Garages/Sheds - Industrial
2021-03-16
2021-02-22
forbes
10-2021-15-1
613 Wongajong Rd, Forbes 2871 NSW
New Single Dwelling
2021-03-16
2021-02-18
forbes
10-2021-13-1
1829 Lachlan Valley Wy, Jemalong 2871 NSW
New Single Dwelling
2021-03-16
2021-02-15
forbes
10-2021-14-1
131A Wambat St, Forbes 2871 NSW
New Ancillary Residential Development - Shed
2021-03-16
2021-02-16
forbes
10-2017-80-3
33-37 Sam St, Forbes 2871 NSW
New Commercial Building
2021-03-16
2021-03-03
forbes
10-2021-19-1
36A William St, Forbes 2871 NSW
Patio Cover
2021-03-16
2021-03-05
forbes
10-2021-20-1
2 Rae St, Forbes 2871 NSW
New Ancillary Residential Development-Shed/Carport
2021-03-16
2021-03-05
forbes
10-2021-18-1
12 Acacia Ave, Forbes 2871 NSW
New Ancillary Residential Development-Shed
2021-03-16
2021-03-02
forbes
10-2021-17-1
10 Thelma St, Forbes 2871 NSW
New Ancillary Residential Development - Shed
2021-03-16
2021-03-02
forbes
10-2021-1-1
Ooma St, Forbes 2871 NSW
Ancillary Residential Development - Awning
2021-03-16
2021-02-25
forbes

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (26 MB) Use the API

rows 10 / 1426

run_at attempt authority_label records_saved unprocessable_records status error_message error_class error_backtrace
2026-08-22T09:46:41+00:00
1
albury
0
0
failed
Failed to open TCP connection to eservice.alburycity.nsw.gov.au:443 (execution expired)
Net::OpenTimeout
/app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape'
2026-08-22T09:46:41+00:00
1
ballina
153
0
successful
2026-08-22T09:46:41+00:00
1
bega_valley
0
0
failed
Failed to open TCP connection to datracker.begavalley.nsw.gov.au:443 (getaddrinfo: Name or service not known)
SocketError
/app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape'
2026-08-22T09:46:41+00:00
1
bogan
0
0
failed
Failed to open TCP connection to datracker.bogan.nsw.gov.au:81 (getaddrinfo: Name or service not known)
SocketError
/app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape'
2026-08-22T09:46:41+00:00
1
broken_hill
10
0
successful
2026-08-22T09:46:41+00:00
1
bundaberg
0
0
failed
403 => Net::HTTPForbidden for https://da.bundaberg.qld.gov.au/ -- unhandled response
Mechanize::ResponseCodeError
/app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape'
2026-08-22T09:46:41+00:00
1
burwood
0
0
failed
Failed to open TCP connection to datracker.burwood.nsw.gov.au:443 (getaddrinfo: Name or service not known)
SocketError
/app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `initialize' /app/vendor/ruby-3.2.2/lib/ruby/3.2.0/net/http.rb:1271:in `open' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape'
2026-08-22T09:46:41+00:00
1
byron
117
0
successful
2026-08-22T09:46:41+00:00
1
cessnock
322
0
successful
2026-08-22T09:46:41+00:00
1
dubbo
0
0
failed
403 => Net::HTTPForbidden for https://planning.dubbo.nsw.gov.au// -- unhandled response
Mechanize::ResponseCodeError
/app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize/http/agent.rb:333:in `fetch' /app/vendor/bundle/ruby/3.2.0/gems/mechanize-2.8.5/lib/mechanize.rb:465:in `get' /app/lib/masterview_scraper.rb:85:in `scrape_api_period' /app/lib/masterview_scraper.rb:45:in `scrape_period' /app/lib/masterview_scraper.rb:19:in `scrape' scraper.rb:21:in `block in scrape'

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (26 MB) Use the API

rows 10 / 62

run_at attempt duration successful failed interrupted successful_count interrupted_count failed_count public_ip
2026-08-22T09:46:41+00:00
1
2321.9
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-22T10:25:23+00:00
2
366.0
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-23T03:53:49+00:00
1
2286.4
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-23T04:31:56+00:00
2
365.1
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-24T01:10:51+00:00
1
2269.1
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-24T01:48:40+00:00
2
366.8
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-25T02:58:37+00:00
1
2328.3
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-25T03:37:26+00:00
2
367.2
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-26T22:16:19+00:00
1
2344.4
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15
2026-08-26T22:55:23+00:00
2
365.9
ballina,broken_hill,byron,cessnock,gunnedah,gympie,maranoa,strathfield
albury,bega_valley,bogan,bundaberg,burwood,dubbo,fairfield,griffith,lismore,muswellbrook,port_macquarie_hastings,port_stephens,shoalhaven,singleton,upper_hunter
8
0
15

Statistics

Average successful run time: about 7 hours

Total run time: 2 months

Total cpu time used: about 5 hours

Total disk space used: 148 MB

History

  • Auto ran revision 2b320e6e and failed .
    385 records added, 375 records removed in the database
  • Auto ran revision 2b320e6e and failed .
    743 records added, 703 records removed in the database
  • Auto ran revision 2b320e6e and failed .
    299 records added, 299 records removed in the database
  • Auto ran revision 2b320e6e and failed .
    450 records added, 429 records removed in the database
  • Auto ran revision 2b320e6e and failed .
    821 records added, 817 records removed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

multiple_masterview / scraper.rb