planningalerts-scrapers / townsville

Townsville City Council Development Applications


City of Townsville Council scraper

  • Server - .NET - XC.Track
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Clearly defined data within a row - Yes
  • Scrape detail page - Yes

Setup MORPH_PERIOD for data recovery, available options are

  • thisweek (default)
  • thismonth
  • lastmonth

Enjoy

Contributors mlandauer daveslutzkin henare hailspuds otherchirps LoveMyData jamezpolley

Last run failed with status code 255.

Console output of last run

Injecting configuration and compiling... [1G [1G-----> PHP app detected [91m-----> [0m [91mBootstrapping... [0m [91m-----> [0m [91mPreparing platform package installation... [0m [91m [1;33mNOTICE:[0m No runtime required in composer.json; requirements [0m [91m from dependencies in composer.lock will be used for selection [1;33mNOTICE:[0m No Composer platform-api-version recorded in composer.lock; file must be very old. Will attempt to use Composer 1 for installation. [0m [91m-----> [0m [91mInstalling platform packages... [0m [91m - php (7.4.33) [0m [91m - ext-sqlite3 (bundled with php) [0m [91m - ext-pdo_sqlite (bundled with php) [0m [91m - ext-mbstring (bundled with php) [0m [91m - ext-gd (bundled with php) [0m [91m - apache (2.4.57) [0m [91m - composer (1.10.26) [0m [91m - nginx (1.24.0) [0m [91m[1;33m ! WARNING: [0m [91mYour selected PHP version has reached end-of-life ! [0m [91m ! No updates or security fixes have been provided for your PHP ! version series by the PHP Group since 2022-11-28. ! [0m [91m ! It is strongly recommended you update your app to a version of [0m [91m ! PHP with "active support" status immediately to ensure you get [0m [91m ! the latest bugfixes and security updates each time you deploy. ! [0m [91m ! You may check the list of versions supported by the PHP Group [0m [91m ! and their EOL dates here: http://php.net/supported-versions.php ! [0m [91m ! For a list of supported runtimes & extensions on Heroku, please ! refer to: https://devcenter.heroku.com/articles/php-support [0m [91m[0m [0m [91m-----> [0m [91mInstalling dependencies... [0m [91m Composer version 1.10.26 2022-04-13 16:39:56 [0m [91m You are using Composer 1 which is deprecated. You should upgrade to Composer 2, see https://blog.packagist.com/deprecating-composer-1-support/ [0m [91m Loading composer repositories with package information [0m [91m Installing dependencies from lock file [0m [91m Package operations: 5 installs, 0 updates, 0 removals [0m [91m - Installing pguardiario/phpuri (1.0): Downloading Failed to download pguardiario/phpuri from dist: The "https://api.github.com/repos/monkeysuffrage/phpuri/zipball/ad0a5ec033fe616cfef55578b9c7f2458be8fbfc" file could not be downloaded (HTTP/1.1 404 Not Found) Now trying to download from source [0m [91m - Installing pguardiario/phpuri (1.0): Cloning ad0a5ec033 [0m [91m [0m [91m [RuntimeException] Failed to clone https://github.com/monkeysuffrage/phpuri.git via https, ssh protocols, aborting. [0m [91m - https://github.com/monkeysuffrage/phpuri.git [0m [91m Cloning into '/tmp/build/vendor/pguardiario/phpuri'... [0m [91m remote: Support for password authentication was removed on August 13, 2021. remote: Please see https://docs.github.com/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication. [0m [91m fatal: Authentication failed for 'https://github.com/monkeysuffrage/phpuri.git/' [0m [91m [0m [91m - git@github.com:monkeysuffrage/phpuri.git Cloning into '/tmp/build/vendor/pguardiario/phpuri'... [0m [91m Host key verification failed. [0m [91m fatal: Could not read from remote repository. [0m [91m [0m [91m Please make sure you have the correct access rights [0m [91m and the repository exists. [0m [91m [0m [91m [0m [91m install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] []... [0m [91m[1;31m ! ERROR: [0m [91mDependency installation failed! ! ! The 'composer install' process failed with an error. The cause [0m [91m ! may be the download or installation of packages, or a pre- or [0m [91m ! post-install hook (e.g. a 'post-install-cmd' item in 'scripts') ! in your 'composer.json'. ! [0m [91m ! Typical error cases are out-of-date or missing parts of code, [0m [91m ! timeouts when making external connections, or memory limits. ! [0m [91m ! Check the above error output closely to determine the cause of [0m [91m ! the problem, ensure the code you're pushing is functioning [0m [91m ! properly, and that all local changes are committed correctly. ! ! For more information on builds for PHP on Heroku, refer to [0m [91m ! https://devcenter.heroku.com/articles/php-support [0m [91m ! [0m [91m ! [1;33mREMINDER:[1;31m the following [1;33mwarnings[1;31m were emitted during the build; [0m [91m ! check the details above, as they may be related to this error: [0m [91m ! [1;33m-[1;31m Your selected PHP version has reached end-of-life [0m [91m[0m [0m

Data

Downloaded 11517 times by slav123 openaustralia dcalde mlandauer LoveMyData jesse-a-reilly DevAppProject cjrh90 TheZepto WilliamYun dandanaus repose-lecture henare odnagroup peteclowes Jedwards157 TRAVISSTR michaeltaylorgit MikeRalphson newmana

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.8 MB) Use the API

rows 10 / 6735

date_scraped description info_url date_received council_reference address comment_url
2011-09-11
Request to Extend the Relevant Period associated with R279.2572
2011-09-09
RC11/0015.02
82 Dunlop Street KELSO QLD 4815
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Request Approval to Allow Construction in 2 Stages associated with R279.2572
2011-09-02
RC11/0015.01
82 Dunlop Street KELSO QLD 4815
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House - Steep slopes Overlay and height Variation
2011-09-01
MC11/0071
29 Braemar Crescent CASTLE HILL QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House (Lot 1 RP 745347 Lease LZ No. 9)
2011-09-01
MI11/0049
7 - 9 Rae Crescent (Private) Cleveland Palms NOME QLD 4816
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House - Steep Lands Overlay
2011-08-30
MC11/0069
348 Stanley Street NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Side Retaining Wall
2011-09-02
OP11/0098
12 Clifton Street NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Congregational Mixed-Use Facility ancillary to existing Educational Establishment and demolition of Four (4) site buildings including a building listed on the Local Heritage Database
2011-08-30
MC11/0070
45 The Strand NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
North Shore Precinct 5 - Operational Works (Sewage Pump Station BU05 Rising Main and Gravity Sewer and Acid Sulphate Soils Test) associated with RC10/0189)
2011-08-30
OP11/0095
0 North Shore Boulevard BURDELL QLD 4818
mailto:eplanning@townsville.qld.gov.au
2011-09-11
North Shore Stage 520 - Operational Works associated with RC10/0189 (23 lots)
2011-09-05
OP11/0099
0 North Shore Boulevard BURDELL QLD 4818
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Vacuum Truck Waste Composting and Soil Conditioning Manufacturing
2011-09-09
MC11/0073
111 Shaw Road SHAW QLD 4818
mailto:eplanning@townsville.qld.gov.au

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.8 MB) Use the API

rows 10 / 3157

id name value_blob type
1
RC12/0052.04
RC12/0052.04
string
2
MC08/0299.11
MC08/0299.11
string
3
MI11/0046.03
MI11/0046.03
string
4
MI08/0077.06
MI08/0077.06
string
5
MC13/0020.01
MC13/0020.01
string
6
RC13/0046.02
RC13/0046.02
string
7
RC12/0010.10
RC12/0010.10
string
8
OP12/0113.11
OP12/0113.11
string
9
OP12/0109.09
OP12/0109.09
string
10
OP12/0104.23
OP12/0104.23
string

Statistics

Average successful run time: half a minute

Total run time: 2 days

Total cpu time used: 21 minutes

Total disk space used: 2.86 MB

History

  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

PHP

townsville / scraper.php