planningalerts-scrapers / townsville

Townsville City Council Development Applications


City of Townsville Council scraper

  • Server - .NET - XC.Track
  • Cookie tracking - No
  • Pagnation - Yes
  • Javascript - No
  • Clearly defined data within a row - Yes
  • Scrape detail page - Yes

Setup MORPH_PERIOD for data recovery, available options are

  • thisweek (default)
  • thismonth
  • lastmonth

Enjoy

Contributors mlandauer daveslutzkin henare hailspuds otherchirps LoveMyData jamezpolley

Last run failed with status code 255.

Console output of last run

Injecting configuration and compiling... [1G [1G-----> PHP app detected [91m-----> [0m [91mBootstrapping... [0m [91m-----> [0m [91mPreparing platform package installation... [0m [91m [1;33mNOTICE:[0m No runtime required in composer.json; requirements from dependencies in composer.lock will be used for selection [1;33mNOTICE:[0m No Composer platform-api-version recorded in composer.lock; file must be very old. Will attempt to use Composer 1 for installation. [0m [91m-----> [0m [91mInstalling platform packages... [0m [91m - php (7.4.33) [0m [91m - ext-sqlite3 (bundled with php) [0m [91m - ext-pdo_sqlite (bundled with php) [0m [91m - ext-mbstring (bundled with php) [0m [91m - ext-gd (bundled with php) [0m [91m - apache (2.4.57) [0m [91m - composer (1.10.26) [0m [91m - nginx (1.24.0) [0m [91m[1;33m ! WARNING: [0m [91mYour selected PHP version has reached end-of-life ! ! No updates or security fixes have been provided for your PHP ! version series by the PHP Group since 2022-11-28. ! ! It is strongly recommended you update your app to a version of ! PHP with "active support" status immediately to ensure you get ! the latest bugfixes and security updates each time you deploy. ! ! You may check the list of versions supported by the PHP Group ! and their EOL dates here: http://php.net/supported-versions.php ! ! For a list of supported runtimes & extensions on Heroku, please ! refer to: https://devcenter.heroku.com/articles/php-support [0m [91m[0m [0m [91m-----> [0m [91mInstalling dependencies... [0m [91m Composer version 1.10.26 2022-04-13 16:39:56 [0m [91m You are using Composer 1 which is deprecated. You should upgrade to Composer 2, see https://blog.packagist.com/deprecating-composer-1-support/ [0m [91m Loading composer repositories with package information [0m [91m Installing dependencies from lock file [0m [91m Package operations: 5 installs, 0 updates, 0 removals [0m [91m - Installing pguardiario/phpuri (1.0): Downloading Failed to download pguardiario/phpuri from dist: The "https://api.github.com/repos/monkeysuffrage/phpuri/zipball/ad0a5ec033fe616cfef55578b9c7f2458be8fbfc" file could not be downloaded (HTTP/1.1 404 Not Found) Now trying to download from source [0m [91m - Installing pguardiario/phpuri (1.0): Cloning ad0a5ec033 [0m [91m [0m [91m [RuntimeException] [0m [91m Failed to clone https://github.com/monkeysuffrage/phpuri.git via https, ssh protocols, aborting. [0m [91m [0m [91m - https://github.com/monkeysuffrage/phpuri.git [0m [91m Cloning into '/tmp/build/vendor/pguardiario/phpuri'... remote: Support for password authentication was removed on August 13, 2021. remote: Please see https://docs.github.com/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication. [0m [91m fatal: Authentication failed for 'https://github.com/monkeysuffrage/phpuri.git/' [0m [91m [0m [91m - git@github.com:monkeysuffrage/phpuri.git [0m [91m Cloning into '/tmp/build/vendor/pguardiario/phpuri'... [0m [91m Host key verification failed. [0m [91m fatal: Could not read from remote repository. [0m [91m [0m [91m Please make sure you have the correct access rights [0m [91m and the repository exists. install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] []... [0m [91m[1;31m [0m [91m ! ERROR: [0m [91mDependency installation failed! ! ! The 'composer install' process failed with an error. The cause [0m [91m ! may be the download or installation of packages, or a pre- or [0m [91m ! post-install hook (e.g. a 'post-install-cmd' item in 'scripts') ! in your 'composer.json'. ! [0m [91m ! Typical error cases are out-of-date or missing parts of code, [0m [91m ! timeouts when making external connections, or memory limits. ! [0m [91m ! Check the above error output closely to determine the cause of ! the problem, ensure the code you're pushing is functioning ! properly, and that all local changes are committed correctly. ! ! For more information on builds for PHP on Heroku, refer to ! https://devcenter.heroku.com/articles/php-support [0m [91m ! [0m [91m ! [1;33mREMINDER:[1;31m the following [1;33mwarnings[1;31m were emitted during the build; [0m [91m ! check the details above, as they may be related to this error: [0m [91m ! [1;33m-[1;31m Your selected PHP version has reached end-of-life [0m [91m[0m [0m

Data

Downloaded 11489 times by slav123 openaustralia dcalde mlandauer LoveMyData jesse-a-reilly DevAppProject cjrh90 TheZepto repose-lecture WilliamYun dandanaus peteclowes henare odnagroup MikeRalphson newmana Jedwards157 TRAVISSTR michaeltaylorgit

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.8 MB) Use the API

rows 10 / 6735

date_scraped description info_url date_received council_reference address comment_url
2011-09-11
Request to Extend the Relevant Period associated with R279.2572
2011-09-09
RC11/0015.02
82 Dunlop Street KELSO QLD 4815
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Request Approval to Allow Construction in 2 Stages associated with R279.2572
2011-09-02
RC11/0015.01
82 Dunlop Street KELSO QLD 4815
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House - Steep slopes Overlay and height Variation
2011-09-01
MC11/0071
29 Braemar Crescent CASTLE HILL QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House (Lot 1 RP 745347 Lease LZ No. 9)
2011-09-01
MI11/0049
7 - 9 Rae Crescent (Private) Cleveland Palms NOME QLD 4816
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Detached House - Steep Lands Overlay
2011-08-30
MC11/0069
348 Stanley Street NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Side Retaining Wall
2011-09-02
OP11/0098
12 Clifton Street NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Congregational Mixed-Use Facility ancillary to existing Educational Establishment and demolition of Four (4) site buildings including a building listed on the Local Heritage Database
2011-08-30
MC11/0070
45 The Strand NORTH WARD QLD 4810
mailto:eplanning@townsville.qld.gov.au
2011-09-11
North Shore Precinct 5 - Operational Works (Sewage Pump Station BU05 Rising Main and Gravity Sewer and Acid Sulphate Soils Test) associated with RC10/0189)
2011-08-30
OP11/0095
0 North Shore Boulevard BURDELL QLD 4818
mailto:eplanning@townsville.qld.gov.au
2011-09-11
North Shore Stage 520 - Operational Works associated with RC10/0189 (23 lots)
2011-09-05
OP11/0099
0 North Shore Boulevard BURDELL QLD 4818
mailto:eplanning@townsville.qld.gov.au
2011-09-11
Vacuum Truck Waste Composting and Soil Conditioning Manufacturing
2011-09-09
MC11/0073
111 Shaw Road SHAW QLD 4818
mailto:eplanning@townsville.qld.gov.au

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (2.8 MB) Use the API

rows 10 / 3157

id name value_blob type
1
RC12/0052.04
RC12/0052.04
string
2
MC08/0299.11
MC08/0299.11
string
3
MI11/0046.03
MI11/0046.03
string
4
MI08/0077.06
MI08/0077.06
string
5
MC13/0020.01
MC13/0020.01
string
6
RC13/0046.02
RC13/0046.02
string
7
RC12/0010.10
RC12/0010.10
string
8
OP12/0113.11
OP12/0113.11
string
9
OP12/0109.09
OP12/0109.09
string
10
OP12/0104.23
OP12/0104.23
string

Statistics

Average successful run time: half a minute

Total run time: 2 days

Total cpu time used: 21 minutes

Total disk space used: 2.86 MB

History

  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • Auto ran revision d9dabe9a and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

PHP

townsville / scraper.php