jamezpolley / ipswich

README.md

City of Ipswich Council scraper

Accept Terms - Yes
Cookie tracking - Yes
Request actual DA page for data - No
Clearly defined data within a row - No but reasonable
Limited to 10 pages only

Setup MORPH_PERIOD for data recovery, available options are

not set = past 14 days (default)
thismonth
lastmonth
custom [201808] for 2018 August

Contributors

Last run failed 2024-11-14 with status code 999.

Console output of last run

Data

Downloaded 1 time by

data

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (61 KB) Use the API

rows 10 / 148

info_url	council_reference	date_received	description	address	date_scraped	comment_url
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=771784	MCU-8599/2018	2018-11-02	Single Residential in a Character Zone and affected by a Development Constraints Overlay	11-13 Horan Street WOODEND QLD 4305	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=772155	MCU-8623/2018	2018-11-02	Material Change of Use - General Industry (Storage and Truck Depot) and Service/Trades Use (Builders/Contractors Depot, Repair Station and Storage)	136 Chalk Street WULKURAKA QLD 4305	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=772175	MCU-8625/2018	2018-11-02	Material Change of Use - Single Residential in a Rural Zone	287-355 Sippels Road CALVERT QLD 4340	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=773614	MCU-8715/2018	2018-11-07	Material Change of Use - Single Residential - Building Setback Plan	7001 Essex Street KARALEE QLD 4306	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=774064	MCU-8747/2018	2018-11-08	Material Change of Use - Single Residential (including Auxiliary Unit) in a Character Zone	14 George Street BLACKSTONE QLD 4304	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=774336	MCU-8766/2018	2018-11-08	General Industry - Concrete Column Manufacturing Plant	3 Mary Street BLACKSTONE QLD 4304	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=772257	CA-8631/2018	2018-11-02	Material Change of Use - Temporary Sales Office Advertising Device - One (1) Billboard and Four (4) Wall Signs	7001 Karrabin Rosewood Road ROSEWOOD QLD 4340	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=771594	OW-8583/2018	2018-11-01	Earthworks and Clearing Vegetation	48 Austin Street NEW CHUM QLD 4303	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=771644	OW-8586/2018	2018-11-02	Rate 3 Street Lighting	7001 Baird Circuit REDBANK PLAINS QLD 4301	2018-11-13	mailto:plandev@ipswich.qld.gov.au
http://pdonline.ipswich.qld.gov.au/pdonline/modules/applicationmaster/default.aspx?page=wrapper&key=772914	OW-8674/2018	2018-11-06	Earthworks - Edens Crossing Stages 10 to 14	Lot 89 Unnamed Road REDBANK PLAINS QLD 4301	2018-11-13	mailto:plandev@ipswich.qld.gov.au

Statistics

Average successful run time: half a minute

Total run time: about 13 hours

Total cpu time used: 26 minutes

Total disk space used: 99.9 KB

History

Auto ran and failed 2024-11-14.

run time 0 s
Auto ran and failed 2024-11-13.

run time 0 s
Auto ran and failed 2024-11-12.

run time 0 s
Auto ran and failed 2024-11-11.

run time 0 s
Auto ran and failed 2024-11-10.

run time 0 s
...
Created on morph.io 2018-11-13

Show complete history

Scraper code

Ruby

ipswich / scraper.rb

git clone URL