LoveMyData / maitland

Maitland Development Applications


Maitland City Council Scraper

This council involves the followings

  • Server - nginx/1.8.0
  • Cookie tracking - Yes
  • Pagnation - No - However, modified query to pull max 100 records
  • XML - Yes
  • Horizon system

Setup MORPH_PERIOD for data recovery, available options are

  • thisweek (default, today -7 days)
  • thismonth
  • lastmonth

Enjoy

Contributors CloCkWeRX LoveMyData equivalentideas

Last run failed with status code 999.

Console output of last run

Please install the Morph Github App on LoveMyData so that Morph can access this repository on GitHub. Please go to https://github.com/apps/morph-io/installations/new/permissions?repository_ids%5B%5D=49949842&repository_ids%5B%5D=134056878&repository_ids%5B%5D=47324812&repository_ids%5B%5D=88161914&repository_ids%5B%5D=68444421&repository_ids%5B%5D=47314642&repository_ids%5B%5D=93590295&repository_ids%5B%5D=68878229&repository_ids%5B%5D=72811725&repository_ids%5B%5D=56903014&repository_ids%5B%5D=170608803&repository_ids%5B%5D=108057954&repository_ids%5B%5D=92110889&repository_ids%5B%5D=67974508&repository_ids%5B%5D=87934825&repository_ids%5B%5D=145681800&repository_ids%5B%5D=47260725&repository_ids%5B%5D=88051332&repository_ids%5B%5D=91232291&repository_ids%5B%5D=47496147&repository_ids%5B%5D=68881868&repository_ids%5B%5D=47331141&repository_ids%5B%5D=87817830&repository_ids%5B%5D=177515286&repository_ids%5B%5D=46924700&repository_ids%5B%5D=106624312&repository_ids%5B%5D=46796215&repository_ids%5B%5D=91168909&repository_ids%5B%5D=67973091&repository_ids%5B%5D=107615216&repository_ids%5B%5D=85133473&repository_ids%5B%5D=91752802&repository_ids%5B%5D=47815673&repository_ids%5B%5D=99768398&repository_ids%5B%5D=67903220&repository_ids%5B%5D=104952368&repository_ids%5B%5D=51361943&repository_ids%5B%5D=47970061&repository_ids%5B%5D=67978193&repository_ids%5B%5D=104017335&repository_ids%5B%5D=47314661&repository_ids%5B%5D=46924659&repository_ids%5B%5D=91053021&repository_ids%5B%5D=156531533&repository_ids%5B%5D=96767060&repository_ids%5B%5D=57299199&repository_ids%5B%5D=156799498&repository_ids%5B%5D=159443632&repository_ids%5B%5D=47876278&repository_ids%5B%5D=159422813&repository_ids%5B%5D=47313800&repository_ids%5B%5D=107520757&repository_ids%5B%5D=88119130&repository_ids%5B%5D=95849825&repository_ids%5B%5D=68373869&repository_ids%5B%5D=46966464&repository_ids%5B%5D=102549557&repository_ids%5B%5D=152169956&repository_ids%5B%5D=69199374&repository_ids%5B%5D=47871639&repository_ids%5B%5D=108206347&repository_ids%5B%5D=92701523&repository_ids%5B%5D=124995950&repository_ids%5B%5D=98845885&repository_ids%5B%5D=68511763&repository_ids%5B%5D=85155894&repository_ids%5B%5D=47400030&repository_ids%5B%5D=105615080&repository_ids%5B%5D=90943444&repository_ids%5B%5D=97595478&repository_ids%5B%5D=68342363&repository_ids%5B%5D=46921298&repository_ids%5B%5D=68098781&repository_ids%5B%5D=47917197&repository_ids%5B%5D=174301699&repository_ids%5B%5D=88171467&repository_ids%5B%5D=92243788&repository_ids%5B%5D=103522127&repository_ids%5B%5D=97825255&repository_ids%5B%5D=57381657&repository_ids%5B%5D=107633285&repository_ids%5B%5D=58468748&repository_ids%5B%5D=107853888&repository_ids%5B%5D=107861107&repository_ids%5B%5D=91777660&repository_ids%5B%5D=107852602&repository_ids%5B%5D=49410115&repository_ids%5B%5D=47873122&repository_ids%5B%5D=47874423&repository_ids%5B%5D=100552305&repository_ids%5B%5D=92278881&repository_ids%5B%5D=96265559&suggested_target_id=15683093 Why? See https://morph.io/documentation/github_app

Data

Downloaded 2252 times by LoveMyData dcalde gdmhunter

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (1.88 MB) Use the API

rows 10 / 6828

council_reference address description info_url comment_url date_scraped date_received
CCP-2016-2159
5 Lorn Street LORN 2320
Pool
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
DA-2016-2158
7 Justine Parade RUTHERFORD 2320
Tree Application
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
DA-2016-2157
56 Kyle Street RUTHERFORD 2320
Recreation Facility
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CC-2016-2156
23 McKeachie Drive ABERGLASSLYN 2320
Garage
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
DA-2016-2155
23 McKeachie Drive ABERGLASSLYN 2320
Garage
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CC-2016-2154
24 Rivergum Drive ABERGLASSLYN 2320
Dwelling House
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
DA-2016-2153
24 Rivergum Drive ABERGLASSLYN 2320
Dwelling House
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CDP-2016-2152
52 Hillcrest Drive GILLIESTON HEIGHTS 2321
Dwelling House
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CCP-2016-2151
14 Mitchell Drive EAST MAITLAND 2323
Shop
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15
CDP-2016-2150
28 Queen Street RUTHERFORD 2320
Dwelling alteration/addition
mailto:info@maitland.nsw.gov.au
2016-09-16
2016-09-15

Statistics

Average successful run time: less than a minute

Total run time: 1 day

Total cpu time used: 35 minutes

Total disk space used: 1.92 MB

History

  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • Auto ran and failed .
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

maitland / scraper.rb