blablupcom / sp_EFR003_DFEFARA_gov

Scrapes data.gov.uk and data.defra.gov.uk

data.gov.uk

Contributors woodbine blablupcom

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Python app detected [1G-----> Installing python-2.7.6 [1G-----> Noticed cffi. Bootstrapping libffi. [1G $ pip install -r requirements.txt [1G Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r requirements.txt (line 2)) [1G Cloning http://github.com/openaustralia/scraperwiki-python.git (to morph_defaults) to ./.heroku/src/scraperwiki [1G Collecting BeautifulSoup==3.2.0 (from -r requirements.txt (line 9)) [1G /app/.heroku/python/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning. [1G SNIMissingWarning [1G /app/.heroku/python/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. [1G InsecurePlatformWarning [1G Downloading BeautifulSoup-3.2.0.tar.gz [1G Collecting Creoleparser==0.7.4 (from -r requirements.txt (line 10)) [1G Downloading Creoleparser-0.7.4.zip [1G Collecting Genshi==0.6 (from -r requirements.txt (line 11)) [1G Downloading Genshi-0.6.tar.gz (433kB) [1G Collecting Jinja2==2.6 (from -r requirements.txt (line 12)) [1G Downloading Jinja2-2.6.tar.gz (389kB) [1G Collecting Markdown==2.2.0 (from -r requirements.txt (line 13)) [1G Downloading Markdown-2.2.0.tar.gz (236kB) [1G Collecting Pygments==1.4 (from -r requirements.txt (line 14)) [1G Downloading Pygments-1.4.tar.gz (3.5MB) [1G Collecting SQLAlchemy==0.6.6 (from -r requirements.txt (line 15)) [1G Downloading SQLAlchemy-0.6.6.tar.gz (2.1MB) [1G Collecting Twisted==11.1.0 (from -r requirements.txt (line 16)) [1G Downloading Twisted-11.1.0.tar.bz2 (2.8MB) [1G Collecting Unidecode==0.04.9 (from -r requirements.txt (line 17)) [1G Downloading Unidecode-0.04.9.tar.gz (196kB) [1G Collecting anyjson==0.3.3 (from -r requirements.txt (line 18)) [1G Downloading anyjson-0.3.3.tar.gz [1G Collecting argparse==1.2.1 (from -r requirements.txt (line 19)) [1G Downloading argparse-1.2.1.tar.gz (69kB) [1G Collecting beautifulsoup4==4.1.3 (from -r requirements.txt (line 20)) [1G Downloading beautifulsoup4-4.1.3.tar.gz (58kB) [1G Collecting bitlyapi==0.1.1 (from -r requirements.txt (line 21)) [1G Downloading bitlyapi-0.1.1.tar.gz [1G Collecting blinker==1.2 (from -r requirements.txt (line 22)) [1G Downloading blinker-1.2.tar.gz (66kB) [1G Collecting cartodb==0.6 (from -r requirements.txt (line 23)) [1G Downloading cartodb-0.6.tar.gz [1G Collecting certifi==0.0.8 (from -r requirements.txt (line 24)) [1G Downloading certifi-0.0.8.tar.gz (118kB) [1G Collecting chardet==2.1.1 (from -r requirements.txt (line 25)) [1G Downloading chardet-2.1.1.tar.gz (178kB) [1G Collecting ckanclient==0.10 (from -r requirements.txt (line 26)) [1G Downloading ckanclient-0.10.tar.gz [1G Collecting colormath==1.0.8 (from -r requirements.txt (line 27)) [1G Downloading colormath-1.0.8.tar.gz [1G Collecting csvkit==0.3.0 (from -r requirements.txt (line 28)) [1G Downloading csvkit-0.3.0.tar.gz [1G Collecting dataset==0.5.2 (from -r requirements.txt (line 29)) [1G Downloading dataset-0.5.2.tar.gz [1G Collecting demjson==1.6 (from -r requirements.txt (line 30)) [1G Downloading demjson-1.6.tar.gz (64kB) [1G Collecting dropbox==1.4 (from -r requirements.txt (line 31)) [1G Downloading dropbox-1.4.tar.gz [1G Collecting errorhandler==1.1.1 (from -r requirements.txt (line 32)) [1G Downloading errorhandler-1.1.1.tar.gz [1G Collecting feedparser==5.0.1 (from -r requirements.txt (line 33)) [1G Downloading feedparser-5.0.1.zip (2.0MB) [1G Collecting fluidinfo.py==1.1.2 (from -r requirements.txt (line 34)) [1G Downloading fluidinfo.py-1.1.2.tar.gz [1G Collecting gdata==2.0.15 (from -r requirements.txt (line 35)) [1G Downloading gdata-2.0.15.tar.gz (2.0MB) [1G Collecting geopy==0.94.1 (from -r requirements.txt (line 36)) [1G Downloading geopy-0.94.1.tar.gz [1G Collecting gevent==0.13.6 (from -r requirements.txt (line 37)) [1G Downloading gevent-0.13.6.tar.gz (289kB) [1G Collecting google-api-python-client==1.0beta8 (from -r requirements.txt (line 38)) [1G Downloading google-api-python-client-1.0beta8.zip (631kB) [1G Collecting googlemaps==1.0.2 (from -r requirements.txt (line 39)) [1G Downloading googlemaps-1.0.2.tar.gz (60kB) [1G Collecting greenlet==0.3.2 (from -r requirements.txt (line 40)) [1G Downloading greenlet-0.3.2.zip (50kB) [1G Collecting html5lib==0.90 (from -r requirements.txt (line 41)) [1G Downloading html5lib-0.90.tar.gz (86kB) [1G Collecting httplib2==0.7.4 (from -r requirements.txt (line 42)) [1G Downloading httplib2-0.7.4.tar.gz (106kB) [1G Collecting imposm.parser==1.0.3 (from -r requirements.txt (line 43)) [1G Downloading imposm.parser-1.0.3.tar.gz [1G Collecting jellyfish==0.2.0 (from -r requirements.txt (line 44)) [1G Downloading jellyfish-0.2.0.tar.gz [1G Collecting mechanize==0.2.5 (from -r requirements.txt (line 45)) [1G Downloading mechanize-0.2.5.tar.gz (383kB) [1G Collecting mock==0.7.2 (from -r requirements.txt (line 46)) [1G Downloading mock-0.7.2.zip (938kB) [1G Collecting networkx==1.6 (from -r requirements.txt (line 47)) [1G Downloading networkx-1.6.zip (1.0MB) [1G Collecting ngram==3.3.0 (from -r requirements.txt (line 48)) [1G Downloading ngram-3.3.0.tar.gz [1G Collecting nose==1.1.2 (from -r requirements.txt (line 49)) [1G Downloading nose-1.1.2.tar.gz (729kB) [1G Collecting oauth2==1.5.170 (from -r requirements.txt (line 50)) [1G Downloading oauth2-1.5.170.tar.gz [1G Collecting oauth==1.0.1 (from -r requirements.txt (line 51)) [1G Downloading oauth-1.0.1.tar.gz [1G Collecting oauthlib==0.1.2 (from -r requirements.txt (line 52)) [1G Downloading oauthlib-0.1.2.tar.gz [1G Collecting openpyxl==1.5.7 (from -r requirements.txt (line 53)) [1G Downloading openpyxl-1.5.7.tar.gz (67kB) [1G Collecting ordereddict==1.1 (from -r requirements.txt (line 54)) [1G Downloading ordereddict-1.1.tar.gz [1G Collecting pbkdf2==1.3 (from -r requirements.txt (line 55)) [1G Downloading pbkdf2-1.3.tar.gz [1G Collecting pdfminer==20110515 (from -r requirements.txt (line 56)) [1G Downloading pdfminer-20110515.tar.gz (4.1MB) [1G Collecting pexpect==2.4 (from -r requirements.txt (line 57)) [1G Downloading pexpect-2.4.tar.gz (113kB) [1G Collecting pipe2py==0.9.2 (from -r requirements.txt (line 58)) [1G Downloading pipe2py-0.9.2.tar.gz (57kB) [1G Collecting pyOpenSSL==0.13 (from -r requirements.txt (line 59)) [1G Downloading pyOpenSSL-0.13.tar.gz (250kB) [1G Collecting pycrypto==2.5 (from -r requirements.txt (line 60)) [1G Downloading pycrypto-2.5.tar.gz (426kB) [1G Collecting pycurl==7.19.0 (from -r requirements.txt (line 61)) [1G Downloading pycurl-7.19.0.tar.gz (70kB) [1G Collecting pyephem==3.7.5.1 (from -r requirements.txt (line 62)) [1G Downloading pyephem-3.7.5.1.tar.gz (703kB) [1G Collecting pyparsing==1.5.6 (from -r requirements.txt (line 63)) [1G Downloading pyparsing-1.5.6.zip (1.7MB) [1G Collecting pyth==0.5.6 (from -r requirements.txt (line 64)) [1G Downloading pyth-0.5.6.tar.gz [1G Collecting python-Levenshtein==0.10.2 (from -r requirements.txt (line 65)) [1G Downloading python-Levenshtein-0.10.2.tar.gz (45kB) [1G Collecting python-dateutil==1.5 (from -r requirements.txt (line 66)) [1G Downloading python-dateutil-1.5.tar.gz (233kB) [1G Collecting python-gflags==2.0 (from -r requirements.txt (line 67)) [1G Downloading python-gflags-2.0.tar.gz (65kB) [1G Collecting python-modargs==1.2 (from -r requirements.txt (line 68)) [1G Downloading python-modargs-1.2.tar.gz [1G Collecting python-stdnum==0.7 (from -r requirements.txt (line 69)) [1G Downloading python-stdnum-0.7.tar.gz (113kB) [1G Collecting pytz==2011k (from -r requirements.txt (line 70)) [1G Downloading pytz-2011k.zip (517kB) [1G Collecting rdflib==3.1.0 (from -r requirements.txt (line 71)) [1G Downloading rdflib-3.1.0.tar.gz (249kB) [1G Collecting requests-foauth==0.1.1 (from -r requirements.txt (line 72)) [1G Downloading requests-foauth-0.1.1.tar.gz [1G Collecting requests==1.0.4 (from -r requirements.txt (line 73)) [1G Downloading requests-1.0.4.tar.gz (336kB) [1G Collecting selenium==2.5.0 (from -r requirements.txt (line 74)) [1G Downloading selenium-2.5.0.tar.gz (2.4MB) [1G Collecting simplejson==2.2.1 (from -r requirements.txt (line 75)) [1G Downloading simplejson-2.2.1.tar.gz (49kB) [1G Collecting suds==0.4 (from -r requirements.txt (line 76)) [1G Downloading suds-0.4.tar.gz (104kB) [1G Collecting tweepy==1.7.1 (from -r requirements.txt (line 77)) [1G Downloading tweepy-1.7.1.tar.gz [1G Collecting tweetstream==1.1.1 (from -r requirements.txt (line 78)) [1G Downloading tweetstream-1.1.1.tar.gz [1G Collecting w3lib==1.0 (from -r requirements.txt (line 79)) [1G Downloading w3lib-1.0.tar.gz [1G Collecting xlrd==0.7.1 (from -r requirements.txt (line 81)) [1G Downloading xlrd-0.7.1.zip (125kB) [1G Collecting xlutils==1.4.1 (from -r requirements.txt (line 82)) [1G Downloading xlutils-1.4.1.tar.gz (40kB) [1G Collecting xlwt==0.7.2 (from -r requirements.txt (line 83)) [1G Downloading xlwt-0.7.2.zip (131kB) [1G Collecting xmltodict==0.4 (from -r requirements.txt (line 84)) [1G Downloading xmltodict-0.4.tar.gz [1G Collecting zope.interface==3.8.0 (from -r requirements.txt (line 88)) [1G Downloading zope.interface-3.8.0.tar.gz (111kB) [1G Collecting lxml==2.3.3 (from -r requirements.txt (line 89)) [1G Downloading lxml-2.3.3.tar.gz (3.1MB) [1G Collecting chromium-compact-language-detector==0.031415 (from -r requirements.txt (line 90)) [1G Downloading chromium_compact_language_detector-0.031415.tar.gz (2.2MB) [1G Collecting icalendar==3.0.1b1 (from -r requirements.txt (line 93)) [1G Downloading icalendar-3.0.1b1.tar.gz [1G Collecting pyquery==1.0 (from -r requirements.txt (line 96)) [1G Downloading pyquery-1.0.tar.gz [1G Collecting scrapely==0.9 (from -r requirements.txt (line 98)) [1G Downloading scrapely-0.9.tar.gz [1G Collecting Fom==0.9.8 (from -r requirements.txt (line 103)) [1G Downloading Fom-0.9.8.zip (79kB) [1G Collecting PyYAML==3.10 (from -r requirements.txt (line 105)) [1G Downloading PyYAML-3.10.tar.gz (241kB) [1G Collecting Scrapy==0.14.1 (from -r requirements.txt (line 107)) [1G Downloading Scrapy-0.14.1.tar.gz (719kB) [1G Collecting adspygoogle.adwords==15.6.2 (from -r requirements.txt (line 109)) [1G Downloading adspygoogle.adwords-15.6.2.tar.gz (166kB) [1G Collecting nltk==3.0.2 (from -r requirements.txt (line 111)) [1G Downloading nltk-3.0.2.tar.gz (991kB) [1G Collecting pydot==1.0.2 (from -r requirements.txt (line 113)) [1G Downloading pydot-1.0.2.tar.gz [1G Collecting M2Crypto==0.22.3 (from -r requirements.txt (line 115)) [1G Downloading M2Crypto-0.22.3.tar.gz (74kB) [1G Collecting dumptruck>=0.1.2 (from scraperwiki->-r requirements.txt (line 2)) [1G Downloading dumptruck-0.1.6.tar.gz [1G Collecting alembic>=0.6.2 (from dataset==0.5.2->-r requirements.txt (line 29)) [1G Downloading alembic-0.8.8.tar.gz (970kB) [1G Collecting python-slugify>=0.0.6 (from dataset==0.5.2->-r requirements.txt (line 29)) [1G Downloading python-slugify-1.2.1.tar.gz [1G Collecting distribute (from python-stdnum==0.7->-r requirements.txt (line 69)) [1G Downloading distribute-0.7.3.zip (145kB) [1G Collecting numpy (from scrapely==0.9->-r requirements.txt (line 98)) [1G Downloading numpy-1.11.2-cp27-cp27m-manylinux1_x86_64.whl (15.3MB) [1G Collecting Mako (from alembic>=0.6.2->dataset==0.5.2->-r requirements.txt (line 29)) [1G Downloading Mako-1.0.4.tar.gz (574kB) [1G Collecting python-editor>=0.3 (from alembic>=0.6.2->dataset==0.5.2->-r requirements.txt (line 29)) [1G Downloading python-editor-1.0.1.tar.gz [1G Collecting MarkupSafe>=0.9.2 (from Mako->alembic>=0.6.2->dataset==0.5.2->-r requirements.txt (line 29)) [1G Downloading MarkupSafe-0.23.tar.gz [1G Installing collected packages: dumptruck, requests, scraperwiki, BeautifulSoup, Genshi, Creoleparser, Jinja2, Markdown, Pygments, SQLAlchemy, zope.interface, Twisted, Unidecode, anyjson, argparse, beautifulsoup4, bitlyapi, blinker, httplib2, oauth2, cartodb, certifi, chardet, ckanclient, colormath, xlrd, python-dateutil, csvkit, MarkupSafe, Mako, python-editor, alembic, python-slugify, PyYAML, dataset, demjson, oauth, simplejson, dropbox, errorhandler, feedparser, fluidinfo.py, gdata, geopy, greenlet, gevent, python-gflags, google-api-python-client, googlemaps, html5lib, imposm.parser, jellyfish, mechanize, mock, networkx, ngram, nose, pycrypto, oauthlib, openpyxl, ordereddict, pbkdf2, pdfminer, pexpect, pipe2py, pyOpenSSL, pycurl, pyephem, pyparsing, pyth, python-Levenshtein, python-modargs, distribute, python-stdnum, pytz, rdflib, requests-foauth, selenium, suds, tweepy, tweetstream, w3lib, xlwt, xlutils, xmltodict, lxml, chromium-compact-language-detector, icalendar, pyquery, numpy, scrapely, Fom, Scrapy, adspygoogle.adwords, nltk, pydot, M2Crypto [1G Running setup.py install for dumptruck: started [1G Running setup.py install for dumptruck: finished with status 'done' [1G Running setup.py install for requests: started [1G Running setup.py install for requests: finished with status 'done' [1G Running setup.py develop for scraperwiki [1G Running setup.py install for BeautifulSoup: started [1G Running setup.py install for BeautifulSoup: finished with status 'done' [1G Running setup.py install for Genshi: started [1G Running setup.py install for Genshi: finished with status 'done' [1G Running setup.py install for Creoleparser: started [1G Running setup.py install for Creoleparser: finished with status 'done' [1G Running setup.py install for Jinja2: started [1G Running setup.py install for Jinja2: finished with status 'done' [1G Running setup.py install for Markdown: started [1G Running setup.py install for Markdown: finished with status 'done' [1G Running setup.py install for Pygments: started [1G Running setup.py install for Pygments: finished with status 'done' [1G Running setup.py install for SQLAlchemy: started [1G Running setup.py install for SQLAlchemy: finished with status 'done' [1G Running setup.py install for zope.interface: started [1G Running setup.py install for zope.interface: finished with status 'done' [1G Running setup.py install for Twisted: started [1G Running setup.py install for Twisted: finished with status 'done' [1G Running setup.py install for Unidecode: started [1G Running setup.py install for Unidecode: finished with status 'done' [1G Running setup.py install for anyjson: started [1G Running setup.py install for anyjson: finished with status 'done' [1G Running setup.py install for argparse: started [1G Running setup.py install for argparse: finished with status 'done' [1G Running setup.py install for beautifulsoup4: started [1G Running setup.py install for beautifulsoup4: finished with status 'done' [1G Running setup.py install for bitlyapi: started [1G Running setup.py install for bitlyapi: finished with status 'done' [1G Running setup.py install for blinker: started [1G Running setup.py install for blinker: finished with status 'done' [1G Running setup.py install for httplib2: started [1G Running setup.py install for httplib2: finished with status 'done' [1G Running setup.py install for oauth2: started [1G Running setup.py install for oauth2: finished with status 'done' [1G Running setup.py install for cartodb: started [1G Running setup.py install for cartodb: finished with status 'done' [1G Running setup.py install for certifi: started [1G Running setup.py install for certifi: finished with status 'done' [1G Running setup.py install for chardet: started [1G Running setup.py install for chardet: finished with status 'done' [1G Running setup.py install for ckanclient: started [1G Running setup.py install for ckanclient: finished with status 'done' [1G Running setup.py install for colormath: started [1G Running setup.py install for colormath: finished with status 'done' [1G Running setup.py install for xlrd: started [1G Running setup.py install for xlrd: finished with status 'done' [1G Running setup.py install for python-dateutil: started [1G Running setup.py install for python-dateutil: finished with status 'done' [1G Running setup.py install for csvkit: started [1G Running setup.py install for csvkit: finished with status 'done' [1G Running setup.py install for MarkupSafe: started [1G Running setup.py install for MarkupSafe: finished with status 'done' [1G Running setup.py install for Mako: started [1G Running setup.py install for Mako: finished with status 'done' [1G Running setup.py install for python-editor: started [1G Running setup.py install for python-editor: finished with status 'done' [1G Running setup.py install for alembic: started [1G Running setup.py install for alembic: finished with status 'done' [1G Running setup.py install for python-slugify: started [1G Running setup.py install for python-slugify: finished with status 'done' [1G Running setup.py install for PyYAML: started [1G Running setup.py install for PyYAML: finished with status 'done' [1G Running setup.py install for dataset: started [1G Running setup.py install for dataset: finished with status 'done' [1G Running setup.py install for demjson: started [1G Running setup.py install for demjson: finished with status 'done' [1G Running setup.py install for oauth: started [1G Running setup.py install for oauth: finished with status 'done' [1G Running setup.py install for simplejson: started [1G Running setup.py install for simplejson: finished with status 'done' [1G Running setup.py install for dropbox: started [1G Running setup.py install for dropbox: finished with status 'done' [1G Running setup.py install for errorhandler: started [1G Running setup.py install for errorhandler: finished with status 'done' [1G Running setup.py install for feedparser: started [1G Running setup.py install for feedparser: finished with status 'done' [1G Running setup.py install for fluidinfo.py: started [1G Running setup.py install for fluidinfo.py: finished with status 'done' [1G Running setup.py install for gdata: started [1G Running setup.py install for gdata: finished with status 'done' [1G Running setup.py install for geopy: started [1G Running setup.py install for geopy: finished with status 'done' [1G Running setup.py install for greenlet: started [1G Running setup.py install for greenlet: finished with status 'done' [1G Running setup.py install for gevent: started [1G Running setup.py install for gevent: finished with status 'done' [1G Running setup.py install for python-gflags: started [1G Running setup.py install for python-gflags: finished with status 'done' [1G Running setup.py install for google-api-python-client: started [1G Running setup.py install for google-api-python-client: finished with status 'done' [1G Running setup.py install for googlemaps: started [1G Running setup.py install for googlemaps: finished with status 'done' [1G Running setup.py install for html5lib: started [1G Running setup.py install for html5lib: finished with status 'done' [1G Running setup.py install for imposm.parser: started [1G Running setup.py install for imposm.parser: finished with status 'done' [1G Running setup.py install for jellyfish: started [1G Running setup.py install for jellyfish: finished with status 'done' [1G Running setup.py install for mechanize: started [1G Running setup.py install for mechanize: finished with status 'done' [1G Running setup.py install for mock: started [1G Running setup.py install for mock: finished with status 'done' [1G Running setup.py install for networkx: started [1G Running setup.py install for networkx: finished with status 'done' [1G Running setup.py install for ngram: started [1G Running setup.py install for ngram: finished with status 'done' [1G Running setup.py install for nose: started [1G Running setup.py install for nose: finished with status 'done' [1G Running setup.py install for pycrypto: started [1G Running setup.py install for pycrypto: finished with status 'done' [1G Running setup.py install for oauthlib: started [1G Running setup.py install for oauthlib: finished with status 'done' [1G Running setup.py install for openpyxl: started [1G Running setup.py install for openpyxl: finished with status 'done' [1G Running setup.py install for ordereddict: started [1G Running setup.py install for ordereddict: finished with status 'done' [1G Running setup.py install for pbkdf2: started [1G Running setup.py install for pbkdf2: finished with status 'done' [1G Running setup.py install for pdfminer: started [1G Running setup.py install for pdfminer: finished with status 'done' [1G Running setup.py install for pexpect: started [1G Running setup.py install for pexpect: finished with status 'done' [1G Running setup.py install for pipe2py: started [1G Running setup.py install for pipe2py: finished with status 'done' [1G Running setup.py install for pyOpenSSL: started [1G Running setup.py install for pyOpenSSL: finished with status 'done' [1G Running setup.py install for pycurl: started [1G Running setup.py install for pycurl: finished with status 'done' [1G Running setup.py install for pyephem: started [1G Running setup.py install for pyephem: finished with status 'done' [1G Running setup.py install for pyparsing: started [1G Running setup.py install for pyparsing: finished with status 'done' [1G Running setup.py install for pyth: started [1G Running setup.py install for pyth: finished with status 'done' [1G Running setup.py install for python-Levenshtein: started [1G Running setup.py install for python-Levenshtein: finished with status 'done' [1G Running setup.py install for python-modargs: started [1G Running setup.py install for python-modargs: finished with status 'done' [1G Running setup.py install for distribute: started [1G Running setup.py install for distribute: finished with status 'done' [1G Running setup.py install for python-stdnum: started [1G Running setup.py install for python-stdnum: finished with status 'done' [1G Running setup.py install for pytz: started [1G Running setup.py install for pytz: finished with status 'done' [1G Running setup.py install for rdflib: started [1G Running setup.py install for rdflib: finished with status 'done' [1G Running setup.py install for requests-foauth: started [1G Running setup.py install for requests-foauth: finished with status 'done' [1G Running setup.py install for selenium: started [1G Running setup.py install for selenium: finished with status 'done' [1G Running setup.py install for suds: started [1G Running setup.py install for suds: finished with status 'done' [1G Running setup.py install for tweepy: started [1G Running setup.py install for tweepy: finished with status 'done' [1G Running setup.py install for tweetstream: started [1G Running setup.py install for tweetstream: finished with status 'done' [1G Running setup.py install for w3lib: started [1G Running setup.py install for w3lib: finished with status 'done' [1G Running setup.py install for xlwt: started [1G Running setup.py install for xlwt: finished with status 'done' [1G Running setup.py install for xlutils: started [1G Running setup.py install for xlutils: finished with status 'done' [1G Running setup.py install for xmltodict: started [1G Running setup.py install for xmltodict: finished with status 'done' [1G Running setup.py install for lxml: started [1G Running setup.py install for lxml: finished with status 'done' [1G Running setup.py install for chromium-compact-language-detector: started [1G Running setup.py install for chromium-compact-language-detector: finished with status 'done' [1G Running setup.py install for icalendar: started [1G Running setup.py install for icalendar: finished with status 'done' [1G Running setup.py install for pyquery: started [1G Running setup.py install for pyquery: finished with status 'done' [1G Running setup.py install for scrapely: started [1G Running setup.py install for scrapely: finished with status 'done' [1G Running setup.py install for Fom: started [1G Running setup.py install for Fom: finished with status 'done' [1G Running setup.py install for Scrapy: started [1G Running setup.py install for Scrapy: finished with status 'done' [1G Running setup.py install for adspygoogle.adwords: started [1G Running setup.py install for adspygoogle.adwords: finished with status 'done' [1G Running setup.py install for nltk: started [1G Running setup.py install for nltk: finished with status 'done' [1G Running setup.py install for pydot: started [1G Running setup.py install for pydot: finished with status 'done' [1G Running setup.py install for M2Crypto: started [1G Running setup.py install for M2Crypto: finished with status 'done' [1G Successfully installed BeautifulSoup-3.2.0 Creoleparser-0.7.4 Fom-0.9.8 Genshi-0.6 Jinja2-2.6 M2Crypto-0.22.3 Mako-1.0.4 Markdown-2.2.0 MarkupSafe-0.23 PyYAML-3.10 Pygments-1.4 SQLAlchemy-0.6.6 Scrapy-0.14.1 Twisted-11.1.0 Unidecode-0.4.9 adspygoogle.adwords-15.6.2 alembic-0.8.8 anyjson-0.3.3 argparse-1.2.1 beautifulsoup4-4.1.3 bitlyapi-0.1.1 blinker-1.2 cartodb-0.6 certifi-0.0.8 chardet-2.1.1 chromium-compact-language-detector-0.31415 ckanclient-0.10 colormath-1.0.8 csvkit-0.3.0 dataset-0.5.2 demjson-1.6 distribute-0.7.3 dropbox-1.4 dumptruck-0.1.6 errorhandler-1.1.1 feedparser-5.0.1 fluidinfo.py-1.1.2 gdata-2.0.15 geopy-0.94.1 gevent-0.13.6 google-api-python-client-1.0b8 googlemaps-1.0.2 greenlet-0.3.2 html5lib-0.90 httplib2-0.7.4 icalendar-3.0.1b1 imposm.parser-1.0.3 jellyfish-0.2.0 lxml-2.3.3 mechanize-0.2.5 mock-0.7.2 networkx-1.6 ngram-3.3.0 nltk-3.0.2 nose-1.1.2 numpy-1.11.2 oauth-1.0.1 oauth2-1.5.170 oauthlib-0.1.2 openpyxl-1.5.7 ordereddict-1.1 pbkdf2-1.3 pdfminer-20110515 pexpect-2.4 pipe2py-0.9.2 pyOpenSSL-0.13 pycrypto-2.5 pycurl-7.19.0 pydot-1.0.2 pyephem-3.7.5.1 pyparsing-1.5.6 pyquery-1.0 pyth-0.5.6 python-Levenshtein-0.10.2 python-dateutil-1.5 python-editor-1.0.1 python-gflags-2.0 python-modargs-1.2 python-slugify-1.2.1 python-stdnum-0.7 pytz-2011k rdflib-3.1.0 requests-1.0.4 requests-foauth-0.1.1 scrapely-0.9 scraperwiki selenium-2.5.0 simplejson-2.2.1 suds-0.4 tweepy-1.7.1 tweetstream-1.1.1 w3lib-1.0 xlrd-0.7.1 xlutils-1.4.1 xlwt-0.7.2 xmltodict-0.4 zope.interface-3.8.0 [1G [1G ! Hello! It looks like your application is using an outdated version of Python. [1G ! This caused the security warning you saw above during the 'pip install' step. [1G ! We recommend 'python-2.7.12', which you can specify in a 'runtime.txt' file. [1G ! -- Much Love, Heroku. [1G [1G ! Hello! Your requirements.txt file contains the distribute package. [1G ! This library is automatically installed by Heroku and shouldn't be in [1G ! Your requirements.txt file. This can cause unexpected behavior. [1G ! -- Much Love, Heroku. [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... EFR003_DFEFARA_gov_2016_07 EFR003_DFEFARA_gov_2016_06 EFR003_DFEFARA_gov_2016_05 EFR003_DFEFARA_gov_2016_04 EFR003_DFEFARA_gov_2016_03 EFR003_DFEFARA_gov_2016_02 EFR003_DFEFARA_gov_2016_01 EFR003_DFEFARA_gov_2015_12 EFR003_DFEFARA_gov_2015_11 EFR003_DFEFARA_gov_2015_10 EFR003_DFEFARA_gov_2015_09 EFR003_DFEFARA_gov_2015_08 EFR003_DFEFARA_gov_2015_07 EFR003_DFEFARA_gov_2015_06 EFR003_DFEFARA_gov_2015_05 EFR003_DFEFARA_gov_2015_04 EFR003_DFEFARA_gov_2015_03 EFR003_DFEFARA_gov_2015_02 EFR003_DFEFARA_gov_2015_01 EFR003_DFEFARA_gov_2014_12 EFR003_DFEFARA_gov_2014_11 EFR003_DFEFARA_gov_2014_10 EFR003_DFEFARA_gov_2014_09 EFR003_DFEFARA_gov_2014_08 EFR003_DFEFARA_gov_2014_07 EFR003_DFEFARA_gov_2014_06 EFR003_DFEFARA_gov_2014_05 EFR003_DFEFARA_gov_2014_04 EFR003_DFEFARA_gov_2014_03 EFR003_DFEFARA_gov_2014_02 EFR003_DFEFARA_gov_2014_01 EFR003_DFEFARA_gov_2013_12 EFR003_DFEFARA_gov_2013_11 EFR003_DFEFARA_gov_2013_10 EFR003_DFEFARA_gov_2013_09 EFR003_DFEFARA_gov_2013_08 EFR003_DFEFARA_gov_2013_07 EFR003_DFEFARA_gov_2013_06 EFR003_DFEFARA_gov_2013_05 EFR003_DFEFARA_gov_2013_04 EFR003_DFEFARA_gov_2013_03 EFR003_DFEFARA_gov_2013_02 EFR003_DFEFARA_gov_2013_01 EFR003_DFEFARA_gov_2012_12 EFR003_DFEFARA_gov_2012_11 EFR003_DFEFARA_gov_2012_10 EFR003_DFEFARA_gov_2012_09 EFR003_DFEFARA_gov_2012_08 EFR003_DFEFARA_gov_2012_07 EFR003_DFEFARA_gov_2012_06 EFR003_DFEFARA_gov_2012_05 EFR003_DFEFARA_gov_2012_04 EFR003_DFEFARA_gov_2012_03 EFR003_DFEFARA_gov_2012_02 EFR003_DFEFARA_gov_2012_01 EFR003_DFEFARA_gov_2011_12 EFR003_DFEFARA_gov_2011_11 EFR003_DFEFARA_gov_2011_10 EFR003_DFEFARA_gov_2011_09 EFR003_DFEFARA_gov_2011_08 EFR003_DFEFARA_gov_2011_07 EFR003_DFEFARA_gov_2011_06 EFR003_DFEFARA_gov_2011_05 EFR003_DFEFARA_gov_2011_04 EFR003_DFEFARA_gov_2011_03 EFR003_DFEFARA_gov_2011_02 EFR003_DFEFARA_gov_2011_01 EFR003_DFEFARA_gov_2010_12 EFR003_DFEFARA_gov_2010_11 EFR003_DFEFARA_gov_2010_10 EFR003_DFEFARA_gov_2010_09 EFR003_DFEFARA_gov_2010_08 EFR003_DFEFARA_gov_2010_07 EFR003_DFEFARA_gov_2010_06 EFR003_DFEFARA_gov_2010_05 EFR003_DFEFARA_gov_2010_04

Data

Downloaded 4 times by MikeRalphson

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (22 KB) Use the API

rows 10 / 76

d l f
2016-10-13 10:09:48.540761
http://data.defra.gov.uk/ops/procurement/defra_25k_jul16.csv
EFR003_DFEFARA_gov_2016_07
2016-10-13 10:09:50.131132
http://data.defra.gov.uk/ops/procurement/defra_25k_june16.csv
EFR003_DFEFARA_gov_2016_06
2016-10-13 10:09:51.012242
http://data.defra.gov.uk/ops/procurement/defra_25k_may16.csv
EFR003_DFEFARA_gov_2016_05
2016-10-13 10:09:51.818211
http://data.defra.gov.uk/ops/procurement/defra_25k_april16.csv
EFR003_DFEFARA_gov_2016_04
2016-10-13 10:09:52.739805
http://data.defra.gov.uk/ops/procurement/defra_25k_mar16.csv
EFR003_DFEFARA_gov_2016_03
2016-10-13 10:09:53.726976
http://data.defra.gov.uk/ops/procurement/defra_25k_feb16.csv
EFR003_DFEFARA_gov_2016_02
2016-10-13 10:09:54.526259
http://data.defra.gov.uk/ops/procurement/defra_25k_jan16.csv
EFR003_DFEFARA_gov_2016_01
2016-10-13 10:09:55.373470
http://data.defra.gov.uk/ops/procurement/Defra_25k_Dec15.csv
EFR003_DFEFARA_gov_2015_12
2016-10-13 10:09:56.217298
http://data.defra.gov.uk/ops/procurement/Defra_25k_Nov15.csv
EFR003_DFEFARA_gov_2015_11
2016-10-13 10:09:57.098505
http://data.defra.gov.uk/ops/procurement/Jan-Dec%202015/Over_25k_October_2015.csv
EFR003_DFEFARA_gov_2015_10

Statistics

Average successful run time: 6 minutes

Total run time: 8 minutes

Total cpu time used: less than 5 seconds

Total disk space used: 49.2 KB

History

  • Manually ran revision 210ca339 and completed successfully .
    76 records added, 65 records removed in the database
    78 pages scraped
  • Manually ran revision 01daf87f and failed .
    65 records added in the database
    67 pages scraped
  • Created on morph.io