Feng-Gao / shanghai-api

data.sh.gov.cn

Contributors Feng-Gao

Last run failed with status code 1.

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Python app detected [1G ! The latest version of Python 2 is python-2.7.14 (you are using python-2.7.9, which is unsupported). [1G ! We recommend upgrading by specifying the latest version (python-2.7.14). [1G Learn More: https://devcenter.heroku.com/articles/python-runtimes [1G-----> Installing python-2.7.9 [1G-----> Installing pip [1G-----> Installing requirements with pip [1G DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. [1G Obtaining scraperwiki from git+http://github.com/openaustralia/scraperwiki-python.git@morph_defaults#egg=scraperwiki (from -r /tmp/build/requirements.txt (line 6)) [1G Cloning http://github.com/openaustralia/scraperwiki-python.git (to revision morph_defaults) to /app/.heroku/src/scraperwiki [1G Switched to a new branch 'morph_defaults' [1G Branch morph_defaults set up to track remote branch morph_defaults from origin. [1G Collecting beautifulsoup4==4.4.0 (from -r /tmp/build/requirements.txt (line 8)) [1G Downloading https://files.pythonhosted.org/packages/9d/c8/cd70aabb46af8f30ed83c15287c3d8b1455ba7ee923b03870ee0cdb6ec4f/beautifulsoup4-4.4.0-py2-none-any.whl (81kB) [1G Collecting requests==2.21.0 (from -r /tmp/build/requirements.txt (line 9)) [1G Downloading https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl (57kB) [1G Collecting lxml==3.4.4 (from -r /tmp/build/requirements.txt (line 10)) [1G Downloading https://files.pythonhosted.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz (3.5MB) [1G Collecting cssselect==0.9.1 (from -r /tmp/build/requirements.txt (line 11)) [1G Downloading https://files.pythonhosted.org/packages/aa/e5/9ee1460d485b94a6d55732eb7ad5b6c084caf73dd6f9cb0bb7d2a78fafe8/cssselect-0.9.1.tar.gz [1G Collecting fake_useragent==0.1.11 (from -r /tmp/build/requirements.txt (line 12)) [1G Downloading https://files.pythonhosted.org/packages/d1/79/af647635d6968e2deb57a208d309f6069d31cb138066d7e821e575112a80/fake-useragent-0.1.11.tar.gz [1G Collecting dumptruck>=0.1.2 (from scraperwiki->-r /tmp/build/requirements.txt (line 6)) [1G Downloading https://files.pythonhosted.org/packages/15/27/3330a343de80d6849545b6c7723f8c9a08b4b104de964ac366e7e6b318df/dumptruck-0.1.6.tar.gz [1G Collecting urllib3<1.25,>=1.21.1 (from requests==2.21.0->-r /tmp/build/requirements.txt (line 9)) [1G Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB) [1G Collecting certifi>=2017.4.17 (from requests==2.21.0->-r /tmp/build/requirements.txt (line 9)) [1G Downloading https://files.pythonhosted.org/packages/60/75/f692a584e85b7eaba0e03827b3d51f45f571c2e793dd731e598828d380aa/certifi-2019.3.9-py2.py3-none-any.whl (158kB) [1G Collecting idna<2.9,>=2.5 (from requests==2.21.0->-r /tmp/build/requirements.txt (line 9)) [1G Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) [1G Collecting chardet<3.1.0,>=3.0.2 (from requests==2.21.0->-r /tmp/build/requirements.txt (line 9)) [1G Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) [1G Installing collected packages: dumptruck, urllib3, certifi, idna, chardet, requests, scraperwiki, beautifulsoup4, lxml, cssselect, fake-useragent [1G Running setup.py install for dumptruck: started [1G Running setup.py install for dumptruck: finished with status 'done' [1G Running setup.py develop for scraperwiki [1G Running setup.py install for lxml: started [1G Running setup.py install for lxml: still running... [1G Running setup.py install for lxml: finished with status 'done' [1G Running setup.py install for cssselect: started [1G Running setup.py install for cssselect: finished with status 'done' [1G Running setup.py install for fake-useragent: started [1G Running setup.py install for fake-useragent: finished with status 'done' [1G Successfully installed beautifulsoup4-4.4.0 certifi-2019.3.9 chardet-3.0.4 cssselect-0.9.1 dumptruck-0.1.6 fake-useragent-0.1.11 idna-2.8 lxml-3.4.4 requests-2.21.0 scraperwiki urllib3-1.24.1 [91mDEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. [0m [91mDEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. [0m[1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... http://www.data.sh.gov.cn/query!queryInterface.action?currentPage=1 Traceback (most recent call last): File "scraper.py", line 57, in <module> result = requests.get(url,headers=headers) File "/app/.heroku/python/lib/python2.7/site-packages/requests/api.py", line 75, in get return request('get', url, params=params, **kwargs) File "/app/.heroku/python/lib/python2.7/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, **kwargs) File "/app/.heroku/python/lib/python2.7/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, **send_kwargs) File "/app/.heroku/python/lib/python2.7/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/app/.heroku/python/lib/python2.7/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='www.data.sh.gov.cn', port=80): Max retries exceeded with url: /query!queryInterface.action?currentPage=1 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f47c96ebf90>: Failed to establish a new connection: [Errno -2] Name or service not known',))

Statistics

Total run time: 2 minutes

Total cpu time used: less than 5 seconds

Total disk space used: 32.1 KB

History

  • Manually ran revision 6308ed2c and failed .
    nothing changed in the database
  • Manually ran revision 6308ed2c and failed .
    nothing changed in the database
  • Manually ran revision 6308ed2c and failed .
    nothing changed in the database
  • Manually ran revision 6308ed2c and failed .
    nothing changed in the database
  • Manually ran revision 6308ed2c and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

shanghai-api