ar73m / dou-scraper

Scrapes dou.ua

Сообщество программистов | DOU

Contributors Yegorov ar73m

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G-----> Node.js app detected [1G [1G-----> Reading application state [1G package.json... [1G build directory... [1G cache directory... [1G environment variables... [1G [1G Node engine: unspecified [1G Npm engine: unspecified [1G Start mechanism: Procfile [1G node_modules source: package.json [1G node_modules cached: false [1G [1G NPM_CONFIG_PRODUCTION=true [1G NPM_CONFIG_CAFILE=/etc/ssl/certs/ca-certificates.crt [1G NODE_MODULES_CACHE=true [1G [1G-----> Installing binaries [1G Resolving node version (latest stable) via semver.io... [1G Downloading and installing node 5.4.0... [1G Using default npm version: 3.3.12 [1G [1G-----> Building dependencies [1G No cache available [1G Installing node modules [1G dou-scraper@0.0.1 /tmp/build [1G `-- extractor@0.1.2 [1G `-- jsdom@7.2.2 [1G +-- abab@1.0.3 [1G +-- acorn@2.7.0 [1G +-- acorn-globals@1.0.9 [1G +-- cssom@0.3.0 [1G +-- cssstyle@0.2.30 [1G +-- escodegen@1.8.0 [1G | +-- esprima@2.7.1 [1G | +-- estraverse@1.9.3 [1G | +-- esutils@2.0.2 [1G | +-- optionator@0.8.1 [1G | | +-- deep-is@0.1.3 [1G | | +-- fast-levenshtein@1.1.0 [1G | | +-- levn@0.3.0 [1G | | +-- prelude-ls@1.1.2 [1G | | +-- type-check@0.3.2 [1G | | `-- wordwrap@1.0.0 [1G | `-- source-map@0.2.0 [1G | `-- amdefine@1.0.0 [1G +-- nwmatcher@1.3.7 [1G +-- parse5@1.5.1 [1G +-- request@2.67.0 [1G | +-- aws-sign2@0.6.0 [1G | +-- bl@1.0.0 [1G | | `-- readable-stream@2.0.5 [1G | | +-- core-util-is@1.0.2 [1G | | +-- inherits@2.0.1 [1G | | +-- isarray@0.0.1 [1G | | +-- process-nextick-args@1.0.6 [1G | | +-- string_decoder@0.10.31 [1G | | `-- util-deprecate@1.0.2 [1G | +-- caseless@0.11.0 [1G | +-- combined-stream@1.0.5 [1G | | `-- delayed-stream@1.0.0 [1G | +-- extend@3.0.0 [1G | +-- forever-agent@0.6.1 [1G | +-- form-data@1.0.0-rc3 [1G | | `-- async@1.5.2 [1G | +-- har-validator@2.0.5 [1G | | +-- chalk@1.1.1 [1G | | | +-- ansi-styles@2.1.0 [1G | | | +-- escape-string-regexp@1.0.4 [1G | | | +-- has-ansi@2.0.0 [1G | | | | `-- ansi-regex@2.0.0 [1G | | | +-- strip-ansi@3.0.0 [1G | | | `-- supports-color@2.0.0 [1G | | +-- commander@2.9.0 [1G | | | `-- graceful-readlink@1.0.1 [1G | | +-- is-my-json-valid@2.12.4 [1G | | | +-- generate-function@2.0.0 [1G | | | +-- generate-object-property@1.2.0 [1G | | | | `-- is-property@1.0.2 [1G | | | +-- jsonpointer@2.0.0 [1G | | | `-- xtend@4.0.1 [1G | | `-- pinkie-promise@2.0.0 [1G | | `-- pinkie@2.0.1 [1G | +-- hawk@3.1.2 [1G | | +-- boom@2.10.1 [1G | | +-- cryptiles@2.0.5 [1G | | +-- hoek@2.16.3 [1G | | `-- sntp@1.0.9 [1G | +-- http-signature@1.1.0 [1G | | +-- assert-plus@0.1.5 [1G | | +-- jsprim@1.2.2 [1G | | | +-- extsprintf@1.0.2 [1G | | | +-- json-schema@0.2.2 [1G | | | `-- verror@1.3.6 [1G | | `-- sshpk@1.7.3 [1G | | +-- asn1@0.2.3 [1G | | +-- assert-plus@0.2.0 [1G | | +-- dashdash@1.12.1 [1G | | +-- ecc-jsbn@0.1.1 [1G | | +-- jodid25519@1.0.2 [1G | | +-- jsbn@0.1.0 [1G | | `-- tweetnacl@0.13.3 [1G | +-- is-typedarray@1.0.0 [1G | +-- isstream@0.1.2 [1G | +-- json-stringify-safe@5.0.1 [1G | +-- mime-types@2.1.9 [1G | | `-- mime-db@1.21.0 [1G | +-- node-uuid@1.4.7 [1G | +-- oauth-sign@0.8.0 [1G | +-- qs@5.2.0 [1G | +-- stringstream@0.0.5 [1G | `-- tunnel-agent@0.4.2 [1G +-- sax@1.1.4 [1G +-- symbol-tree@3.1.4 [1G +-- tough-cookie@2.2.1 [1G +-- webidl-conversions@2.0.1 [1G +-- whatwg-url-compat@0.6.5 [1G | `-- tr46@0.0.2 [1G `-- xml-name-validator@2.0.1 [1G [1G [1G-----> Checking startup method [1G Found Procfile [1G [1G-----> Finalizing build [1G Creating runtime environment [1G Exporting binary paths [1G Cleaning npm artifacts [1G Cleaning previous cache [1G Caching results for future builds [1G [1G-----> Build succeeded! [1G [1G dou-scraper@0.0.1 /tmp/build [1G `-- extractor@0.1.2 [1G [1G WARNING: Node version not specified in package.json [1G https://devcenter.heroku.com/articles/nodejs-support#specifying-a-node-js-version [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... Processed http://dou.ua/ Last modified undefined Page record: {"page_title":[{"innerHTML":"Сообщество программистов | DOU","text":"Сообщество программистов | DOU"}],"li":[{"class":"link","href":"http://dou.ua/lenta/articles/docker/","innerHTML":"Виртуализация процесса разработки, часть 1: Docker"},{"class":"g-comments-round","href":"http://dou.ua/lenta/articles/docker/#comments","innerHTML":"30"},{"class":"link","href":"http://dou.ua/lenta/digests/python-digest-5/","innerHTML":"Python дайджест #5: “We will be moving to GitHub”"},{"class":"g-comments-round","href":"http://dou.ua/lenta/digests/python-digest-5/#comments","innerHTML":"14"},{"class":"link","href":"http://dou.ua/lenta/articles/nearables-wearables-experiment/","innerHTML":"Пошук людей у великому офісі: як ми подружили бікони з розумними годинниками"},{"class":"g-comments-round","href":"http://dou.ua/lenta/articles/nearables-wearables-experiment/#comments","innerHTML":"15"},{"class":"link","href":"http://dou.ua/lenta/articles/dont-move-abroad/","innerHTML":"Как я переехал в Польшу и вернулся назад: несколько причин не эмигрировать за границу"},{"class":"g-comments-round","href":"http://dou.ua/lenta/articles/dont-move-abroad/#comments","innerHTML":"1K"},{"class":"link","href":"http://dou.ua/lenta/interviews/it-career-239/","innerHTML":"Беседа с Викторией Амелиной, ex-it’шницей и писательницей"},{"class":"g-comments-round","href":"http://dou.ua/lenta/interviews/it-career-239/#comments","innerHTML":"25"},{"class":"link","href":"http://dou.ua/lenta/articles/dou-projector-oldstyleracing/","innerHTML":"DOU Проектор: OldStyleRacing — бесплатный timekiller для iOS устройств"},{"class":"g-comments-round","href":"http://dou.ua/lenta/articles/dou-projector-oldstyleracing/#comments","innerHTML":"2"},{"class":"link","href":"http://dou.ua/lenta/digests/java-digest-21/","innerHTML":"Java дайджест #21: язык года 2015"},{"class":"g-comments-round","href":"http://dou.ua/lenta/digests/java-digest-21/#comments","innerHTML":"13"},{"class":"link","href":"http://dou.ua/lenta/digests/qa-digest-14/","innerHTML":"QA дайджест #14: Привет новый год. Статистика-2015, тестовые эвристики, уязвимости Grub Linux, PayPal и Instagram"},{"class":"g-comments-round","href":"http://dou.ua/lenta/digests/qa-digest-14/#comments","innerHTML":"1"},{"class":"link","href":"http://dou.ua/lenta/articles/dou-projector-orna/","innerHTML":"DOU Проектор: Орна — фреймворк та CSS-бібліотека в сукупності"},{"class":"g-comments-round","href":"http://dou.ua/lenta/articles/dou-projector-orna/#comments","innerHTML":"65"}]}

Statistics

Average successful run time: 1 minute

Total run time: 1 minute

Total cpu time used: less than 5 seconds

Total disk space used: 20.6 KB

History

  • Manually ran revision c6002033 and completed successfully .
    nothing changed in the database
    1 page scraped
  • Created on morph.io

Scraper code

dou-scraper