ianheggie-oaf / example_ruby_scraper

README.md

This is a scraper that runs on Morph. To get started see the documentation

Contributors ianheggie-oaf

Last run completed successfully .

Console output of last run

Injecting configuration and compiling... [1G [1G-----> Ruby app detected [1G-----> Installing bundler 2.3.10 [1G-----> Removing BUNDLED WITH version in the Gemfile.lock [1G-----> Compiling Ruby [1G-----> Using Ruby version: ruby-3.2.2 [1G-----> Installing dependencies using bundler 2.3.10 [1G Running: BUNDLE_WITHOUT='development:test' BUNDLE_PATH=vendor/bundle BUNDLE_BIN=vendor/bundle/bin BUNDLE_DEPLOYMENT=1 bundle install -j4 [1G Fetching gem metadata from https://rubygems.org/......... [1G Fetching https://github.com/openaustralia/scraperwiki-ruby.git [1G Fetching public_suffix 7.0.5 [1G Using bundler 2.4.10 [1G Fetching connection_pool 3.0.2 [1G Fetching base64 0.3.0 [1G Fetching domain_name 0.6.20240107 [1G Installing base64 0.3.0 [1G Installing connection_pool 3.0.2 [1G Fetching mutex_m 0.3.0 [1G Fetching logger 1.7.0 [1G Installing public_suffix 7.0.5 [1G Installing mutex_m 0.3.0 [1G Installing domain_name 0.6.20240107 [1G Installing logger 1.7.0 [1G Fetching mime-types-data 3.2026.0701 [1G Fetching net-http-digest_auth 1.4.1 [1G Fetching nkf 0.3.0 [1G Installing net-http-digest_auth 1.4.1 [1G Fetching racc 1.8.1 [1G Installing mime-types-data 3.2026.0701 [1G Fetching webrick 1.9.2 [1G Installing nkf 0.3.0 with native extensions [1G Installing racc 1.8.1 with native extensions [1G Installing webrick 1.9.2 [1G Fetching webrobots 0.1.2 [1G Installing webrobots 0.1.2 [1G Fetching sqlite3 2.2.0 (x86_64-linux-gnu) [1G Fetching net-http-persistent 4.0.8 [1G Installing net-http-persistent 4.0.8 [1G Fetching rubyntlm 0.6.5 [1G Installing rubyntlm 0.6.5 [1G Fetching httpclient 2.9.0 [1G Installing httpclient 2.9.0 [1G Installing sqlite3 2.2.0 (x86_64-linux-gnu) [1G Fetching addressable 2.9.0 [1G Installing addressable 2.9.0 [1G Fetching http-cookie 1.1.6 [1G Installing http-cookie 1.1.6 [1G Fetching mime-types 3.7.0 [1G Installing mime-types 3.7.0 [1G Fetching sqlite_magic 0.0.6 [1G Installing sqlite_magic 0.0.6 [1G Using scraperwiki 3.0.1 from https://github.com/openaustralia/scraperwiki-ruby.git (at morph_defaults@fc50176) [1G Fetching nokogiri 1.17.2 (x86_64-linux) [1G Installing nokogiri 1.17.2 (x86_64-linux) [1G Fetching mechanize 2.14.0 [1G Installing mechanize 2.14.0 [1G Bundle complete! 5 Gemfile dependencies, 24 gems now installed. [1G Gems in the groups 'development' and 'test' were not installed. [1G Bundled gems are installed into `./vendor/bundle` [1G Bundle completed (6.20s) [1G Cleaning up the bundler cache. [1G Removing bundler (2.3.10) [1G-----> Detecting rake tasks [1G [1G [1G [1G-----> Discovering process types [1G Procfile declares types -> scraper Injecting scraper and running... Running ruby 3.2.2 on Linux bf66f9b0138e 6.14.3-x86_64-linode168 #1 SMP PREEMPT_DYNAMIC Mon Apr 21 19:47:55 EDT 2025 x86_64 x86_64 x86_64 GNU/Linux 18: Example Domain

Data

Downloaded 0 times

To download data sign in with GitHub

Download table (as CSV) Download SQLite database (12 KB) Use the API

rows 1 / 1

name
Example Domain

Statistics

Average successful run time: half a minute

Total run time: 25 minutes

Total cpu time used: half a minute

Total disk space used: 136 KB

History

  • Manually ran revision ca8b7a2f and completed successfully .
    1 record added, 1 record removed in the database
  • Manually ran revision de3be880 and completed successfully .
    1 record added, 1 record removed in the database
  • Manually ran revision 2ddeb300 and completed successfully .
    1 record added, 1 record removed in the database
  • Manually ran revision 9155c66d and failed .
    nothing changed in the database
  • Manually ran revision 4af0eafa and failed .
    nothing changed in the database
  • ...
  • Created on morph.io

Show complete history

Scraper code

Ruby

example_ruby_scraper / scraper.rb