• Getting started
• Running scrapers on your own machine
• Using any library in your scraper
• Choose language version
• Secret values
• Webhooks
• Scraping JavaScript heavy sites
• How to use the API
• GitHub App

Secret values

Scraper code on morph.io is public and visible to anyone. So, how can you use an authenticated API, say the Twitter API, in a scraper without letting everyone know your authentication credentials?

This is where secret values come in. These are environment variables you can set. Only you and your scraper can see them.

Setting secret values

Let's say you have a secret value sdf8sdbw9Hihf you want to use in your scraper. We will now assign that value to an environment variable. We'll call it MORPH_MYSECRET. The name needs to start with MORPH_.

1. Go to your scraper and click the "Settings" button
2. Click "Add variable" and fill in the environment variable name and value
3. Click "Update"

Before you clicked "Update" it should have looked something like

You can of course set as many different secret values as you want. You just need to click "Add variable" again and give each secret value its own unique name.

Reading secret values in your scraper

  # Use my secret value
  my_secret_value = ENV['MORPH_MYSECRET']

  // Use my secret value
  $my_secret_value = getenv('MORPH_MYSECRET');

  import os
  # Use my secret variable
  my_secret_value = os.environ['MORPH_MYSECRET']

  # Use my secret value
  my $my_secret_value = $ENV{'MORPH_MYSECRET'};

  // Use my secret value
  my_secret_value = process.env.MORPH_MYSECRET;

Examples

Sometimes there's nothing like seeing a real-life example. If you have a scraper you would like to add to this list, please let us know.