A tool to index available programmes on the BBC iPlayer / radio-player for offline analysis.
This is a scraper that runs on Morph. Please read the Morph.io documentation.
It can also be run locally.
Set an environment variable called MORPH_API_KEY
to your Nitro API key. When initially run on an empty / non-existant database, a
full scrape of available programmes is performed. Subsequently, only those programmes broadcast since the last update will be added.
Programmes passed their expiry date are removed.
To build a full index again, either delete the database or set the environment variable MORPH_REBUILD
to true
.
Usage: node scraper.js
Output is an sqlite3 database containing one table, "data":
javascript
var fields = [
'#index',
'type',
'name',
'pid',
'available',
'expires',
'episode',
'seriesnum',
'episodenum',
'versions',
'duration',
'desc',
'channel',
'categories',
'thumbnail',
'timeadded',
'guidance',
'web',
'vpids'
];
Also included is a simple command line utility called query.js
for performing SQL queries against the remote Morph.io database and
returning the rows in JSON format.
Usage: node query {SQL-statement}
To download data sign in with GitHub
rows 10 / 5548
#index | type | name | pid | available | expires | episode | seriesnum | episodenum | versions | duration | desc | channel | categories | thumbnail | timeadded | guidance | web | vpids |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
170108
|
radio
|
You and Yours
|
b00jbrgm
|
2012-11-17T18:30:28Z
|
2147472000
|
|
|
|
default
|
3420
|
Consumer news and issues with Julian Worricker.
|
bbc_radio_four
|
Factual,Consumer,Magazines & Reviews
|
1461194199
|
|
b00jbpwf
|
||
183376
|
radio
|
Home Front - Omnibus
|
b06l3js2
|
2016-03-23T00:05:24Z
|
1774926000
|
25-29 January 1916
|
|
45
|
podcast,default
|
3407
|
Pressures begin to tell at the Bevan hospital.
|
bbc_radio_four_extra
|
Drama
|
1461194613
|
|
p03c0kym,b06l3jrp
|
||
183379
|
radio
|
Home Front - Omnibus
|
b06kvlgq
|
2016-01-01T00:04:57Z
|
1767823080
|
4-8 January 1916
|
|
42
|
podcast,default
|
3403
|
The Bevan hospital becomes a welcome alternative to home for Dorothea.
|
bbc_radio_four
|
Drama
|
1461194613
|
|
p03dq1gh,b06kvlbf
|
||
183380
|
radio
|
Home Front - Omnibus
|
b06kvl04
|
2015-12-25T00:06:25Z
|
1767218280
|
28 December 1915 - 1 January 1916
|
|
41
|
podcast,default
|
3404
|
In the week that the home secretary resigns, the Bevan Hospital celebrates the new year.
|
bbc_radio_four
|
Drama
|
1461194613
|
|
p03dpwvq,b06kvkz6
|
||
183742
|
radio
|
Home Front - Omnibus
|
b06kvjhx
|
2015-12-08T04:47:43Z
|
1766613480
|
21-25 December 1915
|
|
40
|
podcast,podcast,default
|
3404
|
Epic drama series set in Great War Britain a hundred years ago this week.
|
bbc_radio_four
|
Drama
|
1461194616
|
|
p03bv8dj,p03bgwzb,b06kvjfj
|
||
183786
|
radio
|
Home Front - Omnibus
|
b064g17k
|
2015-12-03T04:02:45Z
|
1765771200
|
5-9 October 1915
|
|
36
|
podcast,default
|
3398
|
Individual losses and grief increasingly fuel an interest in spiritualism.
|
bbc_radio_four_extra
|
Drama
|
1461194617
|
|
p03045n8,b064g162
|
||
183790
|
radio
|
Home Front - Omnibus
|
b06492dw
|
2015-11-30T12:37:33Z
|
1765252800
|
7-11 September 1915
|
|
32
|
podcast,default
|
3403
|
The population begins to open up to thoughts of the hereafter.
|
bbc_radio_four_extra
|
Drama
|
1461194617
|
|
p031kvbx,b06492dl
|
||
183792
|
radio
|
Home Front - Omnibus
|
b0612rj1
|
2015-08-26T23:25:26Z
|
1756868400
|
6-10 July 1915
|
|
30
|
podcast,default
|
3409
|
Soldiers on leave find home very different to the one they thought they were fighting for.
|
bbc_radio_four_extra
|
Drama
|
1461194617
|
|
p02w69jw,b0612qzs
|
||
184184
|
radio
|
Home Front - Omnibus
|
b05xqg4p
|
2015-08-20T23:29:51Z
|
1756350000
|
8-12 June 1915
|
|
26
|
podcast,shortened,default
|
3395
|
The epic drama series set in Great War Britain a hundred years ago this week.
|
bbc_radio_four_extra
|
Drama
|
1461194682
|
|
p02szt44,b05xw7yb,b05xqfcv
|
||
184196
|
radio
|
Home Front - Omnibus
|
b054tnzn
|
2015-05-22T11:10:33Z
|
1747710000
|
9-13 March 1915
|
|
21
|
podcast,default
|
3400
|
The Battle of Neuve Chapelle draws to a close while the men at Marshalls spoil for a fight
|
bbc_radio_four_extra
|
Drama
|
1461194682
|
|
p02qmjsw,b054tn28
|
Average successful run time: 20 minutes
Total run time: 29 days
Total cpu time used: about 20 hours
Total disk space used: 63.9 MB