Sunday, 28 July 2013

Raspberry Pi and the HD PVR.. streaming and recording, Part III

It's been a little under 2 months since I posted last, and I've been busy working on a new build of the code, with a web interface, and other such useful things. I kept meaning to post about it, but kept going back to add yet another little tweak.

So, finally, it's time to release the latest version! read on to find out how to use it..

I've setup a github repository to handle the code, although you'll still need to compile it up yourself.

The repository is at :

You'll still want to follow the instructions at to compile it (swap hdpvr.c for hd-pivr.c)

Once built, you can run it as before, it will write recordings to the directory you are in when you run it.

There are now two interfaces, the original simple Http Get interface...
  start - http://ip.of.pi;1101/startrec
  stop - http://ip.of.pi:1101/stoprec
  status - http://ip.of.pi:1101/status
  video - http://ip.of.pi:1101/video (open in vlc)

...and the new Http Post interface, that lets you request status/start/stop, but also, add scheduled events =) It includes the worlds simplest http server, I wouldn't go letting this thing be internet facing, but it's good enough to let the app host its own web user interface.

The Post requests should all be made to http://ip.of.pi:1101/action with the post data carrying the arguments saying the type of request, and any argument payloads.

The simplest Post request is a status request, simply send the argument 'action=status' to receive the same response as you'd get back from the http://ip.of.pi:1101/status request. Not all that useful, it would be more handy if it returned JSON ;p instead, so try sending argument 'action=jstatus' and shiny json responses are yours.

Start and Stop have Post request versions too.. just send 'action=startrec' or 'action=stoprec' to start and stop recording respectively.

Which leads us to the new event engine, which allows you to schedule a start, or a stop for the future. They also allow you to schedule messages to be output as handy test events, and will (almost but not quite yet) allow you to schedule execution of a script with specified arguments, which I hope to use for channel change in the not too distant future.

But first, a few basics about events...

Every event has 2 id's, one owned by engine 'intId' and one owned by the interface 'extId', when you add events, you can specify these, which you'll need if you want to remove the event later. If you omit the intId when creating an event, the engine will assign one for you.

The idea is to schedule a channel change, then a start recording, then a stop recording.. if they share the same IntId, they can be deleted as a group later. The start event will carry the filename to use as part of its arguments. If you set a recording to overlap (note this really only makes sense if the shows are on the same channel) then the data will be written to both files during the overlap. Bear in mind that writing to two files will use more resources than writing to a single file. I've successfully tested writing to two files over a network share though, so I'd be interested to hear how you get on =)

Ok, so here's the Post api to use with events =)

  action=addevent        - requests adding event
  type=(0,1,2,3)           - 0=echo, 1=startrec, 2=stoprec, 3=channelchange
  time=2 Jan 2010 12:45:20     - the time the event should trigger, it MUST be in this format.
  intId=1234567890     - intId to use, if this parm is omitted, one is assigned for you.         - extId to use, set this to something you will use to identify this event later.
  data=message            - valid for type=0, and type=3, text to be output, or used as args to script.
  filename=myrec.ts     - valid for type=1, the filename to record into.
  showid=tt01234567   - additional data for recording, can be left empty if not used.

Adding an event in the past will mean it will trigger immediately. When you add an event, the response is in JSON format, and will include the intId of the added event, and the updated state of the event list. The date parser really is very very daft, so keep your dates exactly in the format expected. Expect 404 responses with error messages if you leave out fields that are required.

Also remember all times are in Pi time. This most likely means if your Pi is running UTC clock, that you'll need to convert times to that clock before adding the events.. check what time your Pi thinks it is by using the date command at the command line.

  action=removeevent   - requests removal of an event
  intId=1234567890      - the intId of the event to remove.          - the extId of the event to remove, if omitted, only intId is matched.

Removing an event will respond with the updated list in JSON format.

  action=listevents         - requests JSON content of current event list.

The current list is returned.. handy for populating tables etc.

  action=startrec            - requests start recording immediately
  intId=1234567890      - if omitted, the fixed id UI.Int.Id is used.          - if omitted, the fixed if UI.Ext.Id is used.

All recordings have associated intId/extId's, so any clashes with scheduled events can be managed.. even if the recorder is already recording, sending startrec with a different intId/extId pair will cause a parallel recording to be made.

  action=stoprec           - requests stop recording immediately
  intId=1234567890      - if omitted, the fixed id UI.Int.Id is used.          - if omitted, the fixed if UI.Ext.Id is used.

Similar to action=startrec, this can be used to terminate a recording already in progress. This action has one last trick though, if you send an empty string as the value for intId and extId, ALL recordings in progress will be terminated.

And that's pretty much it for the API at the moment ;p

By now, if you're still reading you're either already off reading up on DOJO/JQuery etc to build an awesome web page to control all of this.. or slightly bemused and hoping I've already done that for you =)

Good news, I've included 2 fairly simple interfaces to let you experiment & schedule events =)

The extremely simple webserver built into the hd-pvr command will serve any content in the directory you are in when you run the command, under the url http://ip.of.pi:1101/web/

Eg, if you have the file readme.txt in the current directory when you run hd-pivr, http://ip.of.pi:1101/web/readme.txt will serve the page to the browser.

In the repository, there are 4 html files.
  ui.html        - a simple HTTP form based ui, more useful for testing than actual usage. 
  video.html   - used by ui.html to embed the VLC player in a browser.
  dojo.html    - the simplest little ui I could come up with
  status.html  - used by dojo.html to display Pi time to you, recording status, and no of connections

Access these as http://ip.of.pi:1101/web/ui.html or http://ip.of.pi:1101/web/dojo.html

If you come up with a better UI, just let me know =) I'm happy to include them, my javascript skills are rusty at best ;p

I'm still working on the html/javascript side of stuff, and still plan to get the channel change script running.. and have a simple 'add recording' option that combines channelchange, start & stop creation into one action.. but I figured it was worth getting this version out sooner =) especially since it appears there are actually people using it now!



  1. This is brilliant. Exactly what I was after.

    Seems to have a slight bug where it doesn't release its file handles properly though. If I fire up VLC and watch a stream and then close it, it keeps the HDPVR spinning.

    Not a deal breaker when watching live, but the same thing seems to happen with recordings. It means when you start your 2nd ... nth separate recordings, all the files opened previously get the same data appended on them.

    My C is simply terrible so I can't spot why. I'll let you know if I work it out!

  2. Hmm.. that's certainly not the expected behavior.. here if I connect with only vlc then disconnect, the hd pvr is shut down.. if I connect vlc then start recording then disconnect, it will continue the recording then shut down the hd pvr at the end.

    It does support overlapped recordings tho, so if I connect, and start watching, then start recording to file A, then start recording to file B, then stop the recording to file A.. file B will continue to record as expected.. I've had it streaming to 2 vlc clients while recording to 2 different files on a samba mount from the single hd pvr unit.

    If you describe exactly what you're doing to recreate the issue, I'll see if I can figure it out.. its hopefully just my poor documentation ;p need to know which method you used to start and stop the recordings etc..

  3. I think the HDPVR staying up after i've disconnected is just the behaviour of the tcp connection when it isn't closed cleanly. It only happens if I kill VLC, rather than stopping playback.

    The recordings continue to trouble me though. How does a scheduled stop event know which file to stop writing to? I presume it's using either the intID or extID?

    Should I be scheduling my stop events with the intid returned by the start event? I thought just retaining the same extid would do the trick.


  4. Aha.. I've not tried killing VLC off, I guess that'd be the difference.. I'd hope that eventually the process would have to give up writing to the fd.. will read to see what would happen.

    Yes, the intId _and_ extId must match for start/stop events.. the idea is for intId to be something the server generates, and extId to be something that makes sense to remind you what the (set of) events was for. Having the server look after intId for you means you don't need to worry about creating unique id's across all the clients when saving events.

  5. I'm having some issues, no matter if recording, or streaming, the PVR keeps disconnecting. Pass through is perfectly fine, but the video is very choppy, any ideas?

  6. This comment has been removed by the author.

  7. Hello, thank you and I really like hdpivr. I've been using it for a couple of days now. This is a novice question regarding the permissions on the video files. I've set umask to give group read permission but is seems that hdpivr is not honouring it or at least overriding it. In an interactive shell the umask is being honoured under the same user. I'm not sure what I need to do or whether it may be possible to tweak the code. Any help would be very much appreciated.

  8. This is a correction to what I posted above. It is other that I need to give read permission rather than group. The umask i have for the user running hdpivr is 0022.

  9. Have a peek at line 270 or so.. it's there that the code is opening the file to write the recording to, and it's currently set to use S_IRWXU and S_IRWXG (flag values explained at ) .. I think that should give it user rwx and group rwx .. if you wanted to allow other to have read only , then you could add S_IROTH to the mask (eg S_IRWXU | S_IRWXG | S_IROTH ) Hope this helps =)