tag:blogger.com,1999:blog-5133346003972190122024-03-13T23:16:35.447-07:00Dwellers Tech MusingsThoughts on tech things, geekery and other stuff with 9v dc.Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.comBlogger27125tag:blogger.com,1999:blog-513334600397219012.post-87820993302375681082014-08-13T09:35:00.003-07:002014-08-13T09:35:34.810-07:001 year.. no updates.. eek.. So I sign in today, to be greeted by Blogspot telling me I've been awol for an entire year. Zoinks!<br />
<br />
While I'd like to claim I was kidnapped by aliens who needed a lot of floppy disks read, the truth is sadly less impressive.. I've been kidnapped by Canadians who want to learn how to pronounce 'soldering' correctly. Well, almost, I've emigrated to Canada, which is in no small part the reason for the lack of postings.<br />
<br />
The good news is things are now settling down here, and I'm hoping soon to get back into working on the Pi+HD PVR code, and have a play around with some new shinies acquired this side of the pond.<br />
<br />
Expect normal infrequent blog updates to resume in the nearish future!!Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com2tag:blogger.com,1999:blog-513334600397219012.post-78779355442154336882013-08-11T00:02:00.001-07:002013-08-11T14:22:18.919-07:00Playing with the Hauppauge Colossus..I've been using the HD PVR usb unit a lot recently with the Raspberry Pi, but I recently found a great deal on a Hauppauge Colossus, which is like the PCI-e relation to the usb unit.. higher bitrates, no oddball stability issues like the usb unit had, but also sadly not going to work with a Pi..<br />
<br />
I've used a Colossus in my HTPC since release, so I knew what to expect, but hit an interesting snag with the latest drivers Colossus_1_8_31093_WHQL.zip, which although adding some VERY interesting options to the format settings dialog, also seemed to remove any way to adjust the bitrate.<br />
<br />
Thankfully, Hauppauge have been busy developing their own capture app, which <b>does</b> still have a bitrate control, and altering it there lets the value stick so Showbiz is able to use it. Not ideal.. but ok as a workaround for now.. I've emailed support to ask how it's supposed to work ;p<br />
<br />
The new options dialog looks impressive, having video scaling, framerate scaling, source info, mpeg profile level, audio encoding options, and hardware info.. (I'll add a screenshot here once I've stopped my next test capture.. ).. I've not seen most of these on the dialog before, so they offer a lot more control than previously.<br />
<br />
Why not just use the Hauppauge capture app ? because at least for me, recording 1080i with 2 channel audio meant huge audio breakups & video pausing while recording.. not great for a capture app. Plus no way to access those extra options! maybe they'll fix that later, but I get the feeling the app is aimed more at streaming captures of console gaming.<br />
<br />
<br />
<br />
<br />Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com2tag:blogger.com,1999:blog-513334600397219012.post-48144530965883760312013-07-28T13:03:00.000-07:002013-07-28T13:03:16.890-07:00Raspberry Pi and the HD PVR.. streaming and recording, Part IIIIt's been a little under 2 months since I posted last, and I've been busy working on a new build of the code, with a web interface, and other such useful things. I kept meaning to post about it, but kept going back to add yet another little tweak.<br />
<br />
So, finally, it's time to release the latest version! read on to find out how to use it..<br />
<br />
<a name='more'></a><br />
<br />
I've setup a github repository to handle the code, although you'll still need to compile it up yourself.<br />
<br />
The repository is at :<br />
<a href="https://github.com/BarDweller/HD-PiVR">http://github.com/BarDweller/HD-PiVR</a><br />
<br />
You'll still want to follow the instructions at <a href="http://dwellertech.blogspot.co.uk/2013/05/raspberry-pi-and-hd-pvr-streaming-and.html">http://dwellertech.blogspot.co.uk/2013/05/raspberry-pi-and-hd-pvr-streaming-and.html</a> to compile it (swap hdpvr.c for hd-pivr.c)<br />
<br />
Once built, you can run it as before, it will write recordings to the directory you are in when you run it.<br />
<br />
There are now two interfaces, the original simple Http Get interface...<br />
start - http://ip.of.pi;1101/startrec<br />
stop - http://ip.of.pi:1101/stoprec<br />
status - http://ip.of.pi:1101/status<br />
video - http://ip.of.pi:1101/video (open in vlc)<br />
<br />
...and the new Http Post interface, that lets you request status/start/stop, but also, add scheduled events =) It includes the worlds simplest http server, I wouldn't go letting this thing be internet facing, but it's good enough to let the app host its own web user interface.<br />
<br />
The Post requests should all be made to http://ip.of.pi:1101/action with the post data carrying the arguments saying the type of request, and any argument payloads.<br />
<br />
The simplest Post request is a status request, simply send the argument 'action=status' to receive the same response as you'd get back from the http://ip.of.pi:1101/status request. Not all that useful, it would be more handy if it returned JSON ;p instead, so try sending argument 'action=jstatus' and shiny json responses are yours.<br />
<br />
Start and Stop have Post request versions too.. just send 'action=startrec' or 'action=stoprec' to start and stop recording respectively.<br />
<br />
Which leads us to the new event engine, which allows you to schedule a start, or a stop for the future. They also allow you to schedule messages to be output as handy test events, and will (almost but not quite yet) allow you to schedule execution of a script with specified arguments, which I hope to use for channel change in the not too distant future.<br />
<br />
But first, a few basics about events...<br />
<br />
Every event has 2 id's, one owned by engine 'intId' and one owned by the interface 'extId', when you add events, you can specify these, which you'll need if you want to remove the event later. If you omit the intId when creating an event, the engine will assign one for you.<br />
<br />
The idea is to schedule a channel change, then a start recording, then a stop recording.. if they share the same IntId, they can be deleted as a group later. The start event will carry the filename to use as part of its arguments. If you set a recording to overlap (note this really only makes sense if the shows are on the same channel) then the data will be written to both files during the overlap. Bear in mind that writing to two files will use more resources than writing to a single file. I've successfully tested writing to two files over a network share though, so I'd be interested to hear how you get on =)<br />
<br />
Ok, so here's the Post api to use with events =)<br />
<br />
action=addevent - requests adding event<br />
type=(0,1,2,3) - 0=echo, 1=startrec, 2=stoprec, 3=channelchange<br />
time=<span style="background-color: white;">2 Jan 2010 12:45:20 - the time the event should trigger, it MUST be in this format.</span><br />
<span style="background-color: white;"> intId=1234567890 - intId to use, if this parm is omitted, one is assigned for you.</span><br />
<span style="background-color: white;"> extId=my.ext.id - extId to use, set this to something you will use to identify this event later.</span><br />
<span style="background-color: white;"> data=message - valid for type=0, and type=3, text to be output, or used as args to script.</span><br />
<span style="background-color: white;"> filename=myrec.ts - valid for type=1, the filename to record into.</span><br />
<span style="background-color: white;"> showid=tt01234567 - additional data for recording, can be left empty if not used.</span><br />
<br />
Adding an event in the past will mean it will trigger immediately. When you add an event, the response is in JSON format, and will include the intId of the added event, and the updated state of the event list. The date parser really is very very daft, so keep your dates exactly in the format expected. Expect 404 responses with error messages if you leave out fields that are required.<br />
<br />
Also remember <b>all times are in Pi time. </b>This most likely means if your Pi is running UTC clock, that you'll need to convert times to that clock before adding the events.. check what time your Pi thinks it is by using the <b>date</b> command at the command line.<br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"> action=removeevent - requests removal of an event</span><br />
<span style="background-color: white;"> intId=1234567890 - the intId of the event to remove. </span><br />
<span style="background-color: white;"> extId=my.ext.id - the extId of the event to remove, if omitted, only intId is matched.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">Removing an event will respond with the updated list in JSON format.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"> action=listevents - requests JSON content of current event list.</span><br />
<br />
The current list is returned.. handy for populating tables etc.<br />
<br />
action=startrec - requests start recording immediately<br />
intId=1234567890 - if omitted, the fixed id <span style="background-color: white; color: #dd1144; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px; white-space: pre;">UI.Int.Id </span>is used.<br />
extId=my.ext.id - if omitted, the fixed if <span style="background-color: white; color: #dd1144; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px; white-space: pre;">UI.Ext.Id </span>is used.<br />
<br />
All recordings have associated intId/extId's, so any clashes with scheduled events can be managed.. even if the recorder is already recording, sending startrec with a different intId/extId pair will cause a parallel recording to be made.<br />
<br />
action=stoprec - requests stop recording immediately<br />
intId=1234567890 - if omitted, the fixed id <span style="background-color: white; color: #dd1144; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px; white-space: pre;">UI.Int.Id </span>is used.<br />
extId=my.ext.id - if omitted, the fixed if <span style="background-color: white; color: #dd1144; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px; white-space: pre;">UI.Ext.Id </span>is used.<br />
<br />
Similar to action=startrec, this can be used to terminate a recording already in progress. This action has one last trick though, if you send an empty string as the value for intId and extId, ALL recordings in progress will be terminated.<br />
<br />
And that's pretty much it for the API at the moment ;p <br />
<br />
<span style="background-color: white;">By now, if you're still reading you're either already off reading up on DOJO/JQuery etc to build an awesome web page to control all of this.. or slightly bemused and hoping I've already done that for you =)</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">Good news, I've included 2 fairly simple interfaces to let you experiment & schedule events =)</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">The extremely simple webserver built into the hd-pvr command will serve any content in the directory you are in when you run the command, under the url http://ip.of.pi:1101/web/</span><br />
<span style="background-color: white;"><br />Eg, if you have the file readme.txt in the current directory when you run hd-pivr, http://ip.of.pi:1101/web/readme.txt will serve the page to the browser.</span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">In the repository, there are 4 html files.</span><br />
<span style="background-color: white;"> ui.html - a simple HTTP form based ui, more useful for testing than actual usage. </span><br />
<span style="background-color: white;"> video.html - used by ui.html to embed the VLC player in a browser.</span><br />
<span style="background-color: white;"> dojo.html - the simplest little ui I could come up with</span><br />
<span style="background-color: white;"> status.html - used by dojo.html to display Pi time to you, recording status, and no of connections</span><br />
<span style="background-color: white;"><br /></span>
Access these as http://ip.of.pi:1101/web/ui.html or http://ip.of.pi:1101/web/dojo.html<br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;">If you come up with a better UI, just let me know =) I'm happy to include them, my javascript skills are rusty at best ;p</span><br />
<span style="background-color: white;"><br /></span>
I'm still working on the html/javascript side of stuff, and still plan to get the channel change script running.. and have a simple 'add recording' option that combines channelchange, start & stop creation into one action.. but I figured it was worth getting this version out sooner =) especially since it appears there are actually people using it now!<br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"> </span>Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com9tag:blogger.com,1999:blog-513334600397219012.post-56960020228727965802013-06-02T09:54:00.003-07:002013-06-06T06:12:46.734-07:00Raspberry Pi and the HD PVR.. streaming and recording, Part IIIt's working =)<br />
<br />
Finally got to the bottom of the drop outs / glitches.. Some were my fault in the code, some were due to a dodgy cable feeding the HD PVR from the set top box. With the replacement cable in place, I had it stream for 12 hours straight with no reported dropouts.. that's close to a record for this unit =).<br />
<br />
With the new cable in place however, I could still make it glitch, but only when recording.. the very time you didn't want glitches ;p This was while writing data to a network share, mounted via cifs, and the glitches became more frequent if the same network share was used via another client.. all of which got me thinking...<br />
<br />
<a name='more'></a><br />
<br />
I was only using a single thread to handle the read from the device, and write out to all socket clients, and to the recording file.. maybe the writes to the mounted share were taking too long, and the extra delay meant too much data built up at the device node, causing the connection to be dropped..<br />
<br />
I had a chat with the MythTV developers in their IRC channel, figuring they must have hit stuff like this when adding support for the HD PVR, that gave a few ideas.. and I added a buffered writer thread (It's at this point you really miss Java). With a little careful use of locking, and buffer allocation though, I've got a buffered writer thread that writes buffers from a write list, and puts them back onto a free list to be reused.<br />
<br />
(Ok, if you're reading the code, they are arrays, not lists, and thus are pre-allocated to a max length, but the array is only holding pointers to the buffers, not the buffers themselves, so the overall overhead isn't too bad).<br />
<br />
With this new buffered writer in place, the glitches during write disappear.. and I discovered I needed to update the code to large file support (oops, should have seen that one coming!!), thankfully that was as simple as adding a few #defines.<br />
<br />
Lastly, since I now had a buffer pool style solution running, I added a way to monitor the buffer usage..<br />
<b style="font-family: 'Courier New', Courier, monospace;"> http://ip.address.of.pi:1101/status</b><br />
will report the number of connected streaming clients, the number of buffers in the free pool, the number of buffers with data waiting to be written to disk, and the total number of buffers. In an ideal world the total number of buffers will stay small, but it will start climbing rapidly if the network gets busy.. Currently the code is set to allow a max of 4096 buffers, that seems enough to cope with my network, although the amount may need tweaking..<br />
<br />
A quick check on the pi says that a char * pointer takes 4 bytes, so that's 32k for the free/write array storage, and holding 4096 buffers of 4k each, will eat 16MB of ram.. even on a 256MB pi that seems ok =). I've seen my total buffer usage creep up toward 11MB so I might even consider allowing 32MB (8192 buffers) if it helps guarantee those writes.<br />
<br />
Now it's onto the coding of the java xmltv/imdb channel changing recording scheduler... I made pretty good progress on that so far (after getting myself temporarily blocked on Google & Yahoo ;p)<br />
<br />
And watch out for those URL auto suggestions! it seems Chrome actually sends a speculative HTTP GET for a url it offers you, before you select it and hit enter.. obviously, this is bad when you want to request 'status' and it hits the 'stoprec' url ;p<br />
<br />
<i>Update: the device restarts are gone with this version.. but it still seems to write bad data if streaming clients join while recording to disk.. I'll try moving the streaming client writes to a buffered writer too.. see if that helps.. although it'll get fun with 2 consumers of the buffer data ;p </i><br />
<i><br /></i>
<i>Update 2: have moved the socket writes to the buffer output thread, and added another thread to reclaim excessive free buffers over time.. this has fixed the joining client issue.. also tidied up a few bits .. new code to go soon when I get the http parser rewritten.</i><br />
<br />
<a href="https://gist.github.com/anonymous/5694016">Source:</a><br />
<br />
<div style="height: 480px; overflow-y: scroll;">
<script src="https://gist.github.com/anonymous/5694016.js"></script>
</div>
Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com11tag:blogger.com,1999:blog-513334600397219012.post-86115246009060600332013-05-27T23:52:00.000-07:002013-06-02T12:54:26.062-07:00Raspberry Pi and the HD PVR.. streaming and recordingSuccess! sort of...<br />
<br />
I've finished rebuilding the C app, to read from the HDPVR and offer the data via a simple HTTP interface, and embedded a little control to toggle recording on & off. With the HDPVR running around 8mbit, I'm able to stream to about 5 clients simultaneously, which is pretty good going.<br />
<br />
The downside is dropouts are still present.. (<i>Update: <a href="http://dwellertech.blogspot.co.uk/2013/06/raspberry-pi-and-hd-pvr-streaming-and.html">not anymore!!</a></i>) I'm hoping its just the HDPVR's fault, as these units are known to glitch occasionally & require a reopen.. even so I'm not going to call it a day yet, I'll have a bash at compiling MythTV and seeing how it's recording code gets on with the device.<br />
<br />
<br />
<a name='more'></a><br />
I've linked the source below, with a bit of luck, you can save it as hdpvr.c on the pi, and compile it using..<br />
<span style="font-family: Courier New, Courier, monospace;"> <b>gcc -lpthread -o hdpvr hdpvr.c</b></span><br />
<br />
Then run it using..<br />
<span style="font-family: Courier New, Courier, monospace;"><b> ./hdpvr</b></span><br />
<span style="font-family: Courier New, Courier, monospace;"><b><br /></b></span>
You should get a response of..<br />
<span style="font-family: Courier New, Courier, monospace;"><b> waiting for a connection</b></span><br />
<br />
Once it's running, fire up VLC (<a href="http://www.videolan.org/">http://www.videolan.org/</a>) and use 'open network stream' and give it the url..<br />
<span style="font-family: Courier New, Courier, monospace;"><b> http://ip.address.of.pi:1101/video</b></span><br />
<br />
And then you should see the video start playing =) Check my <a href="http://dwellertech.blogspot.co.uk/2013/05/raspberry-pi-and-hd-pvr.html">other post</a> if you need to select inputs other than component/stereo.<br />
<br />
Once the video is playing, you can start recording by visiting (with a browser)..<br />
<b style="font-family: 'Courier New', Courier, monospace;"> http://ip.address.of.pi:1101/startrec</b><br />
<br />
And stop recording by visiting (with a browser)..<br />
<b style="font-family: 'Courier New', Courier, monospace;"> http://ip.address.of.pi:1101/stoprec</b><br />
<div>
<br /></div>
<div>
Video is written to the directory you ran hdpvr from.. remember that the video streams will be large!! I've had success writing the data directly to a mounted SMB share =)</div>
<div>
<br /></div>
<div>
Dropouts are still present, the code will close & reopen the device.. usually the VLC client just glitches a little and then resumes playing. The recordings will need streamfixing afterwards to repair the glitches, (which will eventually end up as short cuts to the recording).. I've tried VideoRedo quick streamfix with mixed results so far.. Once I've got the Myth code recording, I'll see if I get better results.</div>
<div>
<br /></div>
<div>
With this code, the CPU load sits around 12% or so, so there's quite a bit of time free to still do stuff yet!!</div>
<div>
<br /></div>
<a href="https://gist.github.com/anonymous/5660853#file-gistfile1-c">Source:</a><br />
<div style="height: 480px; overflow-y: scroll;">
<script src="https://gist.github.com/anonymous/5660853.js"></script>
</div>
<br />Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com5tag:blogger.com,1999:blog-513334600397219012.post-72000180363756332442013-05-19T00:31:00.000-07:002013-06-02T12:53:29.919-07:00Raspberry Pi and the HD PVR.. coping with dropouts.. The HD PVR is a great bit of kit, but it does have one slight drawback.. it's either lazy, or it's easily distracted ;p You request data from it.. and wheee... it comes back in nice h264 format, great you think, and go make a cup of tea, bake some cookies, etc.. and come back to find the recording just.. stopped?!!<br />
<br />
Read on for a quick guide of what I've tried so far.. and what hasn't worked ;p<br />
<br />
<a name='more'></a>Firstly, from the last post, dd vs cat.. I've seen very little difference between the two, dd might ride out some situations where cat doesn't, but at the end of the day neither dd or cat handle the signal drop out.<br />
<br />
So I've moved over to my own little C application, based an awful lot on the code available from here..<br />
<a href="http://www.avsforum.com/t/1069799/hauppauge-hd-pvr-on-linux#post_15037025">http://www.avsforum.com/t/1069799/hauppauge-hd-pvr-on-linux#post_15037025</a><br />
<br />
Except I've rebuilt the retry logic so it waits a little while before closing & reopening the device after each open.. otherwise my HD PVR never locks onto the signal, and sits there trying to emulate a strobe unit. I've also added date/timestamps to the output messages, which really helps to get a feel for how often the unit drops during recordings.<br />
<br />
I think I'll rebuild the code a little further still to add a simple socket listener data relay from the source, and possibly add a simple socket based 'start/stop' recording interface.<br />
<br />
For now I'm using dd straight from the video0 node to the fifo when not recording. When I want to record, I kill off the dd, start the capture, then relaunch the dd with it's input as the file being recorded. Once the capture app is able to stream the data itself, this will become a lot simpler =)<br />
<br />
The end goal is to have a simple recording app in C that can run all the time, with the ability to connect & monitor the current input remotely. After that, I can port some old java scheduler code over that I can use to issue channel changes to the set top box, and coordinate recordings etc.. Lots of fun ahead!<br />
<br />
<br />Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-81184619817193603382013-05-18T05:39:00.001-07:002013-06-02T12:53:04.215-07:00Raspberry Pi and the HD PVRI had a spare USB Hauppauge HD PVR, and a Raspberry Pi.. and had meant to get around to trying this for quite some time. There were even a few posts in the Hauppauge forums where people were wondering if the device would work well with the Pi.<br />
<br />
The HD PVR is a component video capture device, good for up to 1080i, with optical or stereo audio input, it has a hardware encoder onboard, so the video is sent to the Pi already encoded as an h264 transport stream. I'm using the original HD PVR 1212 here, there are newer Gaming oriented versions, and an HD PVR2 now.. but I don't know the state of the linux drivers for those.<br />
<br />
Once the Pi has the data, it can stream it, record it, or possibly use it as a tuner for a pvr backend.. Read on if you want to follow what I've tried so far..<br />
<br />
<a name='more'></a><h4>
Basic Setup & Testing...</h4>
<div>
The good news is quite how simple it is to setup, starting from the latest Raspbian image, the HD PVR was recognised immediately when connected, and created the <span style="font-family: Courier New, Courier, monospace;">/dev/video0</span> node.<br />
<br />
<i>Note: This assumes the HD PVR is the only video device connected, if you want to be sure.. use <span style="font-family: Courier New, Courier, monospace;">dmesg</span> after connecting the HD PVR, and use the name from the end of the line like..</i><br />
<span style="font-family: Courier New, Courier, monospace;"> hdpvr 1-1.3:1.0: device now attached to video0</span><br />
<br />
From there, a great simple test is..<br />
<span style="font-family: Courier New, Courier, monospace;"> cat /dev/video0 > test.ts</span><br />
<br />
Which should result in the blue lights of awesomeness engaging, and a file spooling out to disk.. leave it going a few seconds, then kill the process.. the resulting test.ts file should playback ok.<br />
<br />
If you are using spdif audio, then you'll need the v4l-ctl tool to toggle the input.. get that installed with <span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace;">sudo apt-get install v4l-utils</span><br />
<span style="font-family: inherit;">And then select the spdif input using the amazingly easy to remember command.. </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace;">sudo v4l2-ctl -d /dev/video0 --set-audio-input=2</span><br />
<span style="font-family: inherit;">The "2" at the end selects SPDIF, "0" would be the back stereo input, and "1" will be the front stereo input.</span><br />
<br />
The same tool can also be used to configure the HD PVR into VBR mode, and set the bitrates..<br />
<span style="font-family: Courier New, Courier, monospace;">sudo v4l2-ctl -d /dev/video0 -c video_peak_bitrate=16500000</span><br />
<span style="font-family: Courier New, Courier, monospace;"> sudo v4l2-ctl -d /dev/video0 -c video_bitrate=10000000</span><br />
<span style="font-family: Courier New, Courier, monospace;"> sudo v4l2-ctl -d /dev/video0 -c video_bitrate_mode=0</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: inherit;">If you are not using the Component input, there'll be more fun to have.. </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace;">sudo v4l2-ctl -d /dev/video0 --set-input=0</span><br />
<span style="font-family: inherit;">The "0" selects the component input, "1" will select Svideo, and "2" will select composite on the front.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">With all that out the way, the Pi should now be creating </span><span style="font-family: Courier New, Courier, monospace;">test.ts</span><span style="font-family: inherit;"> files that playback with audio & video. </span><br />
<br />
I'm just using VLC on Windows to playback, transferring the created file using WinSCP. Being H264, the files should playback pretty well on the Pi with the GPU doing the work.. but I haven't tested that yet..<br />
<br />
Reading around a lot online, I find people suggesting to avoid using <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">cat </span>as that will exit if the device produces any errors.. which will mean an interrupted recording.. the general suggestion seems to be to use <span style="font-family: Courier New, Courier, monospace;">dd</span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span>instead like this..<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace;">dd if=/dev/video0 of=test.ts conv=noerror ibs=1024</span><br />
The noerror bit should mean the recording will continue even if the device does something oddball.<br />
<br />
So now I had the device recording.. I wanted a way to watch the stream live..<br />
<h4>
Live Streaming...</h4>
Since /dev/video0 outputs a nice transport stream, this should be as simple as magically getting the data from the device on the Pi, into my VLC on Windows.. I'm deliberately not looking to transcode, or do any processing on the stream Pi side, as I want to keep that as lightweight as possible.<br />
<br />
The general consensus online is to use Netcat, a wonderful little tool that lets you pipe data in on one host, and pipe it out the other.. In theory, this should work like..<br />
<span style="font-family: inherit;"><b> Pi:</b> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace;">cat /dev/video0 | nc 192.168.0.X 6666</span><br />
<b><span style="font-family: inherit;"> Pc:</span></b><span style="font-family: Courier New, Courier, monospace;"> nc -l 6666 | "c:\program files(x86)\videolan\vlc\vlc.exe" -</span><br />
For this to work, you'll need to install netcat on the pi, and install it on windows, and have more luck than I did.. Every time the vlc player started getting data, the blue lamps would go out on the HD PVR, and the stream would stop. I tried reversing the netcat direction, tried using mplayer instead of vlc, tried using dd instead of cat, and eventually gave up.<br />
<br />
So, here's my alternative approach.. it's a little more entertaining.. but it gets there ;p<br />
<br />
First, we'll be needing apache, so<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace;">sudo apt-get install apache2</span><br />
<br />
Then we'll be creating an amusing small cgi script..<br />
put these lines into a new file at<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace;">/usr/lib/cgi-bin/video.cgi</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">#!/bin/bash</span><br />
<span style="font-family: Courier New, Courier, monospace;">echo -ne "content-type: video/h264\n\n"</span><br />
<span style="font-family: 'Courier New', Courier, monospace;">dd if=/dev/video0 conv=noerror ibs=1024</span><br />
<div>
<br /></div>
<div>
I'm using <span style="font-family: Courier New, Courier, monospace;">dd if=/dev/video0 conv=noerror ibs=1024</span> here instead of <span style="font-family: Courier New, Courier, monospace;">cat </span><span style="font-family: inherit;">as I found that otherwise the stream would interrupt & drop every so often. The lack of an output "</span><span style="font-family: Courier New, Courier, monospace;">of=</span><span style="font-family: inherit;">" argument means the data goes to standard output.. which from a cgi script is the remote process, VLC in our case.</span></div>
<div>
<br /></div>
<div>
This script will read from the video device and splat the data back to VLC.. or it would if it were allowed to read the video device, by default the permissions deny this.. so fix them with </div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace;">sudo chmod 777 /dev/video0</span></div>
<div>
<br /></div>
<div>
Now make sure apache is started <span style="font-family: Courier New, Courier, monospace;">sudo /etc/init.d/apache start</span> and tell VLC to access the stream at <span style="font-family: Courier New, Courier, monospace;">http://ip.address.of.your.pi/cgi-bin/video.cgi</span></div>
<div>
<br /></div>
<div>
Bing! magic live video =) Now.. what if we want to record AND stream?</div>
<div>
<br /></div>
<div>
Here I cheat a little.. we can't have 2 processes trying to read from /dev/video0 at the same time, so we'll copy the data from the device so the webserver can use it, and write it to disk at the same time.. I want to use dd for this, but the regular dd will only output to one place at a time, and while we can solve that with some usage of pipes, and use of tee there's an alternative version of dd that supports multiple outputs.. dcfldd so we'll get that.. </div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">sudo apt-get install dcfldd</span></div>
<div>
<br /></div>
<div>
Now if we use dcfldd, we could read from the video device, and write to disk, we'll write into a fifo for the webserver, so we don't eat space.. and the webserver won't touch our real storage file.. </div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">mkfifo ~pi/video.ts</span></div>
<div>
<br /></div>
<div>
This is a magic not-quite-a-file-really that lets people read from it while you push the data in the other end.. it's use is just like the regular dd, except with multiple "<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">of=</span>" arguments. Here we're outputting to the fifo, and to a file called <span style="font-family: Courier New, Courier, monospace;">myrecording.ts</span> (Remember the HD PVR creates large files, you can easily fill the sdcard, consider storing the recording on a usb disk, or network drive). We run this from a spare terminal.. as it will need to keep running relaying the data from the video device to the outputs.. when we want to stop recording, we can kill the process.</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">dcfldd if=/dev/video0 conv=noerror ibs=1024 of=~pi/video.ts of=~pi/myrecording.ts</span></div>
<div>
<br /></div>
<div>
And edit the last line of that cgi to be </div>
<div>
<br /></div>
<span style="font-family: Courier New, Courier, monospace;">dd if=~pi/video.ts conv=noerror ibs=1024</span></div>
<div>
<br /></div>
<div>
As it now needs to read from the fifo, not from the the video device.</div>
<div>
<br /></div>
<div>
Finally.. reconnect from VLC, and there's your live video feed, while it's storing the data out to disk =)</div>
<div>
<br /></div>
<div>
I'm still seeing some odd drop outs, which I need to debug further.. I suspect even with all the <span style="font-family: Courier New, Courier, monospace;">dd conv=noerror</span> usage, that something can still cause it all just to 'stop'. Eventually I suspect this means I'll need to write my own small bit of C that can read from the dev/video, and write out to disk / stdout / a socket .. but that's work for another day.. I'm playing with other ways of doing this.. if I find one that works.. I'll post an update.</div>
<div>
<br /></div>
<div>
<b>Update</b>: <i>still needs more testing.. but using this instead of dcfldd seems to help a bit.. (this is all just one line to enter.. )</i></div>
<div>
<i> </i><span style="font-family: 'Courier New', Courier, monospace;">dd if=/dev/video0 conv=noerror ibs=1024 | tee >(dd of=~pi/video.ts conv=noerror) | dd of=~pi/myrecording.ts</span></div>
Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com6tag:blogger.com,1999:blog-513334600397219012.post-5365387759063506592012-05-05T04:38:00.002-07:002012-05-05T04:38:11.678-07:00Bye Bye Sky!Its 2012.. and I'm swapping from satellite tv.. to cable tv.. why am I mumbling about this here? because there's a vaguely interesting story.. and possibly some useful info for any Sky users.<br />
<br />
<br />
<a name='more'></a><br />I've been with Sky since around 2001.. I remember the day I had the dish installed, not just a minidish, but an Arcon Sweetie, on a motorised mount.. scanning the skies for French, and English tv.. Working from a list of channels & frequency info.. we gradually stepped the dish across the satellites, stepping through the channels, to setup the ones we wanted to keep. Except all of the channels seemed to be carrying some sort of documentary, from New York.. and then in the background behind the presenters, we saw what's now been repeated worldwide.. it was no documentary, the date was September the 11th, and it really didn't matter which channel we tried to tune, the images were all the same.<br />
<br />
Back then, we had the Sky box hooked via an original Series 1 Tivo, the only model ever to be sold as a standalone unit in the uk.. it was pretty reliable, and with the addition of a turbonet (later cachecard), recordings could be moved on & off the box for watching later, as the disk got full.<br />
<br />
When Sky rolled out their own PVR, "Sky+" we ignored it, and kept using Tivo.. the season passes, wishlists, and community around the Tivo, along with its ability to be tweaked.. made it a far better option..<br />
<br />
But time rolled on.. and Tivo+Sky was becoming problematic.. Sky didn't like the idea that the dish could move to watch something else.. and Tivo was getting slower as it's disk space got larger.. and the inability to record/pause French TV, was becoming more pronouced the more we got used to the PVR.<br />
<br />
I'd started trying to build a replacement for Tivo, and finally with Vista, a Media Center became available that supported satellite, and Freeview, and analog, within the same UI. Vista became Win7, and Media Center became our main window to the world. (More info <a href="http://dwellertech.blogspot.co.uk/2011/05/media-pc-past.html">here</a> )<br />
<br />
Then HD came along, we updated to Sky HD, and used the HD PVR to view & timeshift in HD.. at the sametime, we moved the entire collection of boxes from under the TV to a spare room, and reconnected it all to the TV via HDMI over Cat5 extenders.<br />
<br />
The output was split via a HDMI matrix, able to watch the media pc, or the sky box, and later an xbox 360. This setup worked well, even letting a TV in one room watch something, while the Projector in another was watching a film.<br />
<br />
But then, as I mention <a href="http://dwellertech.blogspot.co.uk/2012/03/moving-set-top-boxes.html">here</a> I ended up with a Virgin Tivo cable box, mainly because I'd wanted faster internet access.. and it made sense to connect the Tivo to the Media PC, and leave the Sky box connected to the HDMI switch, as the Sky box, with "Anytime+" made a great way to watch movies on demand in the evening.<br />
<br />
That concludes the backstory.. ;p and brings us to why I am now, after over 10 years of being a Sky customer.. ditching them.<br />
<br />
I tried to view my Sky box.. and could not.. upon attaching a scart cable, I'm greeted with a message saying my "display is not HDCP compliant".<br />
<br />
This is amusing, and slightly insane, since both Tivo, a Sony BluRay, and the Media PC all require HDCP, and have no issue with any of the displays attached to the HDMI switch. So I try connecting the Sky box directly to the TV via an extender.. 'not HDCP compliant' .. orly? connected it directly with a cable, and it works fine.. umm??!!!<br />
<br />
Further investigation showed that my Sky box has become incapable of functioning on anything other than a direct cable connection to any of my displays. (The very same displays it declares to be HDCP non compliant through the switch). Amusingly, connecting Sky to an A/V Amp, via HDMI, and then connecting the HDMI output of the Amp to the switch, works fine..<br />
<br />
My Sky HD box is well out of warranty.. and I pay for one of Sky's "Top Tier" packages.. so I gave them a ring.. having heard that people cancelling often get offered 6 months free.. surely they'd see that fixing my problem would mean retaining my subscription ?<br />
<br />
But no, apparently "Sky Policy" prevents them offering a customer of 10 years, more than a 25 pound reduction on a 65 pound call out fee.. to obtain an engineer, who 'may replace the box if he agrees its faulty'. Great.. given the box works fine on a direct cable, I can guess where this one would lead.<br />
<br />
So I phone the Cable company, and find I can add a second Tivo, and Sky Movies, to my account with them, for less than I am paying Sky for the equivalent.<br />
<br />
I call Sky to cancel, and surprise, the guy tries to get me to agree to a (now free) engineer visit, to 'replace my box if its faulty'.. too little .. too late.<br />
<br />
I'm looking forward to integrating the 2nd Tivo from the cable company next week.. and I'll put the Sky box up on ebay.. (since most people will only ever hook it to a tv anyways).<br />
<br />
Technical Info:<br />
Sky Box that didnt like HDCP : DXR780 Model#R003.047.25.00P Version#4F3001<br />
<br />
Also tested a DXR890, which worked fine through the switch, also tested 2 other HDMI switches that used to function fine with Sky. Results were consistent, the ONLY box that had issues, was the DXR780.. everything else functioned perfectly.<br />
<br />
In conclusion, if you've come here, because you have a Sky box, that's having issues working through an HDMI switch.. a newer Sky box will likely fix the problem.. but you may find it easier to switch provider than convince Sky to do that for you.. especially if its out of warranty!Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-75617789736815695532012-03-29T03:34:00.002-07:002012-03-29T03:34:36.044-07:00Reading the Sky Planner...Easy one this.. Press Guide, then Green, and "Hey Presto" you are at your planner. Just use the old Mark One Eyeball to make sense of all that data.<br />
<br />
Of course, if you want to do something with that data.. say track which Films you have recorded, how long they were, and how much space they took up.. you'll need to link the Mark One Eyeball to some form of data storage, ideally accessible as at least text.<br />
<br />
After you've written down the content of your planner a few times, you'll realise that a) it's dull, and b) it's very dull. The solution is obviously to automate it.. but how??<br />
<br />
<a name='more'></a>Well, Sky doesn't provide a nice API to query their set top boxes, in fact, Sky doesnt provide <b>Any</b> API to query with, which leaves us back at emulating being a human.<br />
<br />
So, start by gaining control of your Sky box, record the remote up/down/left/right/select/back/guide/green button presses into a usb controller IR sender. I'm using the old MCE USB emitter, but I've had great success with a RedRat USB IR Dongle too. For now, lets just assume that you've achieved that bit, and now have the ability to send button presses from a script.<br />
<br />
Use the script to call up the planner, and navigate down through each entry, entering each one, and then backing out to the list. Stay 'inside' each planner entry for a few seconds.<br />
<br />
Then, record the video output of the script executing. For what we're doing, the old s/video or scart output is fine, hook that up to the PC, and have it record the 'walk through the planner' to an avi, or mpeg, or ts, or whatever format you plan to use.<br />
<br />
Next, we plan to use AviSubDetector to perform OCR on the recorded video. After all those planner entries are just like subtitles.. right ?<br />
<br />
The 1st issue here is most likely that the file you've recorded isn't an AVI (well, they are getting kinda old now!) so we'll want to make it pretend to be one. (Even if it IS an AVI, we'll still want to do this, as we're going to get creative).<br />
<br />
So, we install AviSynth, I used 2.5.8, and create a new .avs file in the same dir as the video recording, and give it the content..<br />
<br />
DirectShowSource("MyPlannerWalkthrough.TS", fps=25)<br />
<br />
Where 'MyPlannerWalktrough.TS' is the name of the video file you recorded.<br />
<br />
Then we open the avs file (not the original video!) in AviSubDetector, and hit the preview button (below the OCR(Experimental) button) to see our video.. the long slidey bar under the preview button can be used to yoink the position in the video around.<br />
<br />
If like me, loading the video through AviSynth caused it to become 'upside down', then change the line to be..<br />
<br />
FlipVertical(DirectShowSource("
MyPlannerWalkthrough.TS", fps=25))<br />
<br />
Which does pretty much what it says on the tin.<br />
<br />
Ok, now if you happen to have the ability to record in high def, you'll notice that AviSubDetector seems best optimised for standard def video, and you either need to go record it again in SD, or cheat & resize the video on the fly.. say.. like this..<br />
<br />
FlipVertical(DirectShowSource("
MyPlannerWalkthrough.TS", fps=25)).Lanczos4Resize(960,540)<br />
<br />
Even if you don't need to resize it, bear in mind all the x/y offsets in the rest of this post, will be assuming a 960,540 framesize.<br />
<br />
Great.. so now you have a video in AviSubDetector.. go ahead and set up the crop to focus on the two lines that say the recording name, and length information, when 'inside' a planner entry. (Use the yoink bar to find a place where the recording is inside a planner entry, then use the settings tab, and adjust the crop sliders to highlight just those 2 lines of text.<br />
<br />
If you run in OCR mode now.. everything will go horribly wrong. Because the Sky UI is using colors on those two lines that means sometimes deep blue is the font color, and sometimes it's the background color, and AviSubDetector doesn't like that. So.. we'll fix it in AviSynth.<br />
<br />
<br />
<span style="font-size: x-small;">vid = FlipVertical(DirectShowSource("MyPlannerWalkthrough.TS", fps=25)).Lanczos4Resize(960,540)</span><br />
<span style="font-size: x-small;"><br /></span><br />
<span style="font-size: x-small;">line1 = Crop(vid,0,290,960,30)</span><br />
<span style="font-size: x-small;">testForLine1 = Crop(line1,570,5,16,20)</span><br />
<span style="font-size: x-small;">invertedLine1 = Invert(line1)</span><br />
<span style="font-size: x-small;">line1 = ConditionalFilter(convertToYV12(testForLine1), line1, invertedLine1, "AverageLuma()", "lessthan", "100")</span><br />
<span style="font-size: x-small;"><br /></span><br />
<span style="font-size: x-small;"><br /></span><br />
<span style="font-size: x-small;">line2 = Crop(vid,0,320,960,30)</span><br />
<span style="font-size: x-small;">testForLine2 = Crop(line2,570,5,16,20)</span><br />
<span style="font-size: x-small;">invertedLine2 = Invert(line2)</span><br />
<span style="font-size: x-small;">line2 = ConditionalFilter(convertToYV12(testForLine2), line2, invertedLine2, "AverageLuma()", "lessthan", "100")</span><br />
<span style="font-size: x-small;"><br /></span><br />
<span style="font-size: x-small;">line1 = Levels(line1, 122,1,200,0,255)</span><br />
<span style="font-size: x-small;">line1 = GreyScale(line1)</span><br />
<span style="font-size: x-small;">line2 = Levels(line2, 122,1,200,0,255)</span><br />
<span style="font-size: x-small;">line2 = GreyScale(line2)</span><br />
<span style="font-size: x-small;"><br /></span><br />
<span style="font-size: x-small;"></span><br />
<span style="font-size: x-small;">vid = Overlay(vid,line1,0,290)</span><br />
<span style="font-size: x-small;">Overlay(vid,line2,0,320)</span><br />
<br />
<span style="font-size: x-small;"><br /></span><br />
That rather scary looking bit of code selects each of the lines using Crop, then extracts a small portion of each line (using Crop on the cropped part), then uses a ConditionalFilter to evaluate the "Luma" of the the small portion, and Inverts the colors for that segment if the small portion has Luma<100. Finally the color range in the resulting line is cropped, making light greys become white, and dark greys become black, then the entire line is flipped to greyscale (which thanks to the level adjust is more like black and white).<br />
<br />
What did all that achieve? Pretty much it means those two lines from the UI will now always have white text, on a black background, regardless of what colors they started with.. pretty neat =).<br />
<br />
Now when running AviSubDetector, it's pretty happy with the result.. Almost happy enough that this could work.. except for transitions.<br />
<br />
Transitions are when the video moves from the Planner, to the Planner entry and back. During these periods there's a little cross-fade present from the set-top box or the video encoder.. AviSubDetector isn't a great fan of these, and tries to OCR the fuzzy mix of both sets of text muddled together. Worse, once it's decided that was the important thing to do, it then ignores the 'clean' data, because it didn't differ enough from the fuzzy gunk.<br />
<br />
Ideally there would be some way to add a slight delay to the AviSubDetector to make it wait a while after a change.. but I gave up looking, and instead made the 1st 3 lines of my AviSynth script say..<br />
<br />
<br />
<span style="font-size: x-small;">vid = FlipVertical(DirectShowSource("2012_3_28_15_20_38.TS", fps=25)).Lanczos4Resize(960,540)</span><br />
<span style="font-size: x-small;">vid = ChangeFPS(vid, 0.5, false)</span><br />
<span style="font-size: x-small;">vid = ChangeFPS(vid, 25, false)</span><br />
<span style="font-size: x-small;"><br /></span><br />
This basically tells AviSynth.. drop the framerate so that there is only 1 frame every 2 seconds, do this by throwing away all the other frames. This would be a pretty short video.. (and AviSubDetector is kinda expecting subtitles, at human speed, not per frame text).. so we then blow it back up to 25fps, which is done by duplicating all the frames.. Overall effect is the video now only updates once every 2 seconds, which means the chance of hitting a transition is a) very small, and b) if it does occur, 2 seconds later there should be a big enough change to cause a re-read.<br />
<br />
With these changes in place, I almost had a clean read of the entire planner.. except one recording, which had a very short info .. just "98mins" turns out that made it too small for the OCR engine to care about.. this was caused by the Block Count being too high, tweaking it down to '1' (use Settings Tab, tick 'All(text)' and edit at panel at base of screen) solved that nicely.<br />
<br />
So there you go.. how to 'read' the Sky Planner. Enough to discover the Name & Length of every recording.. Handy.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-69699349401990942742012-03-27T05:35:00.001-07:002012-03-27T05:35:47.012-07:00WHS: The referenced account is locked outAfter a brief, but entirely planned power outage that lasted for a good few hours, I discovered that the home PCs could no longer 'see' the home server.<div>
<br /></div>
<div>
Symptoms were an unhappy grey looking house in the systray, and any attempt to access the shares resulted in a cryptic message saying "the referenced account is locked out".. Remote desktop to the server worked, as did opening the server console.. </div>
<div>
<br /></div>
<div>
Thankfully, there was a very easy fix...</div>
<div>
<a name='more'></a>It turns out that the clocks on the accessing pc's have to be 'fairly close' to the clock on the home server. </div>
<div>
<br /></div>
<div>
In my case the Server had forgotten all time when powered down (it's an old server, but then so will be most Windows Home Server v1's) as its ickle watch battery thing has died years ago. </div>
<div>
<br /></div>
<div>
So I used the remote desktop to the server, and brought it forward from 1st Jan 2005, to the current day, and bing, the little house went yellow, and my shares work again.</div>
<div>
<br /></div>
<div>
I had a quick google, and the suggestions for fixing this can vary a lot from unlocking the account on the server, to making sure the passwords still match, to advice suggesting to wait a while, because lockouts can lock you out for a period of time.. thankfully there was even one guy on a microsoft forum that gave the the idea to check the server clock.</div>
<div>
<br /></div>
<div>
So if you've recently rebooted your WHS install, and now can't connect to it.. check the date & time on the WHS via remote desktop, and check if it's correct.</div>
<div>
<br /></div>
<div>
In retrospect, I should have twigged that something like this would happen right from when I realised the server wasn't responding to ping, and checked it to found it stuck at the BIOS saying 'CMOS checksum invalid, F1 to continue or DEL to enter BIOS'. I chose to reload setup defaults in bios & let the system boot.. but I won't take all the blame, Microsoft failed here on usability, a nice message suggesting "Unable to Authenticate User due to excessive time difference between server and client".. would have been a little more useful, than an irrelevant message saying "user is locked out".</div>
<div>
<br /></div>
<div>
<br /></div>Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-87659709993468423272012-03-16T02:15:00.000-07:002012-03-16T02:15:14.921-07:00Moving set top boxes..My current Media PC records from 2 Freesat tuners, 2 Freeview tuners, and an HD Satellite Set Top box via a Hauppauge Colossus. This setup has served me well for quite a few years now, and there was no real reason to change it, until Cable broadband arrived in my area.<br />
<br />
Why should Cable broadband mean changing the media pc? Well, the DSL line can only get 4mbps, which isn't a great speed, but Cable can offer me 50mbps. Once you pay the nice Cable people for a fast line like that, it doesn't become that much more to add Cable TV to the bill. And I kinda figured I'd add the Cable set top as a 6th tuner to the media pc...<br />
<br />
<a name='more'></a><br />
Of course, time has this habit of rolling onward, and nearly 2 months after the Cable was installed, there we were, with the same old setup, and a Cable box on a 2nd HDMI input.<br />
<br />
The more I thought about it, the more I thought that the reason I wasn't adding the box into the Media PC's line up, was because I really wasn't looking forward to the time it would take to resolve all the channels into the guide and appropriately merge the sources together. Plus, I'd kind of gotten used to having a 2nd box that could be used when the Media PC was 'busy' ;p<br />
<br />
So I decided to swap the boxes over.. and have the Media PC use the Cable box, rather than the Satellite box.. that way (since the Satellite box had a movies subscription) .. I could work on another project I have in mind with the one no longer in use.<br />
<br />
The Media PC uses a batch script to change the channel on the set top box.. today this is sending remote control signals, but the new box supports changing via ethernet.. much nicer, and in theory, will remove those oh so annoying channel mischanges.. A quick hunt around found a nice program that could do the channel changes, but there was a slight issue that the Media PC believed it was still talking to the Satellite box, and the channel numbers don't line up..<br />
<br />
As I really wanted to avoid redoing the channel line-up on the Media PC, I opted for another solution.. creating a map of the channel numbers between the two boxes, then updating the script to do the mapping when a channel change was requested. This almost worked, except the script took too long to resolve the channel mapping..<br />
<br />
I did what any good geek would do, and downloaded Visual C++ express, and converted the script into a small exe, and then for good measure added the ability for the exe to send the channel change too..<br />
<br />
If you happen to be in the incredibly small number of people trying to do the same thing (move from a setup where a batch file changed a channel, to having it remap and send the channel via tcp/ip) I'm happy to share the code =)<br />
<br />
Result: Media PC has been recording from the Cable box now for weeks, without a missed channel change! it's totally oblivious to the change of set top box.<br />
<br />
Now to get back to that other project with the satellite box..<br />
<br />Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-90385276249341514412012-02-07T00:12:00.000-08:002012-02-28T02:12:19.125-08:00Converting all my Amiga Disks..The time has come to finally get rid of all my 3.5"DD floppy disks.. they are taking up valuable cupboard space that I need back for shinier and more recent items. I know that <i>some</i> of them have data on that I'd like to keep, source I wrote back in the late 80's / early 90's.. midi files, soundtracker / octamed tunes.. The problem is figuring out which disks..<br />
<br />
You see since I stopped using the Amiga regularly (this sounds like a meeting of Amiga Anonymous, of which I'm sure there'd be many many members ;p).. I've "looked after" the disks by storing them in some large wooden crates, I've got 6 of these crates, each holding around 500 disks, plus a couple of large shoe boxes, and an old 80 capacity smoked plastic lid box, each crammed as full as can be.. all in all there's somewhere around 5000 disks.. That's rather a lot to go through.. especially when you didn't label many of them appropriately..<br />
<br />
<a name='more'></a>Around 3 years ago, I tried reading the disks manually.. I made it through a hundred or so, before I realised my brain had gone numb, and I was at risk of chewing my tongue off and needing replacement eyeballs. In short, it was one of those tedious repetitive tasks that everyone really enjoys ;p<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace;"> 10 Insert disk, run capture program<br /> 20 Wait a couple of minutes<br /> 30 Eject disk<br /> 40 Insert new disk, run capture program<br /> 50 Rename last captured disk<br /> 60 Goto 20</span><br />
<br />
Ideally line 60 would read<br />
<span style="font-family: 'Courier New', Courier, monospace;"> 60 If bDisksRemaining Goto 20 Else Print "All Done!"</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
But that would just be hopelessly optimistic, given the quantity of disks to get through.<br />
<br />
So roll forward to 2 years ago, when I rediscovered the Lego Mindstorms kit I had sitting in a cupboard (probably couldn't find it due to all the floppy disks). I had an idea that if I could get it to do the Insert/Eject part, that pretty much I could deal with the rest 'later'.. and that the entire process could be automated.<br />
<br />
Cue the fantastic Lego Mindstorms Floppy Autoloader..
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJRx2C6w_D-GsO4FkVKb2hPC7hEDj_vamiBy3v7ZKU3Q6pL-EiL-7PodNNpvgGl7P6JEKXpuJelwiAgshQFwcHQURHmsAZLtk-xV31E6URWSG0Wkf8_aQ0OGFhrpCPjlt9J3jyEUoFnsQr/s1600/IMG_2256.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJRx2C6w_D-GsO4FkVKb2hPC7hEDj_vamiBy3v7ZKU3Q6pL-EiL-7PodNNpvgGl7P6JEKXpuJelwiAgshQFwcHQURHmsAZLtk-xV31E6URWSG0Wkf8_aQ0OGFhrpCPjlt9J3jyEUoFnsQr/s320/IMG_2256.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDnRNJ2Ri6Fjs9JsHXLqsJ374dbjmGDc2F3MV_Z_DQLX_6Zx6H25PnEAtdzxHrlwiIf9yWHAqFVUg8a3cWPBp1y5_m6MBh-L-yI30-xq2qTqWZpQ1JAq22-z52VoHUDHNKg6IUuz7BIWz1/s1600/IMG_2258.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDnRNJ2Ri6Fjs9JsHXLqsJ374dbjmGDc2F3MV_Z_DQLX_6Zx6H25PnEAtdzxHrlwiIf9yWHAqFVUg8a3cWPBp1y5_m6MBh-L-yI30-xq2qTqWZpQ1JAq22-z52VoHUDHNKg6IUuz7BIWz1/s200/IMG_2258.JPG" width="120" /></a>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZLLrIbDjrE7sFbH8S1aphOMxchfIdAUhIq08JCei8TsCgNcgA5jXRLqMqhrPGiCwg1gLsK6V9w6BEevu3dKD1gvumBSlgwuOPg3d6Ze6aHr0Mf73GQuJ5pBVKDvonJXQoA4b72rHrUZSC/s1600/IMG_2261.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZLLrIbDjrE7sFbH8S1aphOMxchfIdAUhIq08JCei8TsCgNcgA5jXRLqMqhrPGiCwg1gLsK6V9w6BEevu3dKD1gvumBSlgwuOPg3d6Ze6aHr0Mf73GQuJ5pBVKDvonJXQoA4b72rHrUZSC/s200/IMG_2261.JPG" width="120" /></a>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh81LfIXZzkFzAVs_R0HT8VKXkF3SGXKxH35V1V_Jjyb_3TyVONMmeS_gwSWCHUTlDInl4mhn0Y0E1hKfJTio4QniQQSaGhKVY1Xi92nX7nyT0mmT2OYaGjMzRd4qz16pr9aBD3e39tplwu/s1600/IMG_2263.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh81LfIXZzkFzAVs_R0HT8VKXkF3SGXKxH35V1V_Jjyb_3TyVONMmeS_gwSWCHUTlDInl4mhn0Y0E1hKfJTio4QniQQSaGhKVY1Xi92nX7nyT0mmT2OYaGjMzRd4qz16pr9aBD3e39tplwu/s200/IMG_2263.JPG" width="120" /></a>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEbxrl4D6oNQDPndrF8zywStSOnS5HCc9wTzb0VGIoLTIIc7ee35BrHxrD9eRLJ4sgxOl6HYr8SNNSTvMZSKVP1V6fT-9GG9u0_IRNiwZ6BSLGDFj2S3MSEe1jio5nf_g3j4nnerE24J2x/s1600/IMG_2269.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEbxrl4D6oNQDPndrF8zywStSOnS5HCc9wTzb0VGIoLTIIc7ee35BrHxrD9eRLJ4sgxOl6HYr8SNNSTvMZSKVP1V6fT-9GG9u0_IRNiwZ6BSLGDFj2S3MSEe1jio5nf_g3j4nnerE24J2x/s200/IMG_2269.JPG" width="120" /></a>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEis1awxnwRIyrSVtuzFpVKTVavlOVmq4SQLJSbTyTD2gr9ybQKbw0Z5EhN-fQFKQ7L86cxJTmvbWXtZNYw-Ks_lazg6n0DD9t1JMb1vMGrpv7l9e4qAoauMZWESoauRV3nlk-MBBhkegmp_/s1600/IMG_2284.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEis1awxnwRIyrSVtuzFpVKTVavlOVmq4SQLJSbTyTD2gr9ybQKbw0Z5EhN-fQFKQ7L86cxJTmvbWXtZNYw-Ks_lazg6n0DD9t1JMb1vMGrpv7l9e4qAoauMZWESoauRV3nlk-MBBhkegmp_/s200/IMG_2284.JPG" width="120" /></a>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdy9Oh_T5gQMFXImgMWzRbQ2S_XWTczqb4KPa29DVX4wH_ejsBrheiVlbgzpoP-U8UVRJz0FRujHMPSMhI2BJZOLruTtsBPH0nFEWvlSpi4WKUwXjREST7c3fsHQHamif1MUfI0HJDHBri/s1600/IMG_2288.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdy9Oh_T5gQMFXImgMWzRbQ2S_XWTczqb4KPa29DVX4wH_ejsBrheiVlbgzpoP-U8UVRJz0FRujHMPSMhI2BJZOLruTtsBPH0nFEWvlSpi4WKUwXjREST7c3fsHQHamif1MUfI0HJDHBri/s200/IMG_2288.JPG" width="120" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
Building this was an entertaining challenge, as Mindstorms has a few things working against it for designing a Floppy Autoloader..</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ul>
<li>Lego only supports fixed divisions of it's dimensions, which are not aligned with 3.5" media.</li>
<li>The Mindstorms kit has only 3 motors (ok, 2, but I found a spare)</li>
<li>The Mindstorms kit has 2 push sensors, and 1 light/color sensor.</li>
<li>The RCX is fairly dim.</li>
</ul>
Thankfully you can cheat a little with the 1st using the rods to create adjustable gates etc, and the RCX can be reflashed to run <a href="http://lejos.sourceforge.net/">Lejos</a>, a cut down Java JVM, which even lets it communicate back to the host while it's running.. handy, since there's no way that RCX is ever going to read disks itself.<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The motors & sensors let to the odd design above, rather than trying to create a set of lego 'fingers' to insert, and eject & extract the disk, the drive rotates, allowing just one linear motion to insert the disk, cause eject, and extraction for free via gravity.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Did it work? yes. Was it reliable enough to use? Not really, the tolerances for accidentally getting 2 disks from the stack instead of one, were a little fine, and the Lego would drift over time.. after just a few disks, it would generally mess up. I'd make better progress manually.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="" style="clear: both; text-align: left;">
So I put the whole thing in a cupboard.. figuring I'd revisit it another day.. </div>
<div class="" style="clear: both; text-align: left;">
<br /></div>
<div class="" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2Pjaes9Bmto4_1UcIB0qdr5EA-Esf9sTeN05wIHpYhJl_kptaqS53NJymM3KtNiGtK48Ui86e9IOgFhx4U5UYYIPJtImY3mQTFRwvA-LSfrGQop96Yk14yEm1TlFdbPLcQn0HpYqX1YTs/s1600/cp2000.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="162" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2Pjaes9Bmto4_1UcIB0qdr5EA-Esf9sTeN05wIHpYhJl_kptaqS53NJymM3KtNiGtK48Ui86e9IOgFhx4U5UYYIPJtImY3mQTFRwvA-LSfrGQop96Yk14yEm1TlFdbPLcQn0HpYqX1YTs/s200/cp2000.jpg" width="200" /></a>And then I saw one of these on ebay.. It's a commerical disk duplicator.. with one of these, I could take each of my 5000 disks, and copy it onto a million more disks, until I could take over the world!!!</div>
<div class="" style="clear: both; text-align: left;">
</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Well.. either that, or I figured that regardless of what the unit was actually intended to do, at least part of its operation MUST involve sticking disks into a drive, and getting them out.. and it must be able to do that in a fairly reliable manner..<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="" style="clear: both; text-align: left;">
So I bought it.. and I won't revisit the surgery I performed on it here, as that's well covered in <a href="http://dwellertech.blogspot.com/2011/11/first-you-take-it-apart.html">this post</a> and <a href="http://dwellertech.blogspot.com/2011/11/then-you-put-it-back-together.html">this post</a>, and the more recent update <a href="http://dwellertech.blogspot.com/2012/01/woodworking-for-software-engineer.html">here</a>. </div>
<div class="" style="clear: both; text-align: left;">
<br /></div>
<div class="" style="clear: both; text-align: left;">
In summary, I ripped out the controller, replaced it with an arduino, a motor controller, and a <a href="http://www.kryoflux.com/">kryoflux </a> Mounted the entire thing on a custom stand to improve the eject chances with badly labelled disks, added an expanded capacity input chute, repurposed the solenoid from inside to act as a disk stop in the eject chute, wrote some code to control the whole thing, and added a camera to photograph the disk as it exited.</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="http://www.flickr.com/photos/bardweller/6817303085/" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;" title="Copypro CP-2000 with homemade disk chute by AblazeTheMage, on Flickr"><img alt="Copypro CP-2000 with homemade disk chute" height="320" src="http://farm8.staticflickr.com/7147/6817303085_8a9350a21a.jpg" width="240" /></a>This last part finally meant I could just load it up with disks and leave it to it. It will hold around 80 disks a time in the hopper, and each one is fed into the drive, ripped to stream & adf format by the kryoflux, then ejected. As it pops out, the solenoid blocks it from going too far, the camera takes its picture and then the solenoid fires letting the disk fall to the ground. Check out a few more pictures of the oddball woodwork & the unit <a href="http://www.flickr.com/photos/bardweller/6817300963/in/set-72157628480578825/">here</a>.</div>
<div class="separator" style="clear: both; text-align: left;">
So far in over 300 disks, it has had 1 disk cause a blockage (am I glad I carefully wrote the code to check that!!) which was due to a badly torn and peeling label. We'll ignore the 20 or so disks I had to redo after I turned out the light in the room where it was running & thus had nice black images instead of disk photos.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
It takes around 3 to 4 minutes per disk, so I've got a few weekends ahead of me with it running as much as possible.. meanwhile, I need to write some software to let me browse & catalogue all this data!</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<iframe allowfullscreen="" frameborder="0" height="315" src="http://www.youtube.com/embed/H5lkxSY7QsI" width="560"></iframe>Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com38tag:blogger.com,1999:blog-513334600397219012.post-73305404896227239672012-01-28T13:32:00.000-08:002012-02-28T02:12:34.896-08:00Woodworking for the Software Engineer...So.. I've been a bit busy recently, and the fun technical progress with arduinos, maples, serial FRAMs, and floppy autoloaders has been somewhat on hold.. plus there was that whole christmas thing ;p<br />
<br />
What have I been up to ?<br />
<br />
<a name='more'></a><br />
<br />
Well firstly.. there's the Rig to mount the autoloader.. after getting it all running, it quickly became clear I would need to build a way to stack more disks as input, and to collect the output.. then after a little more experimentation, I found that disks with anything more than a properly placed glossy label, would often not clear the exit when ejected.. seems that tiny extra bit of thickness is enough to slow the disk enough to prevent the eject spring being sufficient to get the disk clear of the unit.<br />
<br />
A bit of testing quickly showed tilting the unit (actually propping the back up on a couple of books) helped a load, but not enough.. standing it on its end helped, but meant I'd need to come up with something to push the disks horizontally into the unit.<br />
<br />
So, eventually, I settled on a 45' angle.. and some ill-prepared woodwork later, I ended up with this..<br />
<br />
<a href="http://www.flickr.com/photos/bardweller/6772018307/" title="Copypro CP-2000 with homemade disk chute by AblazeTheMage, on Flickr"><img alt="Copypro CP-2000 with homemade disk chute" height="375" src="http://farm8.staticflickr.com/7017/6772018307_3e16a93d71.jpg" width="500" /></a><br />
<br />
There are some more pictures of it if you click it & follow back to Flickr, but basically, the idea is the disks are fed in from top right, and there's lots of space under the exit spot to finish up something to snap a photo of the disk.. I'm not done with this one yet!<br />
<br />
The other little project was a bit bigger ;p<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFie12boR3hFee03arFeW627dgHhyxuCK5kG_tOD7F7zJ3D08cBXN_uXRNFguO2hTvEDq2fGa0ERJq7NSt_mnYNCTKZ95Z_etjsfjHMZ_E5j4n6nUVbGrCcs0N-DuC2HAoNAuIXM40JBjV/s1600/P1050122.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFie12boR3hFee03arFeW627dgHhyxuCK5kG_tOD7F7zJ3D08cBXN_uXRNFguO2hTvEDq2fGa0ERJq7NSt_mnYNCTKZ95Z_etjsfjHMZ_E5j4n6nUVbGrCcs0N-DuC2HAoNAuIXM40JBjV/s320/P1050122.JPG" width="240" /></a></div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnI_p4RH2JLFdSTr9WP40hZ6K0D4v9KpT48vY_UZN53vCE8MuR7Bf5MvgYClYXeeD30lIfCycI3saJOmENm9Uesbe4HBAO9LfC1L86xp0iNcOWS4lXZV60GRdRqeRqZErv47M9UtHF_zJ3/s1600/P1050134.JPG" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnI_p4RH2JLFdSTr9WP40hZ6K0D4v9KpT48vY_UZN53vCE8MuR7Bf5MvgYClYXeeD30lIfCycI3saJOmENm9Uesbe4HBAO9LfC1L86xp0iNcOWS4lXZV60GRdRqeRqZErv47M9UtHF_zJ3/s320/P1050134.JPG" width="240" /></a> The server cupboard was becoming more than a little messy, the cables had become a rather untidy mess, and I had to get it all sorted for the Cable guy..<br />
<br />
So I built this thing.. 23m of lumber, a bunch of metal corner braces, a rather large pack of long screws, some careful use of a chopsaw, and bingo.. one DIY media rack..<br />
<br />
Now this one is properly sanded, slides in and out of the cupboard, and has all the cabling neat & tidy.. each red fan is hiding a 4 bay hotswap sata cage, which are connected in sets of 5 into sata port multipliers, terminating in 4 esata cables running to the storage pc.. below that is the media pc, with twin dvb-s, twin dvb-t, and 1080i component capture from the satellite box..<br />
<br />
So.. what have I learnt from this brief journey into wooden engineering.. ?<br />
<br />
<ul>
<li>There's no undo button, drill the wrong place, and you might have to ditch the entire bit you worked on.. </li>
<li>Guessing angles, or lengths by eye, rarely works.. when it does.. it's "luck".. you are NOT improving.</li>
<li>Holding wood while dragging a saw across it, requires concentration, alternatively, you can discover how quickly you heal.</li>
<li>You cannot have screws occupying the same space inside wood, from different directions, even if that would make it all look nicer.</li>
<li>Sanding stuff makes LOTS of sawdust, and probably should only be done OUTSIDE</li>
<li>Long clampy things are awesome.. and can correct for all sorts of minor inaccuracies in both the wood, and the length you cut it to.</li>
<li>Do not kick drills you leave on the floor.. actually, make that, don't leave drills on the floor, it's a silly idea, even if it is the only empty space you have left after cutting all that wood up.</li>
<li>When drilling through sheet steel.. if the drill bit snaps, use the end stubby bit as a new improved less likely to flex and snap again drill!</li>
<li>If you have to sand more than 2mm off of something.. find a smaller bit of wood instead.. or carefully attempt to cut the desired amount off the piece, and then subsequently find a smaller bit of wood you haven't just ruined..</li>
<li>Drill bits, although designed to chew through wood, and glue spirals of wood into themselves, are not supposed to be cleaned of said spirals immediately after use. Oh, and drill bits get hot!</li>
<li>When cutting a circle out of 4mm mdf using a dremel, the grinding bit, ball point grinding bit, vertical drill bit, and grinding stone all are not good choices, the ceramic cutting discs will also either explode, or set fire to the MDF. The metal cutting disc works ok, but will also set fire to the MDF, solution, use the WoodVac to suck the smoke away, and use the metal disc.</li>
<li>Getting something right, takes lots of time, building a prototype does not mean you get to 'improve' it into the final thing.</li>
</ul>
<br />
<br />
Hopefully some of those might help if you decide foolishly to follow in my footsteps !!<br />
<br />
<br />
<br />Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com1tag:blogger.com,1999:blog-513334600397219012.post-14214216236922495082011-11-25T11:15:00.001-08:002011-11-27T03:53:56.866-08:00Then you put it back together..It's done.. =)<br />
<br />
I've completed reassembly of the CopyPro CP2000, with a new mainboard, consisting of a chunk of MDF, with an Arduino, Motor Board, Stripboard & breadboard, and Kryoflux attached to it..<br />
<br />
All in the same space of the original..<br />
<br />
<br />
<a name='more'></a>Here's a pic of it all before I made all the cables internal..<br />
<a href="http://www.flickr.com/photos/bardweller/6396321179/" title="Automated Floppy Imaging Station =) by AblazeTheMage, on Flickr"><img alt="Automated Floppy Imaging Station =)" height="360" src="http://farm7.staticflickr.com/6031/6396321179_5a43123a58_m.jpg" width="540" /></a>
<br />
And here's a quick video of it working, reading a few disks..
<br />
<iframe allowfullscreen="" frameborder="0" height="360" src="http://www.youtube.com/embed/mmTA9SH_VKk?hd=1" width="540"></iframe><br />
<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEittzmdTwp2gCYh-gV7mTClY8U5bnpgToJVSCuaXinTXvpo-rBYh_bodUnsDAYPSWZd-leEoonIfuRBUUxRA18ZrOeVKezHO04V7Unse-_W-tLcwozmtqsMVkDp796uwi9UEfNzVJVk7izU/s1600/P1040996.JPG" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEittzmdTwp2gCYh-gV7mTClY8U5bnpgToJVSCuaXinTXvpo-rBYh_bodUnsDAYPSWZd-leEoonIfuRBUUxRA18ZrOeVKezHO04V7Unse-_W-tLcwozmtqsMVkDp796uwi9UEfNzVJVk7izU/s200/P1040996.JPG" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">New Controller Board fitted</td></tr>
</tbody></table>
I ended up attaching some 2 & 3 pin latching headers to some stripboard, then attaching a row of female pin-header sockets along the same tracks. On the other side of the stripboard, I soldered cables from an old female Molex to the tracks, and pva glued the cables together to give them some rigidity.. (then cut the tracks a few holes down, so the power wasn't hooked to the sensor headers!!)<br />
<br />
This gave me a gap that was an ideal size to stuff a breadboard into, and with all the power & sensors now surfaced as female headers, ideal for hooking up with yet more jumper wires. <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFvgLsv189H9rDf2aR41pupXJynMGpit6gTsdJDCwrepAuvcss-v15wU8S3S6UVDH_9m1ctjX8xJ-LDHlU3FnDzSN90tyy76A0KCytKrGGu_qtMeDgZCk0k8FHXcdoiEaUlMU78H8CEJSA/s1600/P1040994.JPG" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFvgLsv189H9rDf2aR41pupXJynMGpit6gTsdJDCwrepAuvcss-v15wU8S3S6UVDH_9m1ctjX8xJ-LDHlU3FnDzSN90tyy76A0KCytKrGGu_qtMeDgZCk0k8FHXcdoiEaUlMU78H8CEJSA/s320/P1040994.JPG" width="320" /></a></div>
<br />
Plonked in some resistors to limit the current to the IR diodes, and fed the photo transistor outputs across to the Arduino pins. Quick test showed them reading 0 when unobstructed, and 1 when obstructed.<br />
<br />
<div style="text-align: right;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYq5Ms2e6ZxazJ9LEpwDCbjlLRNyCCpYVbYg2wsoBoRhlFvaUknsX4fTw4xZuul53iIUTC9p0A10Cq7mRm3NQIOgDUfliTsT-vEX6VZ09JsxgttF-aoVuG8g7HJOnIxIPR1AamsS5r9v18/s1600/P1040995.JPG" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYq5Ms2e6ZxazJ9LEpwDCbjlLRNyCCpYVbYg2wsoBoRhlFvaUknsX4fTw4xZuul53iIUTC9p0A10Cq7mRm3NQIOgDUfliTsT-vEX6VZ09JsxgttF-aoVuG8g7HJOnIxIPR1AamsS5r9v18/s200/P1040995.JPG" width="200" /></a></div>
The motors, I traced out the old board, and found the 12v input being fed to the L298 as the source voltage for the motors.. So I jumpered my 12v socket, across to the motor board power input. Thankfully, this particular board has the option to create its own 5v (for its logic) via an onboard regulator, so I let it do that. A quick test showed I had 2 pins for each motor channel, and if both pins were High (or both pins were low), then the Motor would be idle. Putting the pins up/down or down/up made the Motor go forward/backward. The Solenoid I was less sure, still 12v, but I wasn't sure which 'direction' to send it.. testing showed it just didn't really matter.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTGjncPcXsW-zoXKRLWLIZ_GkaTpoHKMZEHLkik42SgFmPd56r76xkinnzd3xDSAO5JbxCJ0kUs0qSs-uw8HYgbeqGah1m7qG-lG1GzWg46xMBNid9dvVtFS4hvoqki90Bgbyz7pM0lIfv/s1600/P1040962.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTGjncPcXsW-zoXKRLWLIZ_GkaTpoHKMZEHLkik42SgFmPd56r76xkinnzd3xDSAO5JbxCJ0kUs0qSs-uw8HYgbeqGah1m7qG-lG1GzWg46xMBNid9dvVtFS4hvoqki90Bgbyz7pM0lIfv/s200/P1040962.JPG" width="200" /></a></div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipDDJunss4JWZfbyqg_S3WwGA0WwCCyTHLOc083K032oYZZ4UTNuv5Sq2cUoa0IZoO-jIZ_iJlsEPY1UjtYhS3y88S5IfIldptV0oAfdwQj3AWsQNuFxruKsVLR237GDa6PEKEAs4IOaCy/s1600/P1040966.JPG" imageanchor="1" style="clear: right; display: inline !important; float: right; margin-bottom: 1em; margin-left: 1em; text-align: center;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipDDJunss4JWZfbyqg_S3WwGA0WwCCyTHLOc083K032oYZZ4UTNuv5Sq2cUoa0IZoO-jIZ_iJlsEPY1UjtYhS3y88S5IfIldptV0oAfdwQj3AWsQNuFxruKsVLR237GDa6PEKEAs4IOaCy/s200/P1040966.JPG" width="200" /></a>Tracing out a PCB isn't that hard.. just take some carefully aimed photos using a bright lamp, then import to Photoshop, align the layers, and off you go.. <br />
<br />
<br />
<br />
<br />
That left creating the code to control the device.. <br />
<br />
Turns out one sensor goes to 0 when the disk insertion carriage is at its furthest extents.. (this is kind of important, as the motor must be stopped then, to prevent it straining against the edge of the allowed motion).. slight issue that you can't really tell _which_ extent.. has it just put a disk in? or is it waiting to do that? <br />
<br />
I ended up moving the carriage forward/back a bit, monitoring the sensor, to figure out if it went 1/0 then it was waiting to put a disk in.. if it stayed 0/0, then it was fully forward after putting a disk in. Once the Arduino knew where it was, it could use that until the next power up.<br />
<br />
The insert/eject code was otherwise quite simple, to insert; move carriage forward until either the carriage is fully forward, or until the disk mouth sensor does a 1/0 transition.. to eject; bring the carriage back until you notice the disk mouth sensor go 0/1, or until the carriage is fully backward. (Originally, I tried just bringing the carriage backward, but that can result in the eject button being pressed too hard, and the drive tilted in a way that means disks don't exit cleanly). <br />
<br />
For both insert/eject, have to be very careful to track that the disk went in ok, and exited the unit ok.. the force the unit can generate to insert a disk is quite fearsome, and it's best it doesn't try to put two disks in the drive at the same time..<br />
<br />
I wrapped all that in a simple serial protocol, (serial over usb), and hooked that up to some Java using the rxtx library. <br />
<br />
Both the arduino & the java host s/w run mini state machines, tracking what they are attempting to do, and testing if it looks like that occured ok.. The host relies on the status coming back from the arduino to know if things are still ok, and coordinates performing that initial carriage position seek, inserting disks, invoking the Kryoflux disk imaging commandline tool, ejecting the disk, grabbing an image of the last ejected disk via a webcam, then inserting the next & repeating.. <br />
<br />
I just need to build the 'replacement' input & output chutes, as this unit lost its original ones at some distant point in the past. <br />
<br />
At the mo, I've seen it work through 10 disks, and create Stream & ADFs for each of them.. although I do have to stop wondering why I hear the catflap open/close each time it does an insert/eject.. it's most confusing, as I don't have a catflap anymore..Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-69118928189819295232011-11-18T13:39:00.000-08:002012-02-28T02:12:50.320-08:00First, you take it apart...Well.. my little project to read in my entire Amiga Floppy archive took another step forward, when an Ebay search found a 'CopyPro 2000'.. leading to the purchase of a bunch of related bits & bobs.. <br />
<br />
So now I'm waiting on a motor controller, some connectors, stripboard, I've been slightly busy attempting to read datasheets..<br />
<br />
What is a CopyPro 2000? why is it useful? Read on..<br />
<br />
<br />
<a name='more'></a><br />
<br />
Well.. its a rather industrial looking bit of kit.. intended to run as a standalone floppy duplicator.. here's a quick video of it in action..(safety guards have been removed for educational purposes..)<br />
<br />
<iframe class="twitvid-player" frameborder="0" height="360" src="http://www.twitvid.com/embed.php?guid=RL3VB&autoplay=0" title="Twitvid video player" type="text/html" width="480"></iframe><br />
<br />
Somewhere around 50 seconds in, you see it spit out the green disk it had been reading & remembering.. and grab a new black disk from the stack.. which it then wrote the green disks data to.. isn't that just a thing of beauty?<br />
<br />
No?<br />
<br />
Ah well.. clearly you haven't attempted to build a Floppy autoloader out of Lego.. I have.. and this thing is a touch more refined than my result =)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK1_Kg99EPaJ8T8mQXtxaNZaPL8gBc-Ydyo3WfiHrJtpJs6lrYAnhe1bmOsOi7FRCZQzh6nbOZ25WTcm7qGC7AHeQGFukGI6AkFZcLT_i8QnwiGfGUUJNint6bth3otb7aJPueHy8zoYAK/s1600/17174_291176482843_580642843_3590120_2189612_n.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK1_Kg99EPaJ8T8mQXtxaNZaPL8gBc-Ydyo3WfiHrJtpJs6lrYAnhe1bmOsOi7FRCZQzh6nbOZ25WTcm7qGC7AHeQGFukGI6AkFZcLT_i8QnwiGfGUUJNint6bth3otb7aJPueHy8zoYAK/s320/17174_291176482843_580642843_3590120_2189612_n.jpg" width="320" /></a></div>
<br />
<br />
Of course, the CopyPro didn't turn up all nice and working.. I got it pretty much sold as seen, without its normal floppy input chute, or output collector chute.. I figure these can be built from mdf later. More importantly, it arrived with a dead floppy drive fitted (eek! seems it's spindle motor was dead) and it didn't look too normal.. a Ye Data YD-701B-6331S .. with about 20 customisable jumpers. This could have been bad, if the jumper configuration meant the drive wasn't working like a 'normal' drive, my chances of seeing the CopyPro do it's thing could be pretty small.<br />
<br />
Given the large stack of floppy drives I have.. I tried a couple, and thankfully, it seems there wasn't anything too special about the old one.. or if there was, the controller was smart enough to figure it out.<br />
<br />
So that resulted in the quick video.. and proof that the "CP2000 8.4W" firmware is only smart enough to clone DOS formatted floppies.. testing with an Amiga disk quickly showed it wouldn't tolerate them as source disks at all. I tried throwing an email to the company who built it.. but they haven't replied.. not much chance they will either, since I don't really want to use it, but dismantle it.<br />
<br />
Heres the plan.. remove the old controller board..<br />
<a href="http://www.flickr.com/photos/bardweller/6354046593/" title="CopyPro 2000 Controller Board#1 by AblazeTheMage, on Flickr"><img alt="CopyPro 2000 Controller Board#1" height="375" src="http://farm7.static.flickr.com/6213/6354046593_28635e4d8b.jpg" width="500" /></a><br />
<br />
and have an Arduino take over it's duties.. well.. some of.. the floppy output will go to a PC, via a <a href="http://www.kryoflux.com/">Kryoflux</a> which is kinda like a finished-project version of my floppy reader, combined with some decent interpretation of the data in software.<br />
<br />
What did that brain-board have to do?<br />
<ul>
<li><div>
Connect to the floppy drive to read/write data.</div>
</li>
<li>Communicate with the front panel pcb to show status LEDs, LCD text, and get Button presses.</li>
<li>Read a sensor checking if the disk exit from the unit is blocked</li>
<li>Read a sensor at the disk mouth, seeing if a disk is stuck inserting/ejecting</li>
<li>Monitor a sensor determining if the insert/eject mechanism is fully retracted.</li>
<li>Drive a motor, performing the insert/eject.</li>
<li>Power a solenoid selecting the good/bad disk exit path.</li>
</ul>
So the arduino will just need to read 3 sensors, control a motor and a solenoid, and tell the host when its good to image a disk, wait for the host to respond, then kick the disk out the good/bad path depending on what the host responded.<br />
<br />
The main board of the CopyPro 2000 used a <a href="http://www.cse.dmu.ac.uk/~mgongora/Resources/L298N.pdf">L298</a> (visible top right, where the 2 white & red cables go to the thing with a nut thru it) to drive the Motor & Solenoid. Conveniently, that same controller is used at the core of many motor driver shields for the arduino.. even better, the one on this board has no heatsink, which means the power through it is pretty limited. (Although it does have the equivalent of a thermal cutout bolted to it.. but I suspect thats to stop it cooking inside the enclosure in a warm place..). One DFRobot Motor controller sourced off ebay.. <a href="http://www.dfrobot.com/index.php?route=product/product&path=51&product_id=66">(details)</a> .. The motor controller will take over driving the motor, and the solenoid.<br />
<br />
Next the sensors..<br />
<a href="http://www.flickr.com/photos/bardweller/6354780319/" title="Drive duplicator optical slot switch HOA1877-2 #2 by AblazeTheMage, on Flickr"><img alt="Drive duplicator optical slot switch HOA1877-2 #2" height="151" src="http://farm7.static.flickr.com/6059/6354780319_e0d87089c3_m.jpg" width="240" /></a><a href="http://www.flickr.com/photos/bardweller/6354924321/" title="HOA1877 .. showing orientation spot by AblazeTheMage, on Flickr"><img alt="HOA1877 .. showing orientation spot" height="240" src="http://farm7.static.flickr.com/6120/6354924321_b6d551e752_m.jpg" width="160" /></a><a href="http://www.flickr.com/photos/bardweller/6354772281/" title="Drive duplicator optical slot switch HOA1877-2 #1 by AblazeTheMage, on Flickr"><img alt="Drive duplicator optical slot switch HOA1877-2 #1" height="180" src="http://farm7.static.flickr.com/6058/6354772281_12511df4c6_m.jpg" width="240" /></a><br />
<br />
All 3 are optical slot sensors, (2x <a href="http://www.mosaico.com.br/Midias/Documentacao/HOA1877-002.pdf">HO1877/2</a> 1x <a href="http://datasheet.octopart.com/OPB830W55Z-TT-datasheet-5310085.pdf">OPB830W55</a>) each pairing an IR Diode with a Phototransistor.. the tricky bit being figuring out their orientation.. determined by the little white spot.. which I finally caught on camera with a bit of macro zoom.<br />
<br />
In theory, I just power up all the IR Diodes.. (datasheets for the components call for 16 or 17ma current for the diode), then slap a pull up (or down) resistor before (or after) the transistor side, and connect an ardino pin to the middle of the resistor/transistor.. configure it as input, and I should see it go high or low when the sensor gets blocked / unblocked.<br />
<br />
Then I just need to wire all that on some stripboard, provide pin headers for the sensors/motor, power input, carefully write some stuff to drive the motor, respond to the signals from the sensors, and I have my own host-controlled floppy autoloader.<br />
<br />
While it'd be great to try to reuse the entire front panel too.. there's a lot of work to figure out that panel, and it may just be simpler to remove it, and put my own LCD display with a few LEDs in there. (Plus there's the slight issue that I can't find a hex key small enough to detach it from it's mounts at the mo.. making it tricky to trace the circuits)<br />
<br />
Next update on this once the motor controller & other bits arrive.. this one should be fun!Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-24936242932519678682011-11-12T10:45:00.000-08:002011-11-27T03:54:19.675-08:00Arduinos Floppy Drives, and Maples..Time for some serious geekery documenting =)<br />
<br />
This tale starts with my other half getting intrigued by an arduino.. and picking up an Uno, with a whole exploratory box full of parts, and little projects to build. Then shortly after that, getting an arduino nano, and an ethernet module, with the aim of solving our garage door opening problems..<br />
<br />
This led to me to start looking at arduinos & MIDI, and lcd's, and thus I ended up with an arduino Mega. Then I got a little sidetracked, and started looking at how to read floppy disks, and things just kinda spiralled from there..<br />
<br />
<br />
<a name='more'></a>So, I should probably wind back a bit here.. Garage Door Opening problems? MIDI? Nano?<br />
<br />
Our garage door has an electric opener, which frankly, is a good thing, as it's largish door & pretty heavy, and this way we don't get wet opening the door in the rain. But, the controllers use radio, and just occasionally it would appear something interferes with the signal, leaving the door pretty firmly shut..<br />
<br />
The opener also has a switch to press to open the door, but oddly enough putting a switch on the wall outside the garage would be a bit insecure.. and with the garage being detached from the house, a new solution was needed.. Enter the arduino, a pair of ethernet-over-mains adapters, an ethernet module, some code, and a relay.. and bam.. pull up outside the house, sign into the house wifi from a smartphone, and click 'door open'.. or at least that's the theory.. the small pile of parts on the floor hint that this project may take a while =)<br />
<br />
Can see a preview of it all here.. the top is a switching regulator dropping the 9v to 5v, then hiding up top right is a 3.3v LDO regulator dropping the 5v further to 3.3v. The Ether Module is running off the 3.3v, the arduino nano off the 5v, and the phone showing the webpage coming from it =) Yes, that tiny blob with the green & red LED's showing is the entire 'computer'.. scary tiny =)<br />
<br />
<a href="http://yfrog.com/j210frj" target="_blank" title="yfrog.com - Image And Video Hosting"><img border="0" src="http://a.yfrog.com/img686/1836/10fr.jpg" width="480" /></a><br />
<br />
Ok, MIDI? .. totally ancient connector/protocol for hooking together synths, computers, etc.. but oddly still relevant even today.. great thing about MIDI is it's only serial data at 31250bps, with a pretty simple protocol, and, yes, entirely within the capability of an arduino.. hook the TX & RX up to a set of MIDI jacks, and soon, you end up doing oddball things like..<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="315" src="http://www.youtube.com/embed/rD_8P6LIz2w" width="480"></iframe><br />
<br />
Which wasn't really intended as a psychedelic light show, so much as a way to debug the data flowing without having any real display.. course, now, I have a pile of 16x2 character LCD displays.. which makes using a row of LEDs seem a little .. quaint..<br />
<br />
Right.. onto the floppy disks.. I've got quite the pile of Amiga Floppy Disks, and some time ago started building a floppy disk autochanger (with mindstorms!).. but I got wondering.. could an arduino pretend to be a floppy drive? I mean I know of floppy emulators, like the HxC, and the 3.5" form factor ones that take a usb stick / sd card.. and I know of other projects where it's been done.. so the problem should be pretty well understood..<br />
<br />
After a bit of reading around.. I got about this far..<br />
<iframe class="twitvid-player" frameborder="0" height="360" src="http://www.twitvid.com/embed.php?guid=5IQLE&autoplay=0" title="Twitvid video player" type="text/html" width="480"></iframe><br />
<br />
Which is to say, able to time the Index pulse, and move the head, and be able to understand where Track 0 is.. and even see some RDATA transitions..<br />
<br />
Course, that isn't an arduino there either.. I sort of upgraded along the way, partially because the Maple is a 72mhz board with 20k of ram, vs the Arduino Uno's 16mhz, and 2k, which makes holding a tracks worth of data pretty feasible. the other reason, would be accidentally putting 12v down the arduino Mega's 5v pin.. which didnt just kill it, it _Really_ killed it.. we're talking glowing ball of light carving a burnt canyon through the CPU and vaporising the 5v pin. No, I don't have a video of that..<br />
<br />
Getting this far required a bit of help, so thanks go to the few people left at work who understand things like floating outputs, and pullup resistors, for taking the time to explain that to a software engineer =)<br />
<br />
That gets me pretty much back to 'now', where the Maple is able to see the RDATA transitions, but I'm a long way off seeing them clearly enough to decode the data.. I've had some great email exchanges with the author of <a href="http://techtravels.org/amiga/amigablog/">http://techtravels.org/amiga/amigablog/</a> who's been patient enough to explain a few of the steps I'd become blind to, about how to decode MFM when capturing pulse widths.<br />
<br />
I've just got a cheap 8 channel usb logic analyser (up to 24mhz, samples stored at the host, so can hold an entire track worth of data.. ).. and now I need to spend some time creating some tools to do comparisons of what I grab with the maple, vs what the logic analyser sees.. maybe then I'll be able to debug where it's going odd... (plus I've got a 16 channel 'open logic sniffer' (<a href="http://dangerousprototypes.com/open-logic-sniffer/">http://dangerousprototypes.com/open-logic-sniffer/</a>) on its way from china.. that'll help too =)<br />
<br />
Until then.. I've spent some time reading the datasheets for the Maple, and although the nice friendly library that comes with the Maple doesn't support it yet, I've got Timer Input Capture via DMA up & running.. essentially this means the hardware will monitor a signal line, and every time it goes from high to low, it will copy the value from a hardware timer directly into my array.. all without me getting involved.. all rather freaky if you're used to Java ;p<br />
<br />
Sadly this won't solve the problem by itself, as with 20k of ram, even storing 8bit timer values, I'll only be able to hold around 16000 of them.. and I need to hold more like 56000 of them.. but still.. mebbe if the cpu isn't busy doing the grabs.. I can use it to do something with the data.. possibly push it down to an sdcard, or into a FRAM chip via SPI with DMA.. just not sure of the timings.. the floppy data is pretty unrelenting coming in every 4, 6 or 8us, not leaving much time to do stuff in between..Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com1tag:blogger.com,1999:blog-513334600397219012.post-58003683019395574792011-07-11T13:04:00.000-07:002011-07-11T13:04:07.247-07:00Busy busy busyWell.. life presses on.. and somehow I find it's almost a month since I posted here last..<br />
<br />
I mean.. just how? it's not like I visited another planet, or entered a tech free lifestyle growing corn & making bread.. is it?<br />
<br />
Well, here's some of what I've been up to..<br />
<br />
<a name='more'></a><br />
I got a bit sidetracked with postcode data, as the last blog post will say =).. was trying to get the FTTC info for uk postcodes into a nifty blog post with a google map (yay), and other such goodness. BT having placed the files on their site for people to read, and linked them & discussed them in their own forums, I figure it was worth a bash, since the data wasn't all that hidden any more.<br />
<br />
After playing with my area, (I'm one of the few cabs on an FTTC enabled exchange, that BT has elected to totally ignore.. roads all around me can have FTTC, but I get to sit here & drink beer instead.. ).. I started seeing about putting it in a blog post.<br />
<br />
Why ? well, I don't have my own host, and figured that a post with an embedded app would be kinda nifty, except blog posts need to be about 1mb or so, and the database of info was around 48mb.. ouch. Zip compressed thats still around 4mb, and my first attempt at domain compression left it around 2.3mb, but getting lower than 2mb looked tough.<br />
<br />
I remembered a trick from way back, coding games for the amiga & spectrum, I'd stored some info within the pixels of the gfx data. Info hiding in gfx isn't new, that's steganography, and a quick google search showed a few people in the pc demo scene had tried using PNG's for storage (lossless compression kinda important for data ;-) so jpeg is less than ideal). They were more interested in good PNG compression though, and had found some neat stuff about encoding data just in one channel (eg, the red channel) rather than using all the range.<br />
<br />
I tried it, and my 1 channel PNG came out pretty awesomely small. But blogger wouldn't accept it, apparently images hosted by blogger have a max of 1600x1600. Mine was about 3800x3800, that didn't bode well, except that I'd only used 1 channel, what if I encoded using all 3 (yes, I know there's sort of 4).<br />
<br />
A quick rehack later, and I find yes, I'd need an image of 1800x1800 or so to store in 3 channels.. not quite there.. I did try to use the alpha channel, but clearly I need to understand png encoding better, as that just caused all my data to be corrupt on decode. So I tweaked my data content.. I'd been trying to store neat javascript, but if I removed some of the less important formatting & handled that during the image decompression.. yup.. below the 1600x1600 limit =)<br />
<br />
Now I needed to decode it in javascript.. easy enough, load the image, rip the pixel data, store in string, eval etc.. except the images don't come from the same host as the blog text, and security stuffs in javascript say Noooooo if you try that (ok, they actually say some obscure error that needs google to tell you why it went wrong..)<br />
<br />
Jump in some google app engine thing that can load your image, and squirt back the data as json !<br />
<br />
And that's roughly where I'm at.. far enough along to prove the concept, I _can_ read all the data from a blogpost, with enough room left over for the map, the query api, etc.. and one day real soon now.. I'll put it up =) right after I stop looking at.. oh look.. there's a castle!!Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-32664699375889565052011-06-18T11:01:00.000-07:002011-06-18T11:04:36.191-07:00Compressing Postcodes..The UK has postcodes, which are of a similar purpose to zip codes in the US. You put a postcode, and a house number on an envelope, and in theory that's pretty much all that's needed to get the letter to the destination.<br />
<br />
I've never really put much thought into how they were constructed, until I met a stack of data that was indexed using postcodes as the lookup, very handy, but it was causing the data size to exceed a limit which meant I couldn't embed the data into a blog post ;-)<br />
<br />
I went hunting, and found a document explaining the structure <a href="http://interim.cabinetoffice.gov.uk/govtalk/schemasstandards/e-gif/datastandards/address/postcode.aspx">here</a>. I ran a quick scan over my data, discovering I had 121 postcode areas present in my list, some 129 sub bits of them, and some 4000 or so last bits =) Eg, PO1 3AX, PO being what I'd call an area, 1 being the sub bit, and 3AX being the last bit.<br />
<br />
I started playing around a bit and here's what I ended up with...<br />
<br />
<a name='more'></a>.. the area & sub bit being ~128ish, I figured I can deal with easy enough.. so I started looking at the usage of the last bits.. how much are they reused, how they vary, etc.<br />
<br />
First up, they get reused a LOT =) a quick hack with the data showed that one was reused 465 times, and the average reuse was around 200. Breaking that down I found that the total repeated usage (not counting initial usage) of all last bits was a rather large 799275 times.<br />
<br />
Splitting that out into brackets, I can see that the average wasn't being skewed by a few high reuses..<br />
<blockquote><span class="Apple-style-span" style="font-size: xx-small;"> <50:700 <100:561 <150:238 <200:278 <250:392 <300:721 <350:841 <400:277 <450:20 >449:1</span></blockquote>(thats 0 to 49 reuses, 50 to 99 reuses, 100 to 149 reuses etc..)<br />
<br />
Nice, so if I can find a way to shrink that 3 character extension, I could save a fair number of bytes..<br />
<br />
Easy enough, I just put all the known 3 character endings into a table, then use an index to refer to which one I want when I need it.. given there are ~4000 or so, I can get away with a 2byte index, which should mean I can knock a 3rd off the current storage.<br />
<br />
Of course, I need to get the entire table (and its expansion code) included, if the index is to be usable within a blog post. The simple approach to sending the entire table as a proper table takes a rather large amount of space.<br />
<blockquote><span class="Apple-style-span" style="font-size: x-small;">{"0AA","0AB","0AD","0AE","0AF","0AG","0AH","0AJ","0AL" ....</span><i>(etc)</i></blockquote>So, given the length of each ending is fixed at 3chars, I could just concatenate them all into a single string, and send that.. except that's pretty huge =)<br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><blockquote><span class="Apple-style-span" style="font-size: x-small;">"0AA0AB0AD0AE0AF0AG0AH0AJ0AL0AN0AP0AQ0AR0AS"...</span><i> (etc)</i></blockquote></div>I tried it anyway, and noticed a side effect of sorting the list, was that there were very very strong patterns within the long list of extensions, it was pretty much an incrementing counter, using custom units per column.<br />
<br />
With that in mind, I thought maybe instead of sending every postcode, I could send a postcode, and follow it with instructions on how to increment to the next one. I decided to try with just the last digit, and to send the amount to increment by encoded as letters..<br />
<blockquote><span class="Apple-style-span" style="font-size: x-small;">"0AAbcbbbbccccbbbbbcbbb*0BAbcbbbbcccbbbbbbbcbbb*0DAbcbbbbccccbbbbbcbbb" ...</span> <i>(etc)</i></blockquote>This approach would take 0AA, then use the 'b' to know to increment the last A by 1 to get 0AB, then use the 'c' to know to increment what is now a B by 2 to get 0AD, and so on and so on..<br />
<br />
But then I noticed the "bcbbbbccbbcbbbbbcbbb" data seemed to repeat a bit, although I was processing 121 postcode areas, I only had 8 distinct patterns for the incrementer strings. Well.. maybe I should send those in a table, then just put an index to that table after each base pattern.<br />
<blockquote><span class="Apple-style-span" style="font-size: xx-small;">p="bcbbbbccbbcbbbbbcbbb%bcbbbbcbbccbbbbbcbbb%bbbbbbbccccbbbbbcbbb%bcbbbbccccbbdcd%bcbbbbcccbbbbbbbcbbb%bcbbbbccccbbbbbcbbb%bcbbbbccccbcbbcd%bcbbbbccccbbbbbbbbbb*0AA50BA40DA50EA50FA50GA50HA50JA50LA50MR#0NA20OO#0PA50QA50RA50SA50TA50UA50VV#0WA50XA50YA50ZA31AA51BA51DA51EA51FA51GA51HA51JA51LA51ME#1NA51PA51QA51RA51SA51TA51UA51VV#1WA51XA51YA51ZA52AA52BA22DA52EA52FA52GA52HA52JA52LA52NA52PA52QA52RA52SA52TA22UA52WA52XA52YA72ZA53AA53BA53DA53EA53FA53GA53HA53JA53LA53NA53PA53QA53RA53SA53TA53UA53VV#3WA53XA53YA53ZA54AA54BA54DA54EA54FA54GA54HA54JA54LA54MH#4NA54OR#4PA54QA54RA54SA54TA04UA54WA54XA54YA54ZA55AA55BA55CH#5DA55EA55FA55GA55HA75JA55LA55NA55PA05QA55RA55SA55TA55UA55VV#5WA55XA55YA55ZA56AA56BA56DA56EA56FA56GA56HA56JA56LA56NA56PA56QA56RA56SA56TA56UA56VV#6WA56XA56YA56ZA57AA57BA57DA57EA57FA07GA57HA57JA57LA47NA57PA57QA57RA17SA57TA57UA57VV#7WA57XA57YA57ZA58AA58BA58DA58EA48FA58GA58HA58JA58LA58NA58PA58QA58RA58SA58TA58UA58WA58XA58YA58ZA59AA59BA59DA59EA59FA59GA59HA59JA59LA59NA59PA59QA59RA59SA59TA59UA59VV#9WA59XA59YA59ZA6";</span></blockquote>Well.. thats the entire data table there =) 1010 bytes, instead of 24k worth for the proper table (or 12k for the 'just a string' approach).<br />
<br />
Inflating the data from that String to an Array of Strings in Javascript takes another 500 or so bytes, which overall gives me about 90% compression for the table =) Yes it can probably improve a little from there, but there's a certain effort / reward ratio where I stop at =) Feel free to show me how it can improve further.<br />
<br />
Next up, I get to look at the first parts (The PO1 part of the example) .. I think I can shrink that a little by taking advantage of the fact that the data I'm processing has them in clusters, so I may be able to generate localized lookup tables within the tree, enough to potentially let me encode each postcode as just 3 characters.Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com1tag:blogger.com,1999:blog-513334600397219012.post-91127051272245055382011-06-07T00:18:00.000-07:002011-06-07T00:18:14.269-07:00on Hosting, and other such things..I remember walking along the seafront, back in the mid 1990's, trying to explain these things called 'mp3s' to a few non technical friends. Attempting to explain that here's a way to fit an 'acceptable' quality version of an audio track into something that would fit on a couple of floppy disks. How important it was that the size was small enough to be transferable within an 'acceptable' timeframe, and that most people really didn't care for CD-quality (most of them still using cassette tape for Cars, Walkman's etc).<br />
<br />
I remember saying this will really change things, that I wasn't sure how the music industry would cope with their content becoming available in the same way software was, with all the positives and negatives that brought with it.<br />
<br />
Well.. here we are.. almost 20 years later, and there has been Winamp, Napster, Peer2Peer, Lawsuits, iTunes, YouTube... we're currently in a time where we are able to share information with millions of people, just by making a blog post, like this one.. so I figured I'd try hosting some of my music, and was a little surprised by the process.<br />
<br />
<br />
<a name='more'></a><br />
<br />
First I tried hosting using blogger, since I'm here already, and I know it can upload videos, and images, I figured it would do audio as well.. oddly not. Maybe they are too worried about people uploading hoards of ripped cds, or sharing the current top ten as blog posts ;-)<br />
<br />
Next I try YouTube, at least if I get it there, I can link it from here.. but no, foiled again, YouTube only likes videos. Yes I realise that's kind of apparent in its name, but I guess I considered it more as a media hosting site, maybe they let me create a video with an mp3 ? although they have some sort of slideshow creator, it seemed I would need to set the same image, by hand repeatedly until my audio finished.. not ideal.<br />
<br />
So I find a 3rd party website that will combine a jpeg & an mp3, and create a 'video'. With a little effort I could write something similar, but it does the job.. I get the video online, and get the link to the <a href="http://dwellertunes.blogspot.com/2011/05/doctor.html">blog post</a>.<br />
<br />
Then I discover <a href="http://soundcloud.com/dwellertunes">SoundCloud</a> via twitter, used by another piano player to share a bit they had played. I experimented, found you could embed their player into a blog post (albeit via direct html copy-paste), and it looked pretty nice.<br />
<br />
I check google for other audio hosting, and there's plenty out there, mainly in vein of rapidshare, yes they host audio, but the story stops there, usually you just get a link to the hosted file.<br />
<br />
A week or so after publishing the post, I look back at the statistics, Blogger lets me see all sorts of detail, number of views per post, where links come in from, vague statistics on which countries are reading me, I like that, it's nice to know if twitter instantly causes people to read insane ramblings, or if they arrive via a direct link..<br />
<br />
YouTube offers statistics on par with Blogger, with format specific bits, like 'most watched segment of a video', I can see a few people found the video on YouTube without coming in via the blog.<br />
<br />
Soundcloud tells me how many people have listened to each track, per week. For any further information, I have to hand over cash. Now I understand they have a business to run, but when faced with competition like YouTube/Blogger, although I really like their player widget, I'm not going to give money for stats I get free everwhere else.<br />
<br />
Doing that would be like choosing to go to the only shopping center that charged for parking, when the same shops existed in others right next to it, with the exact same prices, and free parking. Ok, the first shopping center may have nicer plants, but I can buy a cheeseburger at either, so I'll just stop using the expensive one.<br />
<br />
So, I suspect although Soundcloud likes the idea of monetizing the statistics model, in this case, it's going to drive away at least one of the people they were aiming the idea at. An interesting business model for a site that relies on external content.<br />
<br />
If you know any other good audio hosting sites, with statistics, etc.. I'd love to know about them =)Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-8710318737849459062011-05-29T00:53:00.000-07:002011-05-29T00:53:32.686-07:00Portable recording in stereo ?I've got this <a href="http://dwellertunes.blogspot.com/">other blog</a>, where I wibble on about musical stuff, and after watching Dr Who the other night, I ended up with a nice enough bit of noise that some people might call it 'music'.<br />
<br />
"Excellent!!" I thought.. "I shall share this with the entire world!!!"<br />
<br />
Of course, that means somehow getting the noise from the piano, into the intertubes. I'm smart enough to know that my piano is a) digital, and b) has line out jacks. And I know I have a big pile of cables, some of which may even be adequate to use with said jacks.. I've also got a netbook thing, which has some sort of audio built in.<br />
<br />
How hard can it be? about an hours worth harder than it should have been! There may have been simpler ways to do this.. but here's the tale of how I ended up sorting it out...<br />
<br />
<a name='more'></a>Firstly, the netbook had a headphone jack, and a microphone jack.. I guess I've been playing with desktops for too long, with proper line-in jacks, or funky auto configuring multipurpose jacks that decide what they are as you connect the cable. I figured maybe this was one of those, so I thought I'd give it a try.<br />
<br />
Now for some reason, the musical instrument world still loves jacks that look like they fell out of a 1940's telephone exchange, where as the pc world has pretty much standardised on those little headphone type connectors, just to make things even more fun, the rest of the audio world has decided to use RCA style plugs like you'd find on the side of your tv for a camcorder.<br />
<br />
This means that if you've ever tried to let these worlds meet, you end up owning one rather oversized tub of cables that go from one type to another, splitting out mono from stereo plugs, little->big adapters, big->little adapters, rca->little etc.. Tonight, it was late, I didn't want to go hunting too long.. all I could find was a spare headphone adapter (to let you use your ipod headphones in a proper hifi stereo output), some twin-rca->stereo cables, and a stereo-stereo little jack lead..<br />
<br />
The piano, of course, offers 2 mono large jacks, for L & R lineout, although the L one doubles as L+R, so I figure I'd connect my headphone adapter there, then use the stereo-stereo to connect to the netbook mic-input. Woot.. I got noise.. in.. mono. Doh. Mic on netbook is solidly mono-only.. gah.<br />
<br />
I try a couple of other spare laptops, and a really cheap usb headset/soundcard dongle which confirm that recording stereo on a laptop looks like it will be needing 'something else' to make it happen.<br />
<br />
Now, as it happens, I have 2 spare 'something elses' that I could use, a <a href="http://www.northq.com/products/tvvideo/nq6600.html">NorthQ 6600</a>, usb realtime mpeg4 encoder, and a usb <a href="http://www.hauppauge.co.uk/site/products/data_hdpvr.html">HD PVR</a> realtime h264 encoder.. both pretty much meant for video use, but I figured they'd work just fine with an audio feed.<br />
<br />
I rummage around to go find them.. of course, the 6600 has been lent out, so that left me the HD PVR. I carefully drag it all to the piano, connect it to the netbook to be greeted by.. "One of your usb devices has malfunctioned and is not recognised". Argh..Vague memories surface of the reason why this HD PVR is spare.. it ceased functioning reliably & got replaced.. but I remember reading something about the PSU that ships with it <a href="http://valkyriemt.wordpress.com/2011/04/09/a-defect-with-the-hd-pvr-will-eventually-fail-for-all/">being prone to dying.. </a> 5v DC, 2a.. surely I must have something like that laying around.. turns out an old router ran on exactly that, even with the same polarity! awesome. Connected it up via that psu, and yay, we have a hd pvr in device manager.<br />
<br />
Of course, this is where Hauppauge like to kick you repeatedly in the ribs, as to record anything from this device will need win tv, or the arcsoft stuff that shipped with it, you can't download it for some oddball licensing reason, you can only download 'updates' yay. How am I supposed to put this 'CD' into my netbook that has no drive? One "quick" iso & virtualdrive mount later.. I'm up and running..<br />
<br />
Now I have to swap the stereo-stereo for stereo->twin-rca, to hook up to the HD PVR, I fire up the arcsoft capture nubbins gizmosoft, and it sits there.. it would appear that to capture audio with this device, I'm going to need to supply it with a video signal. My piano doesn't do video signals, although apparently a model or two up would have ;-)<br />
<br />
So another hunt around the house for the smallest widget I can find that will output a video signal, that wont matter if its unplugged & moved for a while.. Almost gave up, then I spotted an old media-center-extender.. Problem solved!<br />
<br />
So, recorded a 'video' of the main menu of the extender, with my piano as the audio input. Took the video (in .ts transport stream format) on the netbook, demuxed the audio to an ac3 file, re-encoded that to mp3 using besweet, and tried to see how to host it on blogger.<br />
<br />
Blogger is not a fan of mp3s =) Neither is youtube.. my 1st attempt saw me go convert the mp3 into a video with a single frame image, my 2nd attempt found <a href="http://soundcloud.com/">http://soundcloud.com</a> which has a nifty embeddable widget.<br />
<br />
So finally, it's online.. if you want to listen.. its over <a href="http://dwellertunes.blogspot.com/2011/05/doctor.html">here</a>..<br />
<br />
Now I'm off to hit ebay to find a usb soundcard with a line in jack & some appropriate cables =)Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-18046881842109768282011-05-24T03:55:00.000-07:002011-11-27T03:54:32.791-08:00If Microsoft made cars, you'd need to reboot them..Except in this case, it would appear that the 'Microsoft' might be 'Mercedes'. Discovered that the car didn't want to talk to me this morning, it decided I, the key-holder, was a non-entity. No amount of button pressing would awaken the car from its nightly slumber, and the second key fared no better.<br />
<br />
Odd, unlikely both keys would go flat, especially since the nice people at the Merc service desk take such pride in ensuring they swap out those CR2032s for us on a regular basis ;-) Tried the key for the other car, 'Chrirrup!' .. Ok, it's not radio interference..<br />
<br />
Somewhere around here I guess I should have stopped thinking like I was debugging code, but as we'll see, that wouldn't have been such a bad idea ...<br />
<br />
<a name='more'></a><br />
Thankfully, Mercs have "Mobilo Life", which, provided you meet some requirements, like keeping it serviced with them, etc.. gives you a handy call out service, for free.. I can imagine what my last car dealer would have said if I rang them up over a year after buying a 2nd hand car from them saying 'The car doesn't appear to want to start, do you fancy coming and having a look?", yet that's pretty much exactly what we told the Mobilo Life people, and they had an engineer, a proper Merc engineer with full diagnostic suite etc, arrive within an hour.<br />
<br />
His initial conclusion matched mine.. battery is dead, the question is why? He repeatedly asked if we'd left anything on, lights, etc.. which is disturbingly difficult on this car, as pretty much everything turns itself off within 5 minutes, or will scream at you as you try to leave the vehicle with them on (except for parking lights, which are deliberately a little bit tougher to enable). Accepting my response that we hadn't, sensible engineer tests to see how much current the car is using when its supposed to be asleep.<br />
<br />
Here's where it gets odd.. the car is munching away at 650ma while asleep, instead of the 10 or so he expected.. No aftermarket addons, although there is an aging mobile phone in the cradle.. We unplug the phone, let it go to sleep again.. still 650.. so he starts popping fuses under the bonnet one by one, still 650 (more at one stage when replugging a fuse woke something up).<br />
<br />
Should point out he's not exactly guessing at this point, his uber touch panel of knowledge is doing its best to assist him, although I didnt notice clippy popping up anywhere asking if we were trying to fix a dead battery. His other widget is sucking the cars brain out through a cable, and both of them seem to agree that everything is fine, there's nothing to see here, move along. <br />
<br />
Eventually, running out of other options, he tries the solution he refers to as a 'hard reset' ;-) To anyone who's ever used a computer, this would be familiarly known as 'turning it off and on again'. Awesome, of course a car has no plug, just the battery as its power.. and that's obviously lacking any switch.. so a quick disconnect/reconnect is called for.<br />
<br />
After the system comes back up, we let it go to sleep again, and it returns to a nice happy sleeping level of just 10ma or so. Nice.<br />
<br />
Conclusion, something in the cars network of electronic widgetry forgot how to go to sleep, and a reboot put it back into a known state so it sleeps again. Welcome to the age of rebooting your car to fix problems.<br />
<br />
Of course, who knows if its a one off, and we'll never see this again.. or if we'll be seeing him again next Monday =) If it's the latter they get to take it away & figure it out ;pDwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-58505844062400582262011-05-21T12:59:00.000-07:002011-05-21T12:59:17.101-07:00T-Balancer BigNG .. return of the dusty controller.No, this isn't a post about Thomas the Tank Engine.. this is a post about a USB programmable 4 channel Fan Controller, I know, I know.. but don't worry, Thomas is safe & having a rest.<br />
<br />
This is a tale instead of what happens when you go hunting through a cupboard, and discover a bit of tech you had laying around, and think 'hey, I could probably use that to power the fans in the storage server', it probably happens to you every day... No? Well, you've missed out on all the fun then of discovering how to hook up drivers to your aging widget of choice, for your not so aging operating system..<br />
<br />
Here's my tale of random pluggery =)<br />
<br />
<br />
<a name='more'></a><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitghavHnKIgIaN-GN7WRlgakGrAUw0ajKyQM4W3CxzNnGhyMPqZNKheGbqPTw8ozpOxwG9lnHZuZJm15jAvLH-ubTpVmtWqaSv_Ew90GIohLYUvCCkqX8BbrsIUnSOVqUXALgTTNVpfcwt/s1600/bigNG-c.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br />
<img border="0" height="256" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitghavHnKIgIaN-GN7WRlgakGrAUw0ajKyQM4W3CxzNnGhyMPqZNKheGbqPTw8ozpOxwG9lnHZuZJm15jAvLH-ubTpVmtWqaSv_Ew90GIohLYUvCCkqX8BbrsIUnSOVqUXALgTTNVpfcwt/s320/bigNG-c.jpg" width="320" /></a><br />
<br />
So, somewhere along the long riding road I've wandered in trying to create a silent media htpc, I bought one of these..<br />
<br />
It's an attractive looking widget no? sort of might taste interesting if it were a sandwich.<br />
<br />
1 Molex in, 4 fans out, 2 digital sensor buses, 2 usb ports, a loud beeper, and some glowy lights.. in red, not blue..<br />
<br />
Want to know more? the website is over <a href="http://www.t-balancer.com/english/bng.htm">here</a>.<br />
<br />
At a quick glance, it's still for sale in a whole bunch of places, although it would seem the software hasn't had much love since around 2009ish.<br />
<br />
I used to use this thing to manage all the case fans, cpu fan, and gpu fan in a tower case, to make it run pretty much silent when not under load, and to keep it decently cool. Also ran it in a home theatre pc managing the fans there, where I learned that just because you can control temps & fans, doesn't mean the system will be silent. Just means that 20 minutes into the film, the hair dryer will start getting louder.. not good.<br />
<br />
Ok, so I figure I'll fire this thing back up, use it on the storage server.. 1st problem... where did I leave the manuals.. 2nd problem.. this thing came with like a gazillion cables, and I'm not sure where any of them are =)<br />
<br />
I hooked it up to a usb cable.. a pretty red led comes on, and nothing happens, no new device detected. So I go download the software & try again, still nada.. hmm.. mebbe it needs power.. so I feed it some 4pin molex juice, and it seems happier, the software even detects it.<br />
<br />
Great! lets see what it says then.. it seems the UI is now a server widget in my systray.. I find the client front end, and it wont start, wibblegoat.ocx issues. I wield my almighty sword of 'regsvr32' from an admin shell, and the issues dissolve ahead of me. The ui starts.. I tell it to update the firmware.. it agrees, dire warnings about this could brick the device, more dire warnings about am I really sure.. I click yes, it displays a progress bar.. it hits 100%.. it reboots.. and goes into OMG MY EARS ARE BLEEDING alarm mode. Utoh.<br />
<br />
I try a few things, nothing seems to work, then suddenly it fixes itself, and it all seems to work again, ooookie.. The UI seems to have downloaded all the settings from the unit, which map to the old media pc, and it would appear the entire UI is in german, which is great, because I'm not.<br />
<br />
I stumble through panels till I find the language select, it doesnt work.. I kill the UI, relauch it with admin privs, and now the language select works. Sometimes I despair ;-) then I notice some familiar looking cables on the floor, hey, there's one digital sensor, and one analog.. that's a start!<br />
<br />
So eventually I figure out how to reset all the stuff, and when I discover the remaining cables, I can even try to wire it all up.. in the meantime, I've hit ebay for a bunch of spare stuff to help weld this all together.. if it goes anything like this widgets ressurection, I'll be there for at least 4 hours, have to figure out how to speak korean, and I'll need to install some software written for windows Me.<br />
<br />
Wish me luck!Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-14261108186050846602011-05-14T10:30:00.000-07:002011-05-21T12:25:31.303-07:00Drive Failures Anonymous"Hi.. my names Samsung, and I'm a drive failure.."<br />
"Hi.. my names Western Digital and I'm a drive failure..."<br />
<br />
"Welcome guys, why don't you tell us a bit about yourselves, if you feel like sharing.. everyone here has been through what you have, haven't we people?"<br />
<br />
<br />
<a name='more'></a><br />
<br />
Solemn nodding, especially from the IBM Deskstars sitting in the corner, the Seagate's with the SD1A update try desperately to nod, or otherwise join in, but find themselves incapable of anything other than just sitting there.<br />
<br />
Samsung takes a sip of tasty 12v juice and starts his story.. "Well, you see, we all used to get on real well in Dweller's Home Server, we'd managed to stay off the bad sector pills for almost all our lives, how were we supposed to know the stuff was that addictive.. It's not like we lost <i>all</i> his data after all.. I mean, at most he lost, what, a few gigs? ", Western Digital shrugs, "hey.. you might have lost a few gigs, I might have lost a few gigs, who's pointing fingers here.. I mean.. look who he chose to replace us!"..<br />
<br />
"True, true.. " said Samsung.. "At least you guys come with AFT compatibility jumpers to let you be used in a Home Server.. we just get fed data & try not to chuckle too loudly as we smush it into the platters in a random fashion".<br />
<br />
Three very shiny new drives walk in..<br />
<br />
"Is this where you losers hang out? This dusty shelf, with a single solitary esata dock between the lot of you? Ha! Bet you wish you were still in the storage pool with us, and with the cooling fan girls gently caressing us 24/7 eh.. constantly fed nice clean power.. given interesting data to look after.."<br />
<br />
One of the Three newcomers strolls up behind Samsung.. "You buddy.. are gonna have your data ripped out from you.. it's all mine now.. all mine.. and you can just sit here.. empty.. untrusted.. maybe you'll get used for recovery testing.. or turned into a clock.. "<br />
<br />
With that he dragged Samsung over to the esata dock, wedged him into it, and sat back to wait the estimated 14 or so hours it would take to find out everything Samsung ever knew.. He drifted off into a doze, where he daydreamed about being an inanimate data storage device connected to a hotswap bay in a server.. *shudder*.Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-66748977415482045982011-05-10T23:55:00.000-07:002011-05-21T12:25:48.301-07:00The Network Server has a bad Night.Ever had one of those days where things seemed like they were just sailing along.. your mental todo list full with little things like 'fix why the media pc is confused over which channels are high def', and 'think about adding more storage to the home server' .. and then you wake up to discover a little red house in your systray ?<br />
<br />
If you know what I'm talking about, you'll have an inkling of where this is going. If you're wondering what a little red house is doing in my systray, then we should probably start there...<br />
<br />
<br />
<a name='more'></a><br />
<br />
The little house in the systray is the Windows Home Server status monitor, that gets put in the systray of all machines the server is looking after. If the server is off, or asleep, it's a grey house. If the server is happy, and there's no need to think about it, the house is green. If the server needs a cuddle, the house will be yellow. Lastly, if the server has accidentally amputated 3 of its fingers in a drunken all night data storage orgy, the status will be red.<br />
<br />
Yes. Red. The same color as my status. My Windows Home Server was screaming help in the only color it knew how.<br />
<br />
So, I bring up the console panel, it's a funky remote desktop doodad application from the server. My Server has a frequent habit of getting blindingly drunk, and thinking it amputated half its fingers, when usually it's just forgotten that it ever had any. In tech terms, very infrequently 5 of the drives behind a SATA expander go awol, this happens maybe once every couple of months, and a reboot fixes it. I goto the drive status panel, sure enough there's a block of disks showing up as 'red', the chance of the server actually having lost all that block simultaneously is tiny, unthinkable.. so I tell it to reboot.<br />
<br />
While it reboots, I start thinking, did the server fail 5 drives, or was it 6... did I feel lucky, well, did I ? And slowly the conclusion dawned in my head, that it may have been 6. 6 would be Bad, 6 is one more finger than the server usually drunkenly hallucinates amputating.<br />
<br />
The server comes up.. status is green! yay.. status goes red! utoh.<br />
<br />
I bring up the console, the server is clearly struggling, 5 disks offline already, and 1 showing as 'unhealthy', after the amount of vodka the server drank, this isnt too unsurprising, I ask the server for a diagnosis of the unhealthy disk.. the disk promply falls into a coma, and updates its status to offline.<br />
<br />
So I power down the server.. pull every disk from it's hotswap bay.. power up the server, and start re inserting the disks one by one.. first 4 disks, no problem.. disks 5 & 6 spin up, but refuse to add to the pool, no problem, I pulled them back out, and continued on.. 7,8,9,10,11, all fine, 12.. no go.<br />
<br />
3 disk failure? really? If this were running Raid I'd be a very very unhappy bunny. (Think watership down, rather than playboy). Thankfully, this is running Windows Home Server, will full duplication turned on.. with 3.5tb now 'missing' I know I'm going to lose some data, but I don't know which ;-) I do know the majority will survive..<br />
<br />
So now, Im working through the 'drive removal' process, letting the server know that it's lost those fingers forever, and all the piano pieces it used to play using them, now needed re figuring out using its remaining fingers. The server understands, it takes it over 8 hours for each disk removal. You can almost hear it practicing and repracticing those tricky sections using its remaining fingers.<br />
<br />
Once the process is complete, I'll go grab some more disks (probably have to be WD20EARS with a jumper on them to disable advanced format), and feed it back the space that it lost.<br />
<br />
The other good news, is the 'failed' drives still spin up, although clearly they are taking a fair while to do so.. and Windows Home Server stores its data on each disk, as NTFS, the disks are mountable from another pc, where I'll be able to access & send much of any actually 'lost' data back to the server.<br />
<br />
With duplication enabled for every share, I'll only lose files where the file & its duplicate were stored within the set of disks I'm removing. With 3.5tb total removed, that's a fairly large chunk of possible loss, but relatively small against the remaining 15tb or so.Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0tag:blogger.com,1999:blog-513334600397219012.post-25820816844634455392011-05-08T07:44:00.000-07:002011-05-11T02:00:13.081-07:00Shiny unexpectedness!Ok, I probably should read the manuals from time to time.. but it's just so much more fun not to ;-) After all is a device really all that usable if you have to keep referring to a virtual bit of paper helpfully shipped on a cdrom along with the lump of electronics & plastic that arrived here on a boat from china?<br />
<br />
But I digress.. today I got round to picking up a mini-hdmi to hdmi adapter to enable me to connect the Panasonic Lumix TZ10 to a tv via HDMI rather than composite video. At least then I wont feel like Im staring at 40 inches of smeary blurryness when I'm viewing them via family tv's etc.<br />
<br />
Plugged it all together, and up came a nice 1080i user interface, did the normal 'sit far too close to the large screen due to being tethered to it by the video cable so you can use the camera controls to move to the next picture' when I noticed 2 extra controls listed at the bottom of the UI. 'Red' & 'Green'. Odd, the only red button on the camera is the record button, and there's nothing green on it at all.<br />
<br />
Slowly it dawned on me that there <i>were</i> Red and Green buttons on the TV remote.. surely this camera wasnt thinking.. it was.. omg.. I mean I know HDMI can carry control as well as A/V, and about HDMI-CEC etc, I just didnt expect to see it, in a camera.<br />
<br />
So now I can sit on the sofa like a normal couch potato & control the slideshow from there with the TV remote.. Nice.Dwellerhttp://www.blogger.com/profile/15207795688140161510noreply@blogger.com0