Convert a TV program to a comic strip and "read it" on your Kindle. What an great idea - and it's something that the "iPlayer" app for Kindle makes possible.

Just once in a while someone thinks up a method of doing something and you can't really work out if it is the method or the something that is really amazing.

So it is with the iPlayer app for the Kindle (named after the BBC iPlayer app). This succeeds in showing videos on an E-ink display which is far too slow to respond to show enough frames per second to show videos. The whole thing is a bit of a cheat but one that is very clever and could well have other uses.

The program created by Frisnit Electronic Industrial converts a DTV signal into a digital comic strip format complete with text captions delivering the dialog etc. The result can be delivered as a PDF or HTML file directly to the Kindle or to any other viewer.

If this all sounds impossibly difficult and not a project you would want to work on over the weekend you need to spot the simple fact that a DTV signal already has a caption stream. So your programming problem comes down to extracting which video frames to use to tell the story. The first selection criterion is to display any frame that has a subtitle frame associated with it. However there are frames that are important to telling the story that don't have dialog so this isn't a complete solution. Fortunately there are standard mechanisms to detect the start and end of a scene - even simple measures like a change in average brightness does the job reasonably well. Armed with a list of scene start and stop points the software can take a frame from the middle of the scene in the hope that it summarizes what happens.  Any scene that also has a subtitle frame has it's mid-point frame discarded.

So using nothing but a standard DTV signal and of the self modules the software does a reasonably good job of converting a TV program to an annotated comic format which can be viewed on a wide range of devices and transmitted over low bandwidth links including printing it out and mailing it. The Frisnit website even has a small image player embedded:

There is room for improvement so the task isn't complete. The DTV stream also contains an audio description track which could be put into some speech recognition software to create descriptive text to go with the speech captions. Better scene detection software would help and it's an obvious candidate for some AI or better statistical methods to extract meaningful frames from scenes.

So if you need your TV programs in comic book form - it's easy!

There is nothing more satisfying than when something that sounds really difficult turns out to be very easy.  All you need is to notice that it is possible.

More Information

Frisnit Electronic Industrial

eastenders.pdf

If you would like to be informed about new articles on I Programmer you can either follow us on Twitter or Facebook or you can subscribe to our weekly newsletter.