Coming Underground

Debugging Perl CGI scripts with Lighttpd

Permalink 2010-03-11 10:37, by jaervosz, Categories: General, Gentoo Tips, Universe/English

I've never used Lighttpd much before usually using Apache. However recently I started serviing Nagios from Lighttpd and since my MythTV backend is running on the same system I'm serving my Mythweb from Lighttpd as well. While hacking it a bit I discovered that Lighttpd doesn't log Perl errors to the standard lighttpd error log. It appears that versions after 1.4.19 doesn't allow CGIs to write to the error file or I did something completely wrong with my installation.

I worked around the problem by using CARP:

--- handler.pl.orig     2010-03-10 15:28:29.000000000 +0100
+++ handler.pl  2010-03-10 15:29:06.000000000 +0100
@@ -14,4 +14,10 @@
 # Other includes
     use Sys::Hostname;
+# hack to allow lighttpd to log cgi errors
+# http://redmine.lighttpd.net/boards/2/topics/16
+    my $logfile = ">>/var/log/lighttpd/mycgi-log";
+    use CGI::Carp qw(carpout);
+    open(LOG, $logfile) or  print header(), "Unable to open log $logfile.";
+    carpout(LOG);
 
     require "modules/$Path[0]/tv.pl";
Leave a comment »Send a trackback »

MythTV video streaming hack

Permalink 2010-03-10 15:58, by jaervosz, Categories: General, Gentoo Tips, Universe/English, Mythtv

One of our MythTV frontends is a small Dell netbook with SDD and therefor not much disk space. Therefor I decided to hack MythTV to stream videos from the Mythweb Video interface.

First off I had to enable MP4 streaming since I'm often transcoding to MP4.

--- modules/stream/stream_raw.pl.orig  2010-03-10 14:08:32.000000000 +0100
+++ modules/stream/stream_raw.pl       2010-03-10 14:08:48.000000000 +0100
@@ -39,4 +39,8 @@
         $suffix = '.avi';
     }
+    elsif ($basename =~ /\.mp4$/) {
+        $type   = 'video/avi';
+        $suffix = '.mp4';
+    }
     else {
         print header(),

After enabling MP4 streaming you'll have to hack a bit in the tv.pl script. It starts out by determinig the recording type simply by looking at the request URL.

--- modules/stream/tv.pl.orig  2010-03-04 21:02:28.000000000 +0100
+++ modules/stream/tv.pl       2010-03-10 14:26:16.000000000 +0100
@@ -19,40 +19,69 @@
     }
 
-# Which show are we streaming?
-    our $chanid    = url_param('chanid');
-    our $starttime = url_param('starttime');
-    if ($Path[1]) {
-        $chanid    = $Path[1];
-        $starttime = $Path[2];
-        $starttime =~ s/\.\w+$//;
-    }
+# Determine wether we're streaming video or tv
 
-# Get the basename from the database
-    my $sh = $dbh->prepare('SELECT basename, title, subtitle, endtime-starttime
-                              FROM recorded
-                             WHERE starttime=FROM_UNIXTIME(?)
-                                   AND recorded.chanid   = ?');
-    $sh->execute($starttime, $chanid);
-    our ($basename, $title, $subtitle, $runtime) = $sh->fetchrow_array();
-    $sh->finish;
-
-# No match?
-    unless ($basename =~ /\w/) {
-        print header(),
-              "Unknown recording requested.\n";
-        exit;
-    }
+if ($Path[2]) {
+       $type = "tv";
+} elsif ($Path[1]) {
+       $type = "video";
+}
 
-# Find the local file
-    our $filename;
-    $sh = $dbh->prepare('SELECT DISTINCT dirname
-                           FROM storagegroup');
-    $sh->execute();
-    while (my ($video_dir) = $sh->fetchrow_array()) {
-        next unless (-e "$video_dir/$basename");
-        $filename = "$video_dir/$basename";
-        last;
-    }
-    $sh->finish;
+if ( $type eq "tv" ) {
+       # Which show are we streaming?
+           our $chanid    = url_param('chanid');
+           our $starttime = url_param('starttime');
+           if ($Path[1]) {
+               $chanid    = $Path[1];
+               $starttime = $Path[2];
+               $starttime =~ s/\.\w+$//;
+           }
+
+       # Get the basename from the database
+           my $sh = $dbh->prepare('SELECT basename, title, subtitle, endtime-starttime
+                                     FROM recorded
+                                    WHERE starttime=FROM_UNIXTIME(?)
+                                          AND recorded.chanid   = ?');
+           $sh->execute($starttime, $chanid);
+           our ($basename, $title, $subtitle, $runtime) = $sh->fetchrow_array();
+           $sh->finish;
+
+       # Find the local file
+           our $filename;
+           $sh = $dbh->prepare('SELECT DISTINCT dirname
+                                  FROM storagegroup');
+           $sh->execute();
+           while (my ($video_dir) = $sh->fetchrow_array()) {
+               next unless (-e "$video_dir/$basename");
+               $filename = "$video_dir/$basename";
+               last;
+           }
+           $sh->finish;
+
+           1;
+} elsif ( $type eq "video" ) {
+       # Which show are we streaming?
+           our $vidid     = url_param('vidid');
+           if ($Path[1]) {
+               $vidid     = $Path[1];
+           }
+       # Get the basename from the database
+           my $sh = $dbh->prepare('SELECT title, subtitle, filename
+                                    FROM videometadata
+                                    WHERE intid = ?');
+           $sh->execute($vidid);
+           our ($title, $subtitle, $basename) = $sh->fetchrow_array();
+           $sh->finish;
+
+       # Find the local file
+           our $filename;
+           $sh = $dbh->prepare('SELECT DISTINCT dirname
+                                  FROM storagegroup
+                                  WHERE groupname = "Videos"');
+           $sh->execute();
+           my $video_dir = $sh->fetchrow_array();
+               $filename = "$video_dir/$basename";
+
+           $sh->finish;
 
-    1;
+           1;
+}

All that is now left is to make the webinterface display the new links.

--- modules/video/tmpl/default/video.php.orig   2010-03-10 13:30:31.000000000 +0100
+++ modules/video/tmpl/default/video.php        2010-03-10 13:57:53.000000000 +0100
@@ -307,5 +307,8 @@
         <div id="<?php echo $video->intid; ?>_browse" class="hidden"><?php echo $video->browse; ?></div>
         <div id="<?php echo $video->intid; ?>-title" class="title">
-            <a href="<?php echo $video->url; ?>"><?php echo html_entities($video->title); ?></a><?php 
+<?php echo html_entities($video->title); ?>
+            <a href="<?php echo $video->url; ?>"><img src="<?php echo skin_url ?>/img/video_sm.png"></a>
+            <a href="<?php echo ".//pl/stream/$video->intid" . ".asx"; ?>"><img src="<?php echo skin_url ?>/img/play_sm.png">
+           </a><?php 
                   if (($video->season > 0) && ($video->episode >= 0)) 
                       printf('(S%02d E%02d)', $video->season, $video->episode); 

As you can see this is just a quick hack and could probably have been better integrated into MythTV and my Perl skills feel a bit rusty too.

Sorry for the crappy display of code. If you want the diffs as properly formatted text files just drop me a line.

Let me know if you have any comments or other cool hacks.

Leave a comment »Send a trackback »

Missing directory structure in Mythweb videos

Permalink 2010-03-05 08:31, by jaervosz, Categories: General, Universe/English, Mythtv

Somehow the Directory structure in the Mythweb videos section is empty using mythweb-0.22_p22763.

Perhaps something is b0rked in my setup, so I had to hack it up a bit since the filename in videometadata doesn't overlap with my VideoStartupDir setting.

--- modules/video/handler.php.orig      2010-03-04 10:23:05.000000000 +0100
+++ modules/video/handler.php   2010-03-04 10:25:13.000000000 +0100
@@ -120,5 +120,6 @@
         foreach ($dirs as $dir) {
             if ($dir) {
-                if(strpos($file, $dir) !== false) {
+               #Disable since $file and $dir doesn't overlap
+                #if(strpos($file, $dir) != false) {
                     if (!isset($PATH_TREE[$dir]))
                         $PATH_TREE[$dir] = array('display' => $dir,
@@ -128,5 +129,5 @@
                     $file = str_replace($dir, '', $file);
                     break;
-                }
+                #}
             }
         }
@@ -150,4 +151,6 @@

     function output_path_picker($path, $padding=0) {
+       #Hide VideoStartupDir
+       $path=str_replace(setting('VideoStartupDir', hostname),"",$path);
         for ($i = 0; $i < $padding; $i++)
             echo ' ';

Leave a comment »Send a trackback »

B2evolution antispam list

Permalink 2010-03-01 08:39, by jaervosz, Categories: General, Gentoo Tips, Universe/English, Opensource

Before I implemented comment captchas on this blog I was flodded with comment spam. Captchas helped a lot! Like removing 95+% of all comment spam, however some still gets through. Deleting a spam comment now and then have irritated me greatly, but until earlier this week I haven't really done much about it.

After just a bit of research I decided that my first move was to automatically pull the B2evolution antispam list. I also considered Akismet but one step at a time.

Here is the simple script called from cron I use to pull the updates:

 #!/bin/sh

site=http://example.com/admin.php
name=username
pass=password
cookies=`mktemp`

wget -O /dev/null --post-data="login=$name&pwd=$pass" \
        "${site}?ctrl=antispam&action=poll"

It's not perfect since at least one comment spam went through already.

I'd like to hear what other people to prevent comment spam (other than disabling comments :-) )?

Leave a comment »Send a trackback »

Transcoding LATM packed HE-AAC audio with MythTV

Permalink 2010-02-23 09:01, by jaervosz, Categories: General, Gentoo Tips, Universe/English, Opensource, Mythtv

Since some of the new Danish digital TV channels (on MUX2) are using the HE-AAC audio codec which is not widely supported (ie. on my Pop Corn Hour) I had to transcode the recordings before being able to watch it on my frontend.

I'm using VLC for the transcoding, since ffmpeg couldn't quite grok it yet.

For this I use the following simple script:

#!/bin/bash
vcodec="mp4v"
acodec="a52"
bitrate="VIDEO_BITRATE"
arate="128"
ext="aac"
mux="mp4"
vlc="/usr/bin/vlc"
input=$1
output=$2

#http://www.geekzone.co.nz/forums.asp?forumid=83&topicid=55394
$vlc -v -I dummy "$input" --sout "#transcode{acodec=$acodec,ab=$arate,channels=2,samplerate=48000}:duplicate{dst=std{access=file,mux=ts,dst=\"$output\"}" vlc://quit
exit $?

Note: You have to enable VLC stream support for the --sout to work (USE="stream").

With MythTV I wrap it in the following script for it to work as a normal MythTV UserJob:

#!/bin/bash                                         
VIDEODIR=$1                                         
FILENAME=$2                                         
JOBID=$3                                            
NEWFILENAME=`echo $FILENAME | sed -e "s/\....$//"`.mp4

# Sanity checking, to make sure everything is in order.
if [ -z "$VIDEODIR" -o -z "$FILENAME" ]; then          
        echo "Usage: $0 <VideoDirectory> <FileName>"   
        exit 5                                         
fi                                                     
if [ ! -f "$VIDEODIR/$FILENAME" ]; then                
        echo "File does not exist: $VIDEODIR/$FILENAME"
        exit 6                                         
fi                                                     

cat << EOF | mysql -h mysqlserver -u mythtv -ppassword mythconverg
UPDATE jobqueue SET comment = "Starting transcoding using vlc." WHERE id = $JOBID;
EOF

# Remove previous mp4 if any
if [ -f $VIDEODIR/$NEWFILENAME  ]; then
        rm $VIDEODIR/$NEWFILENAME
fi

/home/mythtv/bin/transcodelatm.sh $VIDEODIR/$FILENAME $VIDEODIR/$NEWFILENAME

ERROR=$?
if [ $ERROR -ne 0 ]; then
        echo "vlc failed for ${FILENAME}.tmp with error $ERROR"
        exit $ERROR
fi

#Update MySQL

cat << EOF | mysql -h mysqlserver -u mythtv -ppassword mythconverg
UPDATE recorded SET basename = "$NEWFILENAME", filesize = $(ls -l $VIDEODIR/$NEWFILENAME | awk '{print $5}') WHERE basename = "$FILENAME";
EOF

#Remove previous version
rm $VIDEODIR/$FILENAME

exit 0

I don't update the jobqueue table once the job is finished, since MythTV immediately overwrites my update. The wiki shows how to use a bit of trickery to update the jobqueue table after the job completes, but the above solution is fine for me.

Leave a comment »Send a trackback »

:: Next >>

©2010 by admin

Contact Jaervosz