19 September, 2007

Piping to Tcl Script

Ok, maybe I've lived up to my name. I may very well be the fat slow
kid.

I struggled for some time trying to figure out how to pipe input to a Tcl script and immediately ventured off on a quest for opening a pipe (|) file descriptor. In actuality however, since I wanted to pipe input into the tcl script my needs are met by:

#!/usr/bin/tclsh
while { [gets stdin line] >= 0 } {
puts "got '$line'"
}

so

$ ls | ./tclPipe

results in

got 'Desktop'
got 'IM'
got 'ImagingManager.tar.gz'
got 'ISC'
got 'tclPipe'


My quest however also recovered if the input command is fixed:

#!/usr/bin/tclsh

set fp [open "|ls" r]
while { [gets $fp line] >= 0 } {
puts "got $line"
}

would surfice.

16 September, 2007

Command Line Processing with Popt

General practice for writing flexible applications lend toward the ability to specify command-line arguments to tailor the behavior of your application. For instance, a common practice of specifying -v to enable a verbose mode where debugging information is echoed as your application runs. Specifying a unique port for networking applications, or enablement/disablement of an application gui are two other examples that are common. Since command line arguments can take the flavor of integer, floats, doubles, strings...the code for parsing these arguments can be far from simple. Tailoring flexible parsers every time you develop an application is labor-intensive and a waste of time if a general mechanism were available.

Lucky for us, such a mechanism does exist; namely the Popt library.

A short example of an application which uses the library follows:

1 #include
2 #include
3 #include
4
5 int main(int argc, char **argv) {
6 /* option parsing variables */
7 char ch;
8 poptContext opt_con; /* context for parsing command-line options */
9 char *extra_arg;
10 int i=0;
11 char *s="";
12 float f=0.0;
13 double d=0.0;
14 int verbose=0;
15
16 static struct poptOption options_table[] = {
17 { "integer", 'i', POPT_ARG_INT, &i, 'i', "grab an integer", "INT" },
18 { "string", 's', POPT_ARG_STRING, &s, 's', "grab a string", "STRING" },
19 { "float", 'f', POPT_ARG_FLOAT, &f, 'f', "grab a float", "FLOAT" },
20 { "double", 'd', POPT_ARG_DOUBLE, &d, 'd', "grab a double", "DOUBLE" },
21 { "verbose", 'v', POPT_ARG_NONE, NULL, 'v', "enable verbose", "" },
22 POPT_AUTOHELP
23 { NULL, 0, 0, NULL, 0 } /* end-of-list terminator */
24 };
25
26 opt_con = poptGetContext(NULL, argc, (const char **)argv, options_table, 0);
27
28 /* Now do options processing */
29 while ((ch = poptGetNextOpt(opt_con)) >= 0) {
30 printf("between while & switch: ch = %c\n", ch);
31 switch (ch) {
32 case 'i':
33 printf("handling 'i' option.\n");
34 break;
35 case 's':
36 printf("handling 's' option.\n");
37 break;
38 case 'f':
39 printf("handling 'f' option.\n");
40 break;
41 case 'd':
42 printf("handling 'd' option.\n");
43 break;
44 case 'v':
45 printf("handling 'v' option.\n");
46 verbose = 1;
47 break;
48 }
49 }
50
51 if (ch < -1) { 52 // the user specified an invalid option, tell them 53 poptPrintHelp(opt_con, stderr, 0); 54 } 55 56 /* non-option args */ 57 while (extra_arg = (char *)poptGetArg(opt_con)) { 58 printf("extra arg: %s\n", extra_arg); 59 exit(1); 60 } 61 62 63 /* cleanup */ 64 poptFreeContext(opt_con); 65 66 printf("(%s:%d) i = %d\n",__FILE__,__LINE__,i); 67 printf("(%s:%d) s = '%s'\n",__FILE__,__LINE__,s); 68 printf("(%s:%d) f = %f\n",__FILE__,__LINE__,f); 69 printf("(%s:%d) d = %lf\n",__FILE__,__LINE__,d); 70 printf("(%s:%d) v = %d\n",__FILE__,__LINE__,verbose); 71 72 73 return EXIT_SUCCESS; 74 }


You compile and link the application by:
g++ -Wall   main.o  -lpopt -o main


An added bonus of using this application is the automatic introduced interfaces for a brief and more verbose help descriptions.


~/Desktop/sourceCode/C/popt$ ./main --help
Usage: main [OPTION...]
-i, --integer=INT grab an integer
-s, --string=STRING grab a string
-f, --float=FLOAT grab a float
-d, --double=DOUBLE grab a double
-v, --verbose enable verbose

Help options:
-?, --help Show this help message
--usage Display brief usage message



~/Desktop/sourceCode/C/popt$ ./main --usage
Usage: main [-v?] [-i|--integer INT] [-s|--string STRING] [-f|--float FLOAT]
[-d|--double DOUBLE] [-v|--verbose] [-?|--help] [--usage]



The heart of the library lies with proper initialization of the popt structure which is defined as follows:

struct poptOption {
const char * longName;
char shortName;
int argInfo;
void * arg;
int val;
const char * descrip;
const char * argDescrip;
};

The ability to specify either a long or short argument name is common practice; -h or --help is a common example of this form.
The long and short forms are specified as the 1st and 2nd element in the popt structure. The 3rd and 4th specify of what type the following argument consists of as well as the address in which the value will be stored. The 6th and 7th arguments specify the argument description and argument field name which is displayed in the help and brief. The 5th field is a bit of a mystery at this time.

That's all for now.

28 August, 2007

Gprof Profiling

Profiling is a mechanism for identifying program 'hotspots', identifying where your program spends a good majority of it's time. These regions are prime candidates for optimizations, where concentration of efficiency will give you the most bang for your buck.

Profiling with Gprof consists of 3-phases: 1) compiling your application with profiling enabled, 2) executing the application to gather profiling metrics, and 3) evaluating the collected metrics.

Compiling
You first must compile your application similar to the way you normally compile it. Two additional flags must be specified however, the -pg option to enable profiling, and the -g option to introduce the debugging symbols for tracking source lines of code. Actually, the -g option is only necessary for line-by-line profiling, but for good measure I suggest specifying it regardless.

$ gcc -pg -g foo.c -o foo


Executing
You run your application in the same manner as your normally run it, specifying the same arguments, inputs, outputs, . . . What you may notice however is that the program will execute slower than normal. This is reasonable when you consider what is taking place. Profiling metrics are collected during the execution of the program. Worth noting, your executable needs to terminate in a normal fashion, by returning from the main routine or calling exit().
Immediately prior to terminating the program, an output file (gmon.out) is generated that contains the collected profiling metrics. This file will later be used in evaluating the program performance.

Evaluation
Evaluation of the results stored in the gmon.out file is the last step in our quest. Various types of reports are available from the collected profiler info, the most readily used is the flat model. This is done by executing the command:

$ gprof foo


This results in the default report, a flat model followed by the call graph. The flat model will depict the list of functions in decreasing order with respect to time spent in each function. This allows for quick evaluation of the functions that your application spends most it's time in. You may notice that two functions appear in every profile; mcount and profil, each function is part of the profiling apparatus. The time spent in both functions can be viewed as profiler overhead.

12 August, 2007

FFMpeg Video Clip Generation

Hey all,

I just published my first two videos on YouTube.

I downloaded a couple videos from our Tivo, used tivodecode to decode to Mpegs and FFMpeg to extract 30 sec clips.


$ ffmpeg -ss 600 -t 30 -i infile.mpg outfile.mpg


Will result in seeking in 10 min, encoding the next 30 seconds from infile.mpg and saving to outfile.mpg

02 August, 2007

Theora/Ogg Example

I've recently been playing with Theora for encoding video. I found few examples for encoding raw video frames, and those I found were more complicated that I wanted. Below you will find my first toe-dip in the Theora realm. It basically generates a spinning dot in the middle of the frame and encodes to video at 10 frames per sec.

Have fun.


#define _FILE_OFFSET_BITS 64

#include <stdio.h>
#include <stdlib.h>
#include <ogg h="">
#include "theora/theora.h"
#include <string.h>
#include <math.h>

static FILE *ogg_fp = NULL;
static ogg_stream_state ogg_os;
static theora_state theora_td;
static theora_info theora_ti;

static int
theora_open(const char *pathname)
{
printf("(%s:%d) out filename: %s\n",__FILE__,__LINE__,pathname);
ogg_packet op;
ogg_page og;
theora_comment tc;

ogg_fp = fopen(pathname, "wb");
if(!ogg_fp) {
fprintf(stderr, "%s: error: %s\n",
pathname, "couldn't open output file");
return 1;
}

if(ogg_stream_init(&ogg_os, rand())) {
fprintf(stderr, "%s: error: %s\n",
pathname, "couldn't create ogg stream state");
return 1;
}

if(theora_encode_init(&theora_td, &theora_ti)) {
fprintf(stderr, "%s: error: %s\n",
pathname, "couldn't initialize theora encoding");
return 1;
}

theora_encode_header(&theora_td, &op);
ogg_stream_packetin(&ogg_os, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
fwrite(og.header, og.header_len, 1, ogg_fp);
fwrite(og.body, og.body_len, 1, ogg_fp);
}

// encode a comment into the packet
theora_comment_init(&tc);
theora_encode_comment(&tc, &op);
ogg_stream_packetin(&ogg_os, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
fwrite(og.header, og.header_len, 1, ogg_fp);
fwrite(og.body, og.body_len, 1, ogg_fp);
}

theora_encode_tables(&theora_td, &op);
ogg_stream_packetin(&ogg_os, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
fwrite(og.header, og.header_len, 1, ogg_fp);
fwrite(og.body, og.body_len, 1, ogg_fp);
}

if(ogg_stream_flush(&ogg_os, &og)) {
fwrite(og.header, og.header_len, 1, ogg_fp);
fwrite(og.body, og.body_len, 1, ogg_fp);
}

return 0;
}

static int
theora_write_frame(unsigned long w, unsigned long h, unsigned char *yuv)
{
yuv_buffer yuv_buf;
ogg_packet op;
ogg_page og;

unsigned long yuv_w;
unsigned long yuv_h;

unsigned char *yuv_y;
unsigned char *yuv_u;
unsigned char *yuv_v;

unsigned int x;
unsigned int y;

/* Must hold: yuv_w >= w */
yuv_w = (w + 15) & ~15;

/* Must hold: yuv_h >= h */
yuv_h = (h + 15) & ~15;

yuv_y = malloc(yuv_w * yuv_h);
yuv_u = malloc(yuv_w * yuv_h / 4);
yuv_v = malloc(yuv_w * yuv_h / 4);

yuv_buf.y_width = yuv_w;
yuv_buf.y_height = yuv_h;
yuv_buf.y_stride = yuv_w;
yuv_buf.uv_width = yuv_w >> 1;
yuv_buf.uv_height = yuv_h >> 1;
yuv_buf.uv_stride = yuv_w >> 1;
yuv_buf.y = yuv_y;
yuv_buf.u = yuv_u;
yuv_buf.v = yuv_v;

for(y = 0; y <x =" 0;" y =" 0;" x =" 0;">> 1) + (y >> 1) * (yuv_w >> 1)] = 0;
yuv_v[(x >> 1) + (y >> 1) * (yuv_w >> 1)] = 0;
}
}

for(y = 0; y < x =" 0;" y =" 0;" x =" 0;">> 1) + (y >> 1) * (yuv_w >> 1)] =
yuv[3 * (x + y * w) + 1];
yuv_v[(x >> 1) + (y >> 1) * (yuv_w >> 1)] =
yuv[3 * (x + y * w) + 2];
}
}

if(theora_encode_YUVin(&theora_td, &yuv_buf)) {
return 1;
}

if(!theora_encode_packetout(&theora_td, 0, &op)) {
return 1;
}

ogg_stream_packetin(&ogg_os, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
fwrite(og.header, og.header_len, 1, ogg_fp);
fwrite(og.body, og.body_len, 1, ogg_fp);
}

free(yuv_y);
free(yuv_u);
free(yuv_v);

return 0;
}

static void
theora_close(void)
{
ogg_packet op;
ogg_page og;
static int theora_initialized = 0;

if (theora_initialized) {
theora_encode_packetout(&theora_td, 1, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
fwrite(og.header, og.header_len, 1, ogg_fp);
fwrite(og.body, og.body_len, 1, ogg_fp);
}

theora_info_clear(&theora_ti);
theora_clear(&theora_td);

fflush(ogg_fp);
fclose(ogg_fp);
}

ogg_stream_clear(&ogg_os);
}

int
main(int argc, char *argv[])
{
int c;
int n;
unsigned i;
const char* const outFile="foo.ogg";
const unsigned NumFrames = 512;

const unsigned Width = 640/2;
const unsigned Height = 480/2;
const int video_fps_numerator = 30;
const int video_fps_denominator = 1;
const int video_aspect_numerator = 0;
const int video_aspect_denominator = 0;
const int video_rate = 0;
const int video_quality = 63;
theora_info_init(&theora_ti);

theora_ti.width = ((Width + 15) >>4)<<4; height =" ((Height">>4)<<4; frame_width =" Width;" frame_height =" Height;" offset_x =" 0;" offset_y =" 0;" fps_numerator =" video_fps_numerator;" fps_denominator =" video_fps_denominator;" aspect_numerator =" video_aspect_numerator;" aspect_denominator =" video_aspect_denominator;" colorspace =" OC_CS_UNSPECIFIED;" pixelformat =" OC_PF_420;" target_bitrate =" video_rate;" quality =" video_quality;" dropframes_p =" 0;" quick_p =" 1;" keyframe_auto_p =" 1;" keyframe_frequency =" 64;" keyframe_frequency_force =" 64;" keyframe_data_target_bitrate =" video_rate" keyframe_mindistance =" 8;" noise_sensitivity =" 1;" i="0;" frame="" d="" of="" n="" r="255;" const="" unsigned="" char="" y="(abs(R" 2104="" g="255;" 4130="" b="255;" 802="" 4096="">> 13);
unsigned char U = (abs(R * -1214 + G * -2384 + B * 3598 + 4096 + 1048576) >> 13);
unsigned char V = (abs(R * 3598 + G * -3013 + B * -585 + 4096 + 1048576) >> 13);
// generate a frame with specified color
{
unsigned i;
for(i = 0; i < r =" 0;" g =" 0;" b =" 255;" y =" (abs(R">> 13);
unsigned char U = (abs(R * -1214 + G * -2384 + B * 3598 + 4096 + 1048576) >> 13);
unsigned char V = (abs(R * 3598 + G * -3013 + B * -585 + 4096 + 1048576) >> 13);
const unsigned cX = Width/2;
const unsigned cY = Height/2;
const unsigned Radius = 50;
static double theta = 0.0;
const unsigned x = Radius * sin(theta) + cX;
const unsigned y = Radius * cos(theta) + cY;
const unsigned k=3*(x + (Width*y));
theta -= 5.0 * 3.14159/180.0;
yuv[k] = Y;
yuv[k+1] = G;
yuv[k+2] = B;
}

if(theora_write_frame(Width, Height, yuv)) {
theora_close();
exit(1);
}
}

theora_close();
return 0;
}

05 July, 2007

Logging to SysLog

Trace logging of significant events is a fundamental method for debugging and isolating errors encountered. User warnings, pop-up notification boxes, and application-level log files are one way to go. Another; to make use of the standard system log services found on *nix systems. The remainder of this post will some brief information I've recently discovered concerning interfacing with the system log services.

The syslog system import defines the interface:


extern void syslog (int __pri, __const char *__fmt, ...)


which allows for variable-lengthed argument lists, much like printf. The interface is pretty simple, the first argument specifies a priority or type of log statement; the second, a formatted string in the same form that printf takes.

A simple C

#include <stdio.h>
#include <syslog.h>

int main(int argc, char *argv[]) {
syslog(LOG_NOTICE,"%s process started", argv[0]);
sleep(3);
syslog(LOG_NOTICE,"%s process terminating", argv[0]);
}


Compile and running this example will result in two log entries in the system logs. Determining what log will contain the log statements can be determined by 1) the log type, and 2) the /etc/syslog.conf file. Note, that the log type is defined as LOG_NOTICE, and the entry in the syslog.conf file entry;

*.=info;*.=notice;*.=warn;\
auth,authpriv.none;\
cron,daemon.none;\
mail,news.none -/var/log/messages


shows that the notice log entries will be evident in /var/log/messages.

Since the files are only accessible to superusers, logging in as root and tailing the log file should show the messages in the same manner as:

# cat /var/tmp/messages
.
.
.
Jul 5 20:30:18 riffraff syslogd 1.4.1#18: restart.
Jul 5 20:42:07 riffraff app: ./app process started
Jul 5 20:42:10 riffraff app: ./app process terminating


You can interface with the syslog service using scripts as well.

$ logger -p user.notice "hello again"
# cat /var/tmp/messages
.
.
.
Jul 5 20:42:07 riffraff app: ./app process started
Jul 5 20:42:10 riffraff app: ./app process terminating
Jul 5 21:04:06 riffraff lipeltgm: hello again


Pretty slick, huh?

17 June, 2007

Software Programming Paradigms

Oh, what did we do before Wikipedia!

My wife is defining some software development training materials focusing on the object-oriented programming paradigm. One of the slides presents the idea that object-oriented programming is _a_ programming technique, not _the_ programming technique.

Far too often I think we get caught up with a new technology and believe that it is the end-all of software development. Object-oriented programming, agile development, Ruby-on-rails, . . . I think we all too often jump on the new technology bandwagon. It is worth a lunch-hour to investigate some of the other programming paradigms.

http://en.wikipedia.org/wiki/Programming_paradigm