Gprof Profiling

Profiling is a mechanism for identifying program 'hotspots', identifying where your program spends a good majority of it's time. These regions are prime candidates for optimizations, where concentration of efficiency will give you the most bang for your buck.

Profiling with Gprof consists of 3-phases: 1) compiling your application with profiling enabled, 2) executing the application to gather profiling metrics, and 3) evaluating the collected metrics.

Compiling
You first must compile your application similar to the way you normally compile it. Two additional flags must be specified however, the -pg option to enable profiling, and the -g option to introduce the debugging symbols for tracking source lines of code. Actually, the -g option is only necessary for line-by-line profiling, but for good measure I suggest specifying it regardless.

$ gcc -pg -g foo.c -o foo

Executing
You run your application in the same manner as your normally run it, specifying the same arguments, inputs, outputs, . . . What you may notice however is that the program will execute slower than normal. This is reasonable when you consider what is taking place. Profiling metrics are collected during the execution of the program. Worth noting, your executable needs to terminate in a normal fashion, by returning from the main routine or calling exit().
Immediately prior to terminating the program, an output file (gmon.out) is generated that contains the collected profiling metrics. This file will later be used in evaluating the program performance.

Evaluation
Evaluation of the results stored in the gmon.out file is the last step in our quest. Various types of reports are available from the collected profiler info, the most readily used is the flat model. This is done by executing the command:

$ gprof foo

This results in the default report, a flat model followed by the call graph. The flat model will depict the list of functions in decreasing order with respect to time spent in each function. This allows for quick evaluation of the functions that your application spends most it's time in. You may notice that two functions appear in every profile; mcount and profil, each function is part of the profiling apparatus. The time spent in both functions can be viewed as profiler overhead.

FFMpeg Video Clip Generation

Hey all,

I just published my first two videos on YouTube.

I downloaded a couple videos from our Tivo, used tivodecode to decode to Mpegs and FFMpeg to extract 30 sec clips.

$ ffmpeg -ss 600 -t 30 -i infile.mpg outfile.mpg

Will result in seeking in 10 min, encoding the next 30 seconds from infile.mpg and saving to outfile.mpg

Theora/Ogg Example

I've recently been playing with Theora for encoding video. I found few examples for encoding raw video frames, and those I found were more complicated that I wanted. Below you will find my first toe-dip in the Theora realm. It basically generates a spinning dot in the middle of the frame and encodes to video at 10 frames per sec.

Have fun.

#define _FILE_OFFSET_BITS 64

#include <stdio.h>
#include <stdlib.h>
#include <ogg h="">
#include "theora/theora.h"
#include <string.h>
#include <math.h>

static FILE *ogg_fp = NULL;
static ogg_stream_state ogg_os;
static theora_state theora_td;
static theora_info theora_ti;

static int
theora_open(const char *pathname)
{
printf("(%s:%d) out filename: %s\n",__FILE__,__LINE__,pathname);
ogg_packet op;
ogg_page og;
theora_comment tc;

ogg_fp = fopen(pathname, "wb");
if(!ogg_fp) {
  fprintf(stderr, "%s: error: %s\n",
    pathname, "couldn't open output file");
  return 1;
}

if(ogg_stream_init(&ogg_os, rand())) {
  fprintf(stderr, "%s: error: %s\n",
    pathname, "couldn't create ogg stream state");
  return 1;
}

if(theora_encode_init(&theora_td, &theora_ti)) {
  fprintf(stderr, "%s: error: %s\n",
    pathname, "couldn't initialize theora encoding");
  return 1;
}

theora_encode_header(&theora_td, &op);
ogg_stream_packetin(&ogg_os, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
  fwrite(og.header, og.header_len, 1, ogg_fp);
  fwrite(og.body, og.body_len, 1, ogg_fp);
}

// encode a comment into the packet
theora_comment_init(&tc);
theora_encode_comment(&tc, &op);
ogg_stream_packetin(&ogg_os, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
  fwrite(og.header, og.header_len, 1, ogg_fp);
  fwrite(og.body, og.body_len, 1, ogg_fp);
}

theora_encode_tables(&theora_td, &op);
ogg_stream_packetin(&ogg_os, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
  fwrite(og.header, og.header_len, 1, ogg_fp);
  fwrite(og.body, og.body_len, 1, ogg_fp);
}

if(ogg_stream_flush(&ogg_os, &og)) {
  fwrite(og.header, og.header_len, 1, ogg_fp);
  fwrite(og.body, og.body_len, 1, ogg_fp);
}

return 0;
}

static int
theora_write_frame(unsigned long w, unsigned long h, unsigned char *yuv)
{
yuv_buffer yuv_buf;
ogg_packet op;
ogg_page og;

unsigned long yuv_w;
unsigned long yuv_h;

unsigned char *yuv_y;
unsigned char *yuv_u;
unsigned char *yuv_v;

unsigned int x;
unsigned int y;

/* Must hold: yuv_w >= w */
yuv_w = (w + 15) & ~15;

/* Must hold: yuv_h >= h */
yuv_h = (h + 15) & ~15;

yuv_y = malloc(yuv_w * yuv_h);
yuv_u = malloc(yuv_w * yuv_h / 4);
yuv_v = malloc(yuv_w * yuv_h / 4);

yuv_buf.y_width = yuv_w;
yuv_buf.y_height = yuv_h;
yuv_buf.y_stride = yuv_w;
yuv_buf.uv_width = yuv_w >> 1;
yuv_buf.uv_height = yuv_h >> 1;
yuv_buf.uv_stride = yuv_w >> 1;
yuv_buf.y = yuv_y;
yuv_buf.u = yuv_u;
yuv_buf.v = yuv_v;

for(y = 0; y <x =" 0;" y =" 0;" x =" 0;">> 1) + (y >> 1) * (yuv_w >> 1)] = 0;
    yuv_v[(x >> 1) + (y >> 1) * (yuv_w >> 1)] = 0;
  }
}

for(y = 0; y < x =" 0;" y =" 0;" x =" 0;">> 1) + (y >> 1) * (yuv_w >> 1)] =
      yuv[3 * (x + y * w) + 1];
    yuv_v[(x >> 1) + (y >> 1) * (yuv_w >> 1)] =
      yuv[3 * (x + y * w) + 2];
  }
}

if(theora_encode_YUVin(&theora_td, &yuv_buf)) {
  return 1;
}

if(!theora_encode_packetout(&theora_td, 0, &op)) {
  return 1;
}

ogg_stream_packetin(&ogg_os, &op);
if(ogg_stream_pageout(&ogg_os, &og)) {
  fwrite(og.header, og.header_len, 1, ogg_fp);
  fwrite(og.body, og.body_len, 1, ogg_fp);
}

free(yuv_y);
free(yuv_u);
free(yuv_v);

return 0;
}

static void
theora_close(void)
{
ogg_packet op;
ogg_page og;
static int theora_initialized = 0;

if (theora_initialized) {
  theora_encode_packetout(&theora_td, 1, &op);
  if(ogg_stream_pageout(&ogg_os, &og)) {
    fwrite(og.header, og.header_len, 1, ogg_fp);
    fwrite(og.body, og.body_len, 1, ogg_fp);
  }

  theora_info_clear(&theora_ti);
  theora_clear(&theora_td);

  fflush(ogg_fp);
  fclose(ogg_fp);
}

ogg_stream_clear(&ogg_os);
}

int
main(int argc, char *argv[])
{
int c;
int n;
unsigned i;
const char* const outFile="foo.ogg";
const unsigned NumFrames = 512;

const unsigned Width = 640/2;
const unsigned Height = 480/2;
const int video_fps_numerator = 30;
const int video_fps_denominator = 1;
const int video_aspect_numerator = 0;
const int video_aspect_denominator = 0;
const int video_rate = 0;
const int video_quality = 63;
theora_info_init(&theora_ti);

theora_ti.width = ((Width + 15) >>4)<<4; height =" ((Height">>4)<<4; frame_width =" Width;" frame_height =" Height;" offset_x =" 0;" offset_y =" 0;" fps_numerator =" video_fps_numerator;" fps_denominator =" video_fps_denominator;" aspect_numerator =" video_aspect_numerator;" aspect_denominator =" video_aspect_denominator;" colorspace =" OC_CS_UNSPECIFIED;" pixelformat =" OC_PF_420;" target_bitrate =" video_rate;" quality =" video_quality;" dropframes_p =" 0;" quick_p =" 1;" keyframe_auto_p =" 1;" keyframe_frequency =" 64;" keyframe_frequency_force =" 64;" keyframe_data_target_bitrate =" video_rate" keyframe_mindistance =" 8;" noise_sensitivity =" 1;" i="0;" frame="" d="" of="" n="" r="255;" const="" unsigned="" char="" y="(abs(R" 2104="" g="255;" 4130="" b="255;" 802="" 4096="">> 13);
  unsigned char U = (abs(R * -1214 + G * -2384 + B * 3598 + 4096 + 1048576) >> 13);
  unsigned char V = (abs(R * 3598 + G * -3013 + B * -585 + 4096 + 1048576) >> 13);
  // generate a frame with specified color
  {
    unsigned i;
    for(i = 0; i < r =" 0;" g =" 0;" b =" 255;" y =" (abs(R">> 13);
    unsigned char U = (abs(R * -1214 + G * -2384 + B * 3598 + 4096 + 1048576) >> 13);
    unsigned char V = (abs(R * 3598 + G * -3013 + B * -585 + 4096 + 1048576) >> 13);
    const unsigned cX = Width/2;
    const unsigned cY = Height/2;
    const unsigned Radius = 50;
    static double theta = 0.0;
    const unsigned x = Radius * sin(theta) + cX;
    const unsigned y = Radius * cos(theta) + cY;
    const unsigned k=3*(x + (Width*y));
    theta -= 5.0 * 3.14159/180.0;
    yuv[k] = Y;
    yuv[k+1] = G;
    yuv[k+2] = B;
  }

  if(theora_write_frame(Width, Height, yuv)) {
    theora_close();
    exit(1);
  }
}

theora_close();
return 0;
}

Logging to SysLog

Trace logging of significant events is a fundamental method for debugging and isolating errors encountered. User warnings, pop-up notification boxes, and application-level log files are one way to go. Another; to make use of the standard system log services found on *nix systems. The remainder of this post will some brief information I've recently discovered concerning interfacing with the system log services.

The syslog system import defines the interface:

   extern void syslog (int __pri, __const char *__fmt, ...)

which allows for variable-lengthed argument lists, much like printf. The interface is pretty simple, the first argument specifies a priority or type of log statement; the second, a formatted string in the same form that printf takes.

A simple C

#include <stdio.h>
#include <syslog.h>

int main(int argc, char *argv[]) {
  syslog(LOG_NOTICE,"%s process started", argv[0]);
  sleep(3);
  syslog(LOG_NOTICE,"%s process terminating", argv[0]);
}

Compile and running this example will result in two log entries in the system logs. Determining what log will contain the log statements can be determined by 1) the log type, and 2) the /etc/syslog.conf file. Note, that the log type is defined as LOG_NOTICE, and the entry in the syslog.conf file entry;

  *.=info;*.=notice;*.=warn;\
 auth,authpriv.none;\
 cron,daemon.none;\
 mail,news.none          -/var/log/messages

shows that the notice log entries will be evident in /var/log/messages.

Since the files are only accessible to superusers, logging in as root and tailing the log file should show the messages in the same manner as:

# cat /var/tmp/messages
.
.
.
Jul  5 20:30:18 riffraff syslogd 1.4.1#18: restart.
Jul  5 20:42:07 riffraff app: ./app process started
Jul  5 20:42:10 riffraff app: ./app process terminating

You can interface with the syslog service using scripts as well.

$ logger -p user.notice "hello again"
# cat /var/tmp/messages
.
.
.
Jul  5 20:42:07 riffraff app: ./app process started
Jul  5 20:42:10 riffraff app: ./app process terminating
Jul  5 21:04:06 riffraff lipeltgm: hello again

Pretty slick, huh?

Software Programming Paradigms

Oh, what did we do before Wikipedia!

My wife is defining some software development training materials focusing on the object-oriented programming paradigm. One of the slides presents the idea that object-oriented programming is _a_ programming technique, not _the_ programming technique.

Far too often I think we get caught up with a new technology and believe that it is the end-all of software development. Object-oriented programming, agile development, Ruby-on-rails, . . . I think we all too often jump on the new technology bandwagon. It is worth a lunch-hour to investigate some of the other programming paradigms.

http://en.wikipedia.org/wiki/Programming_paradigm

Whiz Kid

As I stated in a previous entry, I've recently got back from an SPIE conference. This has been the first I've ever attended and with luck this won't be the last.

An interesting event that occur ed at this conference was the presentation of a technique for 'head-locating' in infrared camera images. While the technique didn't exactly blow my skirt up...while I have some experience in computer vision that isn't my current focus. What did however get my attention was the presenter....a grade school kid with an internship with Johns Hopkins. Not only did he author the paper, he presented as well and by all rights did so with more enthusiasm and professionalism that many of the adults.

Sure, the kid can locate a humans head in a busy scene....he can use terms like 'modality' and solve differential equations....but dollars to donuts he's never seen a boobie first-hand.

That is the only comfort that lets me sleep at night.

Minnesota Bound

I am currently en route from Orlando, FL on my way back from an SPIE conference where a colleague authored and presented a paper. I tagged along as a result of my co-author status.....senor' coat-tails has been the title that comes to mind.

A number of interesting topics were presented, primarily concerned with wireless communications and fields of study concerning sensors. An observation worth noting was the evident differences in the disciplines of the presenters and how it affected the detail of the topic and the presentation style.

As I've said in the past, I am a software engineer by trade, having a BS and MS degree in Computer Science. Both school and industry alike, I've been surrounded by individuals both in Comp Sci as well as other engineering disciplines such as physics, electrical as well as mechanical engineering and the differences in the disciplines are very evident.

As much as a person wishes not to admit it....computer science is still a 'soft' science. Sure, it is far more disciplined than the likes of some other engineering persuasions, but is still far less structured as the likes of pure mathematics, physics, optics, electrical or mechanical engineering. The majority of the presentations were conducted by doctorates of physics, electrical or optical engineering, but every once in a while you'd get a computer science major up there and the presentation took a different tone. The best way I can say it is that the logic was softer or more fuzzy. Most presentations from more formal disciplines would be followed with mathematical description of the technology. Computer science presentations rarely had such formality, with the occasional exception to a presenter that had an undergrad degree in a more disciplined major and a PhD in computer science.

Am I knocking Comp Sci...to a degree I am doing just that. With the minor exception to those that pursue theoretical computer science focuses which are more in line with formal mathematics the majority of us still groan in the presence of a formal proof, a detailed description of the fundamental technology in the only pure language...mathematics. As a result, I think we give up the things that could make our discipline more successful. I can, for one, say that if we took a more structured look at proving techniques on paper rather than jump in and start coding we may have less error-prone technologies.

I'm not touting that we should all begin doing formal proofs on the effectiveness and efficiency of our authored algorithms. Instead, I'm saying that perhaps it's worth a 1/2 a day investigation on paper before firing up your trusty editor out of the gate.