Showing posts with label c++. Show all posts
Showing posts with label c++. Show all posts

28 September, 2013

Google ProtoBuff + ZeroMq -- C++

In the last series of posts we demonstrated ZeroMq as a technology that supports 'sockets on steroids', supporting multiple platforms as well as multiple languages.  The examples to-date have been transmitting strings between senders and receivers.  While interesting, to effectively create a distributed heterogeneous system we need to be capable of transmitting meaningful messages, preferably complex data structures rather than just strings.  That's where Google's Protobuff comes into play: http://code.google.com/p/protobuf/

Building off our previously created Ubuntu 12.04 32-bit VM, let's start by installing the additional necessary packages;


$ sudo apt-get install libprotoc-dev

With the developer libraries installed, we can now extend our previous C++ example to transmit a ProtoBuff message.

We'll extend our Makefile to add the necessary libraries and a target (e.g. msgs) to generate the C++ files for the message.


$ cat Makefile 
CC=g++
SRCS=main.cpp Messages.pb.cc
OBJS=$(subst .cpp,.o,$(SRCS))
INCLUDES += -I.
LIBS += -lpthread -lrt -lzmq -lprotobuf
.cpp.o:
$(CC) -c $<
main: msgs ${OBJS} 
${CC} ${CFLAGS} -o $@ ${OBJS} ${LIBS}
msgs:
${SH} protoc -I. --cpp_out=. Messages.proto
clean:
${RM} ${OBJS} main *.pb.*

Oh, we should take a look at our simple Protobuff message file:

$ cat Messages.proto 
message Person {
  required int32 id=1;
  required string name=2;
}

Finally, our extended main file:

$ cat main.cpp 
#include
#include
#include
#include
#include
#include
#include "Messages.pb.h"
void* ctx=zmq_init(1);
char* EndPoint="tcp://127.0.0.1:8000";
static const int N=100;
static const int BufferSize=128;
void* sender(void*)
{
  printf("(%s:%d) running\n",__FILE__,__LINE__);
  void* pub=zmq_socket(ctx, ZMQ_PUB);
  assert(pub);
  int rc=zmq_bind(pub,EndPoint);
  assert(rc==0);
  Person p;
  p.set_name("fatslowkid");
  p.set_id(01);
  for(int i=0; i
  {
    zmq_msg_t msg;
    std::string S=p.SerializeAsString();
    char* content=(char*)S.c_str();
    int rc=zmq_msg_init_size(&msg, BufferSize);
    assert(rc==0);
    rc=zmq_msg_init_data(&msg, content, strlen(content), 0,0);
    assert(rc==0);
    rc=zmq_send(pub, &msg, 0);
    assert(rc==0);
    ::usleep(100000);
  }
}
void* receiver(void*)
{
  printf("(%s:%d) running\n",__FILE__,__LINE__);
  void* sub=zmq_socket(ctx, ZMQ_SUB);
  assert(sub);
  int rc=zmq_connect(sub,EndPoint);
  assert(rc==0);
  char* filter="";
  rc=zmq_setsockopt(sub, ZMQ_SUBSCRIBE, filter, strlen(filter));
  assert(rc==0);
  for(int i=0; i
  {
    zmq_msg_t msg;
    zmq_msg_init_size(&msg, BufferSize);
    const int rc=zmq_recv (sub, &msg, 0);
    char* content=(char*)zmq_msg_data(&msg);
    Person p;
    p.ParseFromString(content);
    printf("(%s:%d) received: '%s'\n",__FILE__,__LINE__,p.name().c_str());
    zmq_msg_close(&msg);
  }
}
int main(int argc, char* argv[])
{
  printf("(%s:%d) main process initializing\n",__FILE__,__LINE__);
  int major, minor, patch;
  zmq_version (&major, &minor, &patch);
  printf("(%s:%d) zmq version: %d.%d.%d\n",__FILE__,__LINE__,major,minor,patch);
  pthread_t rId;
  pthread_create(&rId, 0, receiver, 0);
  pthread_t sId;
  pthread_create(&sId, 0, sender, 0);
  pthread_join(rId,0);
  pthread_join(sId,0);
  printf("(%s:%d) main process terminating\n",__FILE__,__LINE__);
}

Notice that we now transmit and receive Protobuf messages, serialized as strings.  The value of this is that the serialization mechanism is multi-platform & multi-language support.

Cheers.

14 September, 2013

ZeroMq Installation -- C++

ZeroMq, or 0Mq if you prefer, is considered 'sockets on steroids'.  The framework is aimed at simplifying distributed communications, serves as a concurrency framework, and supports more languages than any normal mortal would ever require.
http://zeromq.org

The topic of this post will be guiding through the installation on Ubuntu 12.04 and writing enough example code to verify all is happy.  Future posts will focus on installation of alternative, and interesting, languages with the overall intent of demonstrating interoperability between the languages.

Let's get started;

Our target installation machine is Ubuntu 12.04 64-bit Desktop edition; http://www.ubuntu.com/download/desktop.  Your mileage may vary with alternative distributions.  I'm starting with a new default installation and will work through the installation process from here on.


$ sudo apt-get install libzmq-dev


After the APT package handling utility is done humping package installations you should have a function, albeit dated, version to begin working with.

While ZeroMq supports a plethora of languages, the focus of this post is on C++.  That said, we'll need to install GCC C++ compiler.


$ sudo apt-get install g++


As I said earlier, the Ubuntu repository has a fairly old version.  At time of this writing, v3.2.3 was the latest stable release but for the purposes of this post we'll continue to use this version.

Let's confirm what version we have by authoring our first C++ application and ensure we can successfully compile/link with the installed libraries.

Start with a simple Makefile;


$ cat Makefile



CC=g++



SRCS=main.cpp

OBJS=$(subst .cpp,.o,$(SRCS))

INCLUDES += -I.

LIBS += -lpthread -lrt -lzmq



.cpp.o:

 $(CC) -c $<



main: ${OBJS}

 ${CC} ${CFLAGS} -o $@ ${OBJS} ${LIBS}



clean:

 ${RM} ${OBJS} main


And now let's look as a trival main program that confirms we can compile/link with Zmq libraries;



$ cat main.cpp



#include 

#include 



int main(int argc, char* argv[])

{

  printf("(%s:%d) main process initializing\n",__FILE__,__LINE__);

  int major, minor, patch;

  zmq_version (&major, &minor, &patch);

  printf("(%s:%d) zmq version: %d.%d.%d\n",__FILE__,__LINE__,major,minor,patch);

  printf("(%s:%d) main process terminating\n",__FILE__,__LINE__);

}



Running it confirms the installed Zmq libraries are near ancient, but again ok for our short-term purposes.


$ ./main



(main.cpp:6) main process initializing

(main.cpp:6) zmq version: 2.1.11
(main.cpp:6) main process terminating

Let's extend this to make it a bit more interesting, classic sender/receiver communication model using the PUB/SUB comm model provided by ZeroMq.



$ cat main.cpp



#include 

#include 

#include 

#include 

#include 

#include 



void* ctx=zmq_init(1);

char* EndPoint="tcp://127.0.0.1:8000";

static const int N=100;

static const int BufferSize=128;



void* sender(void*)

{

  printf("(%s:%d) running\n",__FILE__,__LINE__);

  void* pub=zmq_socket(ctx, ZMQ_PUB);

  assert(pub);

  int rc=zmq_bind(pub,EndPoint);

  assert(rc==0);

  for(int i=0; i  {

    char content[BufferSize];

    sprintf(content, "message %03d",i);

    zmq_msg_t msg;

    int rc=zmq_msg_init_size(&msg, BufferSize);

    assert(rc==0);

    rc=zmq_msg_init_data(&msg, content, strlen(content), 0,0);

    assert(rc==0);

    printf("(%s:%d) sending '%s'\n",__FILE__,__LINE__,content);

    rc=zmq_send(pub, &msg, 0);

    assert(rc==0);

    ::usleep(100000);

  }

}



void* receiver(void*)

{

  printf("(%s:%d) running\n",__FILE__,__LINE__);

  void* sub=zmq_socket(ctx, ZMQ_SUB);

  assert(sub);

  int rc=zmq_connect(sub,EndPoint);

  assert(rc==0);

  char* filter="";

  rc=zmq_setsockopt(sub, ZMQ_SUBSCRIBE, filter, strlen(filter));

  assert(rc==0);

  for(int i=0; i  {

    zmq_msg_t msg;

    zmq_msg_init_size(&msg, BufferSize);

    const int rc=zmq_recv (sub, &msg, 0);

    char* content=(char*)zmq_msg_data(&msg);

    printf("(%s:%d) received: '%s'\n",__FILE__,__LINE__,content);

    zmq_msg_close(&msg);

  }

}



int main(int argc, char* argv[])

{

  printf("(%s:%d) main process initializing\n",__FILE__,__LINE__);

  int major, minor, patch;

  zmq_version (&major, &minor, &patch);

  printf("(%s:%d) zmq version: %d.%d.%d\n",__FILE__,__LINE__,major,minor,patch);



  pthread_t rId;

  pthread_create(&rId, 0, receiver, 0);



  pthread_t sId;

  pthread_create(&sId, 0, sender, 0);



  pthread_join(rId,0);

  pthread_join(sId,0);



  printf("(%s:%d) main process terminating\n",__FILE__,__LINE__);

}


Running this beauty will result in the following;

$ cat /var/tmp/ZmqMb/main.log 
(main.cpp:59) main process initializing
(main.cpp:62) zmq version: 2.1.11
(main.cpp:15) running
(main.cpp:29) sending 'message 000'
(main.cpp:38) running
(main.cpp:29) sending 'message 001'
(main.cpp:52) received: 'message 001'
(main.cpp:29) sending 'message 002'
(main.cpp:52) received: 'message 002'
(main.cpp:29) sending 'message 003'
(main.cpp:52) received: 'message 003'
(main.cpp:29) sending 'message 004'
(main.cpp:52) received: 'message 004'
(main.cpp:29) sending 'message 005'
(main.cpp:52) received: 'message 005'
(main.cpp:29) sending 'message 006'
(main.cpp:52) received: 'message 006'
(main.cpp:29) sending 'message 007'
(main.cpp:52) received: 'message 007'
(main.cpp:29) sending 'message 008'
(main.cpp:52) received: 'message 008'
(main.cpp:29) sending 'message 009'
(main.cpp:52) received: 'message 009'
(main.cpp:29) sending 'message 010'
(main.cpp:52) received: 'message 010'
(main.cpp:29) sending 'message 011'
(main.cpp:52) received: 'message 011'


You'll notice that the first message was lost.  I've never had a sufficient understanding of why this happens, but typically the first message can be lost, primarily due to the pub/sub sockets not necessarily being fully established.  Issuing a 'sleep' between socket creation and between the receiver and sender socket usage doesn't solve the problem, it's more involved than that and I have little to offer as an explanation.

So, there you go; a simple usage of ZeroMq.

Cheers.





11 March, 2008

Global Variables in C

First the disclaimers;
Global variables are bad. Global variables are evil. Global variables are, and will continue to be much like a 6-ft chainsaw, extremely dangerous to use, can regularly be replaced with a safer tool/technique, but are sometimes the proper tool for the job (like cutting down 12-ft diameter trees). None-the-less, on the rare occasion that a global variable is the proper technique, you'll need to know how to define and use them...the subject of the remainder of this post.

The following example will be used to explain techniques for defining and using globals, we will speak to it for the remainder of the post.

package1.h

1 #ifndef PACKAGE1_H
2 #define PACKAGE1_H
3 static const float GlobFloat = 3.14159;
4 static double GlobDouble = 1.23456;
5 extern unsigned GlobUnsigned;
6 //char GlobChar='a';
7
8 void func1(void);
9 #endif

package1.c

1 #include "package1.h"
2 #include
3
4 static int PackageGlobalInt = 1;
5 unsigned GlobUnsigned = 12345;
6
7 void func1() {
8 printf("(%s:%d) PackageGlobalInt(%d)\n",__FILE__,__LINE__,PackageGlobalInt++);
9 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
10 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
11 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
12 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
13 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
14 }
15

package2.h

1 #ifndef PACKAGE2_H
2 #define PACKAGE2_H
3 #include "package1.h"
4
5 extern long GlobLong;
6 void func2(void);
7 #endif

package2.c

1 #include "package2.h"
2 #include
3
4 static int PackageGlobalInt = 2;
5 long GlobLong = 987654321;
6
7 void func2() {
8 printf("(%s:%d) PackageGlobalInt(%d)\n",__FILE__,__LINE__,PackageGlobalInt++);
9 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
10 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
11 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
12 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong++);
13 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
14 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
15 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong++);
16 }
17

main.c

1 #include
2 #include "package1.h"
3 #include "package2.h"
4
5 int main() {
6 printf("(%s:%d) main process initializing\n",__FILE__,__LINE__);
7
8 func1();
9 func2();
10
11 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
12 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble);
13 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned);
14 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong);
15
16
17 printf("(%s:%d) main process terminating\n",__FILE__,__LINE__);
18 }
19


Globals are a form of interface, so rightly are typically defined in a header file. Package 1 attempts to define 3 global variables. You will soon see that it was only successful in defining one. Before we get into the details, we should review some preliminaries.

#includes
As you should already be aware, header files serve little purpose other than organization of code. When encountering a #include directive, the precompiler does little more than copy-n-pasting the contents into the sourcing file. This is significant in our later discussion.

Static Variables
Static variables cannot be accessed outside a translation unit. For all intents and purposes, a translation unit is a C source file. This, along with what we know about #includes will explain why only one of our intended global variable declarations was successful.

Back To Our Example
The contents of package1.h on lines 3-5 are three attempts to declare a global variable. Note, lines 3-4 define the variables as static, line 5 defining it as external linkage. Why are the first two variables defined as static??? Because if I don't, I get a linker error "multiple definition of . . ." error. Pounding on the keyboard, slowly beating the linker into submission is accomplished by declaring the variable as static. Finally, the linker shuts up; we must have fixed it, right? Wrong!

So, why doesn't lines 3-4 work? More importantly, how do we know it doesn't work? Well, we can confirm that line 4 didn't work by comparing the results of lines package1.c:10, package1.c:12, package2.c:10, package2.c:13, and main.c:12. Note that on package 1 & 2 functions, we are incrementing the contents of GlobDouble. If we were successful in defining a global variable, we should see the variable initially assigned 1.23456, incremented 4 times with a final result of 5.23456.


~/StorageClasses$ ./main | grep GlobDouble
(package1.c:10) GlobDouble(1.234560)
(package1.c:12) GlobDouble(2.234560)
(package2.c:10) GlobDouble(1.234560)
(package2.c:13) GlobDouble(2.234560)
(main.c:12) GlobDouble(1.234560)

Hmmmph, we didn't accomplish what we intended, did we? The static storage class, coupled with the precompiler copy-n-paste behavior resulted in 3 independent variable declarations; one in package1.c, one in package2.c, and one in main.c. Each instance is unique to each translation unit, so incrementing one only affects the declaration in that translation unit.

Line 5 however successfully defined a global variable. How? Well, line 5 simply states that we are declaring a variable, to be defined elsewhere. Declaration simply defines a reference to variable that will be defined elsewhere (in this case, line 5 in package1.c). Translation units: package1.c, package2.c and main.c will each declare a reference to GlobUnsigned, but the single definition of the variable resides in package1.c.

We can confirm that all is well by means of:


~/StorageClasses$ ./main | grep GlobUnsigned
(package1.c:11) GlobUnsigned(12345)
(package1.c:13) GlobUnsigned(12346)
(package2.c:11) GlobUnsigned(12347)
(package2.c:14) GlobUnsigned(12348)
(main.c:13) GlobUnsigned(12349)



It is worth noting that lines 3-4 in package1.h have identical results as line 4 in both package1.c and package2.c. All instances are similar in the matter that their scope is limited to the translation unit in which they are declared.

10 March, 2008

Formal Names For Common C Techniques

If you ever written a C application with custom-defined header files, you've used the pre-processor pattern:


#ifndef FOO_H
#define FOO_H
.
.
.
#endif


Ever wonder what this pattern was called? I never did, until the other day when I searched for it. The pattern is referred to as the 'once-only header' pattern, sometimes called the 'wrapper #ifdef'.

Call it what you want, the intention is to ensure that the header is not copy-n-pasted multiple times in a translation unit, due to multiple direct or indirect includes. The FOO_H is referred to as the 'guard' or 'controlling macro'.

Another common technique that you may have made use of goes something like this:

#ifdef SYS1
#include "sys1.h"
#elif . . .
.
.
.
#endif

This is referred to as a 'computed includes' pattern or technique. Not much else to add, just a name for a common face.

16 September, 2007

Command Line Processing with Popt

General practice for writing flexible applications lend toward the ability to specify command-line arguments to tailor the behavior of your application. For instance, a common practice of specifying -v to enable a verbose mode where debugging information is echoed as your application runs. Specifying a unique port for networking applications, or enablement/disablement of an application gui are two other examples that are common. Since command line arguments can take the flavor of integer, floats, doubles, strings...the code for parsing these arguments can be far from simple. Tailoring flexible parsers every time you develop an application is labor-intensive and a waste of time if a general mechanism were available.

Lucky for us, such a mechanism does exist; namely the Popt library.

A short example of an application which uses the library follows:

1 #include
2 #include
3 #include
4
5 int main(int argc, char **argv) {
6 /* option parsing variables */
7 char ch;
8 poptContext opt_con; /* context for parsing command-line options */
9 char *extra_arg;
10 int i=0;
11 char *s="";
12 float f=0.0;
13 double d=0.0;
14 int verbose=0;
15
16 static struct poptOption options_table[] = {
17 { "integer", 'i', POPT_ARG_INT, &i, 'i', "grab an integer", "INT" },
18 { "string", 's', POPT_ARG_STRING, &s, 's', "grab a string", "STRING" },
19 { "float", 'f', POPT_ARG_FLOAT, &f, 'f', "grab a float", "FLOAT" },
20 { "double", 'd', POPT_ARG_DOUBLE, &d, 'd', "grab a double", "DOUBLE" },
21 { "verbose", 'v', POPT_ARG_NONE, NULL, 'v', "enable verbose", "" },
22 POPT_AUTOHELP
23 { NULL, 0, 0, NULL, 0 } /* end-of-list terminator */
24 };
25
26 opt_con = poptGetContext(NULL, argc, (const char **)argv, options_table, 0);
27
28 /* Now do options processing */
29 while ((ch = poptGetNextOpt(opt_con)) >= 0) {
30 printf("between while & switch: ch = %c\n", ch);
31 switch (ch) {
32 case 'i':
33 printf("handling 'i' option.\n");
34 break;
35 case 's':
36 printf("handling 's' option.\n");
37 break;
38 case 'f':
39 printf("handling 'f' option.\n");
40 break;
41 case 'd':
42 printf("handling 'd' option.\n");
43 break;
44 case 'v':
45 printf("handling 'v' option.\n");
46 verbose = 1;
47 break;
48 }
49 }
50
51 if (ch < -1) { 52 // the user specified an invalid option, tell them 53 poptPrintHelp(opt_con, stderr, 0); 54 } 55 56 /* non-option args */ 57 while (extra_arg = (char *)poptGetArg(opt_con)) { 58 printf("extra arg: %s\n", extra_arg); 59 exit(1); 60 } 61 62 63 /* cleanup */ 64 poptFreeContext(opt_con); 65 66 printf("(%s:%d) i = %d\n",__FILE__,__LINE__,i); 67 printf("(%s:%d) s = '%s'\n",__FILE__,__LINE__,s); 68 printf("(%s:%d) f = %f\n",__FILE__,__LINE__,f); 69 printf("(%s:%d) d = %lf\n",__FILE__,__LINE__,d); 70 printf("(%s:%d) v = %d\n",__FILE__,__LINE__,verbose); 71 72 73 return EXIT_SUCCESS; 74 }


You compile and link the application by:
g++ -Wall   main.o  -lpopt -o main


An added bonus of using this application is the automatic introduced interfaces for a brief and more verbose help descriptions.


~/Desktop/sourceCode/C/popt$ ./main --help
Usage: main [OPTION...]
-i, --integer=INT grab an integer
-s, --string=STRING grab a string
-f, --float=FLOAT grab a float
-d, --double=DOUBLE grab a double
-v, --verbose enable verbose

Help options:
-?, --help Show this help message
--usage Display brief usage message



~/Desktop/sourceCode/C/popt$ ./main --usage
Usage: main [-v?] [-i|--integer INT] [-s|--string STRING] [-f|--float FLOAT]
[-d|--double DOUBLE] [-v|--verbose] [-?|--help] [--usage]



The heart of the library lies with proper initialization of the popt structure which is defined as follows:

struct poptOption {
const char * longName;
char shortName;
int argInfo;
void * arg;
int val;
const char * descrip;
const char * argDescrip;
};

The ability to specify either a long or short argument name is common practice; -h or --help is a common example of this form.
The long and short forms are specified as the 1st and 2nd element in the popt structure. The 3rd and 4th specify of what type the following argument consists of as well as the address in which the value will be stored. The 6th and 7th arguments specify the argument description and argument field name which is displayed in the help and brief. The 5th field is a bit of a mystery at this time.

That's all for now.