Getting Started with YARP Devices

For robotics, we need access to all sorts of strange devices. There are three separate concerns related to devices in YARP:

• Implementing specific drivers for particular devices
• Defining interfaces for device families
• Implementing network wrappers for interfaces

The first step, creating drivers for particular devices, is obvious; every robotics project needs to interface with hardware somehow.

The second step, defining interfaces for families of devices, is important in the longer term. If you change your camera or your motor control board, how much of your code needs to change too? If you view your devices through well thought out interfaces, the impact of device change can be minimized.

The third step, network wrappers, is important to give flexibility. You can scale up your computing cluster, or isolate hardware devices that don't play well together, or have specific OS dependencies etc.

A driver in YARP is just a C++ class. Interfaces are simply abstract base classes. And network wrappers are just special cases of devices that happen to use network resources to satisfy their interfaces. Let's look at some examples to make this clear.

Framegrabber interfaces

We call devices that produce a stream of images "framegrabbers" for historical reasons. There are a number of interfaces associated with framegrabbers. Here's one, IFrameGrabberImage: (taken from FrameGrabberInterfaces.h in the YARP source code)

class yarp::dev::IFrameGrabberImage {
public:
    virtual bool getImage(yarp::sig::ImageOf<yarp::sig::PixelRgb>& image) = 0;
    virtual int height() const = 0;
    virtual int width() const = 0;
};

Notice that IFrameGrabberImage doesn't do anything itself -- it just declares methods to get an image and image dimensions. There are several classes in YARP that implement this interface (in other words, they inherit from it and define the methods). One example is DragonflyDeviceDriver. This interfaces with a Point Grey digital camera, using different libraries on different operating systems. It implements several interfaces; here are just two of them:

Since you may not have any hardware devices available to you right now, let's make a "fake" framegrabber that implements the IFrameGrabberImage interface:

class FakeFrameGrabber : public yarp::dev::IFrameGrabberImage {
private:
    int w, h;
public:
    FakeFrameGrabber(int w, int h) : w(w), h(h) {
    }

    virtual bool getImage(yarp::sig::ImageOf<yarp::sig::PixelRgb>& image) {
        Time::delay(0.5);  // simulate waiting for hardware to report
        image.resize(w,h);
        image.zero();
        return true;
    }

    virtual int height() const {
        return h;
    }

    virtual int width() const {
        return w;
    }
};

All this does is return blank images roughly every half second, but it does indeed implement the IFrameGrabberImage interface.

We very nearly have a YARP device driver. Sometimes we want to be able to create devices flexibly, based on a configuration file, or without having to worry about where their header file is. For these purposes, YARP requires that:

• All device drivers inherit from the "DeviceDriver" class.
• All device drivers implement an open and close method.
• Device drivers should be using no hardware resources before open is called or after close is called. YARP in return will promise that open will be the first method called after construction, and that if open returns false no other methods will be called before destruction. The open method takes a configuration object which can contain arbitrary properties.

This is quite straightforward to do for our fake framegrabber (this is from example/dev/FakeFrameGrabber.h in the YARP source code):

#include <yarp/os/all.h>
#include <yarp/sig/all.h>

class FakeFrameGrabber : public yarp::dev::IFrameGrabberImage, 
                         public yarp::dev::DeviceDriver {
private:
    int w, h;
    int ct;
    yarp::sig::PixelRgb circlePixel;
public:
    FakeFrameGrabber() {
        h = w = 0;
        ct = 0;
        circlePixel = yarp::sig::PixelRgb(255,0,0);
    }

    bool open(int w, int h) {
        this->w = w;
        this->h = h;
        return w>0 && h>0;
    }

    virtual bool open(yarp::os::Searchable& config) { 
        // extract width and height configuration, if present
        // otherwise use 128x128
        int desiredWidth = config.check("w",yarp::os::Value(128)).asInt();
        int desiredHeight = config.check("h",yarp::os::Value(128)).asInt();
        return open(desiredWidth,desiredHeight);
    }

    virtual bool close() { 
        return true; // easy
    }

    virtual bool getImage(yarp::sig::ImageOf<yarp::sig::PixelRgb>& image) {
        yarp::os::Time::delay(0.1);  // simulate waiting for hardware to report
        image.resize(w,h);
        image.zero();
        yarp::sig::draw::addCrossHair(image,circlePixel,
                                      ct,h/2,h/8);
        ct = (ct+4)%w;
        return true;
    }

    virtual int height() const {
        return h;
    }

    virtual int width() const {
        return w;
    }
};

Now we have a good YARP device driver (even if it is fake). We've chosen to require that the width and height of images be greater than zero, which seems reasonable. (A moving crosshair has been added to the image just to make things more interesting).

Creating and configuring the new device

We can create, configure, and use our device directly, without any bureaucracy:

FakeFrameGrabber fakey;
fakey.open(640,480);
ImageOf<PixelRgb> img;
fakey.getImage(img);
...

If we're smart, we'd make as much of our code as possible depend just on the interface IFrameGrabberImage, so that we can reuse it or substitute in a different framegrabber later:

// creation and configuration -- depends on specific device type
FakeFrameGrabber fakey;
fakey.open(640,480);
IFrameGrabberImage& genericGrabber = fakey;
// from here on, we only care that our device implements IFrameGrabberImage
ImageOf<PixelRgb> img;
genericGrabber.getImage(img);
...

This is a standard software engineering technique for minimizing unnecessary coupling between modules.

Suppose we want to go further, and let the framegrabber we use be controlled by a command line option. So it could either be our FakeFrameGrabber, or a real device like DragonflyDeviceDriver or PicoloDeviceDriver (another framegrabber), or RemoteFrameGrabber (a proxy for a framegrabber device on another machine).

YARP comes with a helper for doing this. It maintains a simple database of the drivers you have compiled and available. You can see this database by running "yarpdev", which tells you something like:

...
Here are devices listed for your system:

Device <dragonfly>
   documented by the C++ class DragonflyDeviceDriver
   Wrapped for the network by <grabber>

Device <test_grabber>
   documented by the C++ class TestFrameGrabber
   Wrapped for the network by <grabber>

Device <grabber>
   documented by the C++ class ServerFrameGrabber
   Does not need a network wrapper.
...

We see a "dragonfly" listing associated with the DragonflyDeviceDriver class -- "dragonfly" is this devices common name to which it is referred to in configuration files, command line options, etc.. There is also a "test_grabber" listing associated with a TestFrameGrabber class -- this is in fact a more elaborate version of the fake framegrabber we've been working on, for testing purposes. We can ignore it for now. There is also a "grabber" device listed, with is a network wrapper that works for all framegrabber devices.

You can instantiate any device listed here from your code as follows:

  #include <yarp/dev/PolyDriver.h>
  #include <yarp/dev/FrameGrabberInterface.h>
  using namespace yarp::dev;
  PolyDriver dd("dragonfly");
  if (!dd.isValid()) {
    printf("Dragonfly not available\n");
    exit(1);
  }
  IFrameGrabberImage *grabber;
  dd.view(grabber);
  if (grabber!=NULL) {
    printf("*** Device can supply images\n");
    ImageOf<PixelRgb> img;
    if (grabber->getImage(img)) {
      printf("*** Got a %dx%d image\n", img.width(), img.height());
    }
  }

Just replace the name "dragonfly" with the device you want.

If we want to be able to create a FakeFrameGrabber through the same mechanism, we need to add a factory for it to the device driver database. Here's how:

DriverCreator *fakey_factory = 
   new DriverCreatorOf<FakeFrameGrabber>("fakey","grabber","FakeFrameGrabber");
Drivers::factory().add(fakey_factory); // hand factory over to YARP

The name "fakey" is an arbitrary common name we pick. The name "grabber" gives a network wrapper for this device ("" if there is none). The class name "FakeFrameGrabber" is recorded to let the user know where to look for documentation on this device.

Now we can do:

  PolyDriver dd("fakey");
  IFrameGrabberImage *grabber;
  dd.view(grabber);
  if (grabber==NULL) { exit(1); } // failed
  // from here on, we only care that our device implements IFrameGrabberImage
  ImageOf<PixelRgb> img;
  grabber->getImage(img);
  ...

This form is calling FakeFrameGrabber::open without any configuration information set up. To pass in configuration options, we do:

  Property config("(device fakey) (w 640) (h 480)");
  PolyDriver dd(config);
  ...

Or to get options from the command line, we do:

  Property config;
  config.fromCommand(argc,argv) // parse command line
  PolyDriver dd(config);
  ...

As a review, here's a full program with what we've done: (from example/dev/fake_grabber.cpp):

#include <stdio.h>
#include <yarp/dev/FrameGrabberInterfaces.h>
#include <yarp/dev/PolyDriver.h>
#include <yarp/dev/Drivers.h>
#include "FakeFrameGrabber.h"
using namespace yarp::os;
using namespace yarp::sig;
using namespace yarp::dev;

int main(int argc, char *argv[]) {
  Network::init();

  // give YARP a factory for creating instances of FakeFrameGrabber
  DriverCreator *fakey_factory = 
    new DriverCreatorOf<FakeFrameGrabber>("fakey",
                                          "grabber",
                                          "FakeFrameGrabber");
  Drivers::factory().add(fakey_factory); // hand factory over to YARP

  // use YARP to create and configure an instance of FakeFrameGrabber
  Property config;
  if (argc==1) {
    // no arguments, use a default
    config.fromString("(device fakey) (w 640) (h 480)");
  } else {
    // expect something like "--device fakey --w 640 --h 480"
    //                    or "--device dragonfly"
    //                    or "--device test_grabber --period 0.5 --mode [ball]"
    config.fromCommand(argc,argv);
  }
  PolyDriver dd(config);
  if (!dd.isValid()) {
    printf("Failed to create and configure a device\n");
    exit(1);
  }
  IFrameGrabberImage *grabberInterface;
  if (!dd.view(grabberInterface)) {
    printf("Failed to view device through IFrameGrabberImage interface\n");
    exit(1);
  }

  ImageOf<PixelRgb> img;
  grabberInterface->getImage(img);
  printf("Got a %dx%d image\n", img.width(), img.height());

  dd.close();

  Network::fini();
  return 0;
}

This assumes we placed the FakeFrameGrabber class in a header file called "FakeFrameGrabber.h".

Here are a few ways to call the program, all of which result in different devices being used:

./fake_grabber --device fakey --w 640 --h 480
./fake_grabber --device dragonfly
./fake_grabber --device test_grabber --period 0.5 --mode [ball]

The last possibility, test_grabber, is a device driver very much like fakey that is already compiled into the YARP device library, libYARP_dev.

Since fakey implements the IFrameGrabberImage, we can use the grabber (ServerFrameGrabber) network wrapper to put it on the network: (from example/dev/fake_grabber_net.cpp)

#include <stdio.h>
#include <yarp/dev/FrameGrabberInterfaces.h>
#include <yarp/dev/PolyDriver.h>
#include <yarp/dev/Drivers.h>
#include "FakeFrameGrabber.h"
using namespace yarp::os;
using namespace yarp::sig;
using namespace yarp::dev;

int main() {
  Network::init();

  // give YARP a factory for creating instances of FakeFrameGrabber
  DriverCreator *fakey_factory = 
    new DriverCreatorOf<FakeFrameGrabber>("fakey",
                                          "grabber",
                                          "FakeFrameGrabber");
  Drivers::factory().add(fakey_factory); // hand factory over to YARP

  // use YARP to create and configure a networked of FakeFrameGrabber
  Property config;
  config.fromString("(device grabber) (name /fakey) (subdevice fakey) (w 200) (h 200)");
  PolyDriver dd(config);
  if (!dd.isValid()) {
    printf("Failed to create and configure a device\n");
    exit(1);
  }

  // snooze while the network device operates
  while (true) {
    printf("Network device is active...\n");
    Time::delay(5);
  }

  Network::fini();
  return 0;
}

If you run this program, you should see something like:

Subdevice fakey
Created device <fakey>.  See C++ class FakeFrameGrabber for the interfaces supported.
yarp: Port /fakey listening at tcp://17.166.9.122:10002
Created wrapper <grabber>.  See C++ class ServerFrameGrabber for the interfaces supported.
Network device is active...
Server grabber starting
Network framegrabber writing a 200x200 image...
Network device is active...
Network device is active...

The YARP port "/fakey" is now a network access point for an instance of our fake framegrabber. One way to see this is by starting a viewer:

./yarpview --name /view
yarp connect /fakey /view

You should see a moving red crosshair.

We can also access the framegrabber via code (example/dev/grabber_client.cpp):

#include <stdio.h>
#include <yarp/os/all.h>
#include <yarp/sig/all.h>
#include <yarp/dev/FrameGrabberInterfaces.h>
#include <yarp/dev/PolyDriver.h>
#include <yarp/dev/Drivers.h>
using namespace yarp::os;
using namespace yarp::sig;
using namespace yarp::dev;

int main() {
  Network::init();

  Property config;
  config.fromString("(device remote_grabber) (local /client) (remote /fakey)");
  PolyDriver dd(config);
  if (!dd.isValid()) {
    printf("Failed to create and configure device\n");
    exit(1);
  }
  IFrameGrabberImage *grabberInterface;
  if (!dd.view(grabberInterface)) {
    printf("Failed to view device through IFrameGrabberImage interface\n");
    exit(1);
  }

  ImageOf<PixelRgb> img;
  grabberInterface->getImage(img);
  printf("Got a %dx%d image\n", img.width(), img.height());

  dd.close();

  Network::fini();
  return 0;
}

If you have left fake_grabber_net running, and now run this, you should see:

yarp: Port /client listening at tcp://172.16.93.1:10002
yarp: Receiving input from /fakey to /client using tcp
yarp: Sending output from /client to /fakey using tcp
Created device <remote_grabber>.  See C++ class RemoteFrameGrabber for the interfaces supported.
Got a 200x200 image
yarp: Removing input from /fakey to /client
yarp: Removing output from /client to /fakey

Note the image size is correct. Here's a rough diagram of what's going on:

For code that depends only on IFrameGrabberImage, there is no apparent difference between a FakeFrameGrabber and RemoteFrameGrabber.

Framegrabber control interface

Let's look at another interface: IFrameGrabberControls (taken from FrameGrabberInterfaces.h in the YARP source code)

class yarp::dev::IFrameGrabberControls {
public:
    virtual bool setBrightness(double v) = 0;
    virtual bool setShutter(double v) = 0;
    virtual bool setGain(double v) = 0;
    virtual double getBrightness() const = 0;
    virtual double getShutter() const = 0;
    virtual double getGain() const = 0;
};

Let's use this interface to control the brightness of the shape shown by FakeFrameGrabber (this is from example/dev/FakeFrameGrabber2.h in the YARP source code, make sure you update from CVS for this section):

#include <yarp/os/all.h>
#include <yarp/sig/all.h>

class FakeFrameGrabber : public yarp::dev::IFrameGrabberImage, 
                         public yarp::dev::IFrameGrabberControls, 
                         public yarp::dev::DeviceDriver {
private:
    ...
    yarp::sig::PixelRgb circlePixel;
    double brightness;
public:
    ...
    virtual bool setBrightness(double v) {
        if (v>1) v = 1;
        if (v<0) v = 0;
        circlePixel = yarp::sig::PixelRgb((unsigned char)(255*v),0,0);
        brightness = v;
        return true;
    }

    virtual bool setShutter(double v) {
        return false;
    }

    virtual bool setGain(double v) {
        return false;
    }

    virtual double getBrightness() const {
        return brightness;
    }

    virtual double getShutter() const {
        return 0;
    }

    virtual double getGain() const {
        return 0;
    }
};

As before, the grabber (ServerFrameGrabber) network wrapper will put this on the network just as before (see and run example/dev/fake_grabber_net2.cpp).

And as before, we can see the image produced by starting a viewer:

./yarpview --name /view
yarp connect /fakey /view

You should see a moving red crosshair.

But now we can also query and command the fake framegrabber. From code, we can use the remote_grabber device driver,which implements IFrameGrabberControls. From the command line, we can use yarp rpc:

yarp rpc /fakey
RPC connection to /fakey at tcp://172.16.232.1:10002 (connection name anon_rpc)
[get] [bri]
Response: [is] [bri] 1.0
[set] [bri] 0.5
Acknowledged
[get] [bri]
Response: [is] [bri] 0.5

The "[set]" command will cause the red crosshair to fade in brightness.

Relationship between devices and ports

There is no mandated relationship between devices and ports in YARP. There are some guidelines:

• Whenever possible, communication should be unidirectional. For framegrabbers, this translates to streaming images out on a buffered port. Such communication can be heavily optimized.

• For occasional communication, or communication that requires a response, then two-way "RPC" (remote procedure call) style communication is acceptable. This is always slower than the first option, and so must be used with care.

Framegrabbers in YARP are currently represented on the network using a single port, which has buffered streaming output for images, and unbuffered RPC-style input for commands and queries that can't be satisfied from the image stream.

---