The DRG system for sketch understanding and recognition

Overview of what's up

Currently we have three implementations of code that does some form of sketch recognition: one done by Christine and Mike, one done by Tracy, and one done by Metin. All implementations overlap in functionality somewhat; we would like to integrate them into one coherant system. This document begins to describe and define such a unified system.

There are two different aspects of integrating our systems:

  1. Recognition Toolkit: A system that you can query with input and it will tell you whether your stroke (or group of strokes, or whatever we deem to be the appropriate input) is a circle, square, ... etc. It will also provide access to some measure of the error or certainty of the classification.
  2. Recognition System: A system that integrates the output of the recognition toolkit and decides which interpretation to display. Basically, this is the system from stroke input to user output, and probably relies on the recognition toolkit for base recognition. For example a frame work in which you could impliment both a UML drawing system and a mechanical device drawing system.

We are focusing on creating a Recognition Toolkit first. In conjuctions with the toolkit is a geometry package. This package has representations of and operations on basic geometric shapes.

Metin has the most robust low-level recognizers, so the low level recognizers are based on his code. Included (as a start) in this toolkit is recognition funtionality for circles, lines, polygons, and complex shapes which are the combination of lines and bezier curves. In the future we may add other basic shapes (arrows, check marks, simple mechanical devices, specific shapes, etc).

The next step is to convert our existing systems to use this toolkit. This will ensure that the toolkit provides sufficient functionality and ease of use.

The step following this is to build a comprehensive sketch system of the sort designed by Luke and Christine, which provides a language in which to describe how semantics can be used to disambiguate ambiguous interpretations.

Getting Started

This setup is really new (i.e. possibly buggy). I have tried really hard to make it general and portable. It will work on Linux with Bash and CSH and under Cygwin (a collection of Linux tools ported to Windows including Bash, make, ssh, cvs, etc... If you want to use the drg sources under Cygwin but haven't used Cygwin with cvs, shh, etc talk to me and I can help you out. In general if you aren't familiar with these tools in Cygwin, Linux will probably be a bit easier to manage.)

If something is unclear or wrong/broken please send me (moltmans@ai.mit.edu) a note. There are a few things you need to know to be able to compile, write, and run code in this system:

  1. get the sources from CVS
  2. set environment variables
  3. run the Makefile
  4. run TicTacToe

Get the sources from CVS

The CVS root is located in our directory as /projects/drg/CVSROOT Linux:
cd ~
cvs -d /projects/drg/CVSROOT checkout drg

Cygwin: (You also have to have CVS_RSH=ssh. Please don't samba mount the repository since there are some subtle problems with this)
cd c:/
cvs -d :ext:fracas.ai.mit.edu:/projects/drg/CVSROOT checkout drg

Set Environment Variables

There are two scripts to help setup your environment. Assuming that you have put the sources into ~/drg on Linux (or c:\ on Windows) all you should need to do is: It is also a good idea to include these lines in your shell's resource file (i.e. .cshrc or .bashrc).

If you did not put the drg directory in your home directory then you will need to change the DRG_DIR variable in the appropriate script mentioned above.

IMPORTANT: don't check your changes to these files into CVS.

Other variables you may want to change in these scripts are:

DRG_DIR
The location of the top level drg directory. On Linux this should look something like: ${HOME}/drg or if you put is somewhere else ${HOME}/code/drg etc... On windows it should be a windows style path but with forward facing slashes (`/'). For example: c:/drg or d:/research/code/drg or whatever.
DRG_JAVAC
The Java compiler you want to use. The defaults are javac on Windows and jikes and Linux. There is however a version of Jikes for Cygwin as well at: http://oss.software.ibm.com/developerworks/projects/jikes/
DRG_JAVA_LIB
If you are using Jikes you will need to specify where the java core classes are. This should be either /jdk/jdk1.3/jre/lib/rt.jar or c:/jdk1.3/jre/lib/rt.jar or wherever you have it installed on your system.
JFLAGS
The flags to pass to the java compiler. For example you may want debug information: "-g" Or you may want to tell jikes to output errors that emacs can parse: "+D".
CLASSPATH
If you find yourself needing to change the CLASSPATH variable you should probably think again. All the java classes should be in our tree (drg/code/classess) if we wrote it or in the lib directory (drg/lib) if we didn't.
Jar files should be unzipped so that the CLASSPATH does not need to be updated. To help enforce this the makefiles use a separate CLASSPATH from that used by the shell. Talk to Mike Oltmans (moltmans@ai.mit.edu) if you really need to change it.
Binary libraries should be included in drg/lib if they are platform independent or in the appropriate platform specific subdirectory if they are platform dependent.

Run the Makefile

You should now be ready to compile and run the sources (remember to restart your shell if you changed any of the parameters in the scripts).

cd drg/code/src/edu
make world

If this produces errors there is probably something set incorrectly in your environment setup or I have screwed something up. Feel free to ask me about it in either case.

Run TicTacToe

If your environment is set up properly and everything compiled as expected you should just have to type:

java edu.mit.sketch.toolkit.TicTacToe

You first draw the board and then play with yourself or a friend. Feel free to write a computer player at your own leisure.

Developing

Makefiles

Within the DRG system there should be no need to ever write a makefile. If you need a new one, it can be automagically generated for you. (Isn't that nice?)

How to use the makefiles

There are several top level targets for each makefile. Each operation is performed in the current directory and all of the subdirectories listed in the Makefile. Here are the targes (e.g. make all):

Creating and Maintaining

To create a makefile in a new directory simply run: new_drg_makefile. This will create the files "Makefile" and "files" which are:

Makefile
We are using a hierarchical makefile structure with one makefile called "Makefile" in each directory. Makefiles contain a variable (SUBDIRS) that specifies which sub-directories should be made when this directory is made. This is the only variable you should have to edit in this file.
files
Contains a list of all files that should be made in the current directory.
In addition there is a Make.default file: For the most part you should not edit this file. Rather you should set environment variables to do what you want. In general the variables you are likely to want to change will be taken from the environment if they exist already.

CVS

Updates

Commits

Emacs pcl-cvs

Coding standards and conventions

Variable and class names

Spacing, tabs, and braces

Debug output, error handling, error reporting

Documentation

Packages and API's

The API

Mike Oltmans (moltmans@ai.mit.edu)

Last modified: Thu Apr 5 14:33:33 EDT 2001