The DRG system for sketch understanding and recognition

Overview of what's up

Currently we have three implementations of code that does some form of sketch recognition: one done by Christine and Mike, one done by Tracy, and one done by Metin. All implementations overlap in functionality somewhat; we would like to integrate them into one coherent system. This document begins to describe and define such a unified system.

There are two different aspects of integrating our systems:

  1. Recognition Toolkit: A system that you can query with input and it will tell you whether your stroke (or group of strokes, or whatever we deem to be the appropriate input) is a circle, square, ... etc. It will also provide access to some measure of the error or certainty of the classification.
  2. Recognition System: A system that integrates the output of the recognition toolkit and decides which interpretation to display. Basically, this is the system from stroke input to user output, and probably relies on the recognition toolkit for base recognition. For example a frame work in which you could implement both a UML drawing system and a mechanical device drawing system.

We are focusing on creating a Recognition Toolkit first. In conjunction with the toolkit is a geometry package. This package has representations of and operations on basic geometric shapes.

Metin has the most robust low-level recognizers, so the low level recognizers are based on his code. Included (as a start) in this toolkit is recognition functionality for circles, lines, polygons, and complex shapes which are the combination of lines and bezier curves. In the future we may add other basic shapes (arrows, check marks, simple mechanical devices, specific shapes, etc).

The next step is to convert our existing systems to use this toolkit. This will ensure that the toolkit provides sufficient functionality and ease of use.

The step following this is to build a comprehensive sketch system of the sort designed by Luke and Christine, which provides a language in which to describe how semantics can be used to disambiguate ambiguous interpretations.

Getting Started

This setup is really new (i.e. possibly buggy). I have tried really hard to make it general and portable. It will work on Linux with Bash and CSH and under Cygwin (a collection of Linux tools ported to Windows including Bash, make, ssh, cvs, etc... If you want to use the drg sources under Cygwin but haven't used Cygwin with cvs, shh, etc talk to me and I can help you out. In general if you aren't familiar with these tools in Cygwin, Linux will probably be a bit easier to manage.)

If something is unclear or wrong/broken please send me (moltmans@ai.mit.edu) a note. There are a few things you need to know to be able to compile, write, and run code in this system:

  1. get the sources from CVS
  2. set environment variables
  3. run the Makefile
  4. run TicTacToe
There is some more general information about setting up Win 2K, Java, emacs, cvs, etc... in the drg-howto

Get the sources from CVS

The CVS root is located in our directory as /projects/drg/CVSROOT Linux:
cd ~
cvs -d /projects/drg/CVSROOT checkout drg

Cygwin: You also have to have CVS_RSH=ssh. Please don't samba mount the repository (this is when you map a network drive in windows) since there are some subtle problems with this that will make your life unhappy.
cd c:/
cvs -d :ext:fracas.ai.mit.edu:/projects/drg/CVSROOT checkout drg

Set Environment Variables

There are two scripts to help setup your environment. Assuming that you have put the sources into ~/drg on Linux (or c:\ on Windows) all you should need to do is: It is also a good idea to include these lines in your shell's resource file (i.e. .cshrc or .bashrc).

IMPORTANT: don't modify these files to personalize your environment variables.

For example, if you did not put the drg directory in your home directory then you will need to change the DRG_DIR variable by overriding the default location. Create a file in your home directory (type echo $HOME to find out what it is set to) called .drg_setup.sh or .drg_setup.csh (according to your shell). The commands in these files will override the defaults in setup_env. If you need to change the way the path or classpath are set in the default script you must set both the path and classpath in your .drg_setup file and also set the variable: DRG_DONT_CHANGE_PATH="TRUE".

Other variables you may want to change in these scripts are:

DRG_DIR
The location of the top level drg directory. On Linux this should look something like: ${HOME}/drg or if you put is somewhere else ${HOME}/code/drg etc... On windows it should be a windows style path but with forward facing slashes (`/'). For example: c:/drg or d:/research/code/drg or whatever.
DRG_JAVAC
The Java compiler you want to use. The defaults are javac on Windows and jikes for Linux. There is however a version of Jikes for Cygwin as well at: http://oss.software.ibm.com/developerworks/projects/jikes/
DRG_JAVA_LIB
If you are using Jikes you will need to specify where the java core classes are. This should be either /jdk/jdk1.3/jre/lib/rt.jar or c:/jdk1.3/jre/lib/rt.jar or wherever you have it installed on your system.
JFLAGS
The flags to pass to the java compiler. For example you may want debug information: "-g" Or you may want to tell jikes to output errors that emacs can parse: "+D".
PATH
You should include at least the default paths in your PATH. You also need to make sure that you are using Java 1.3. To check this type java -version if it is less than 1.3 you should make sure that the bin directory of a Java 1.3 installation is the first java directory listed in your path (e.g. /jdk/jdk1.3/bin).
See the note for DRG_DONT_CHANGE_PATH if you need to change the path in your local setup.
CLASSPATH
If you find yourself needing to change the CLASSPATH variable you should probably think again. All the java classes should be in our tree (drg/code/classess) if we wrote it or in the lib directory (drg/lib) if we didn't.
Jar files should be unzipped so that the CLASSPATH does not need to be updated. To help enforce this the makefiles use a separate CLASSPATH from that used by the shell. Talk to Mike Oltmans (moltmans@ai.mit.edu) if you really need to change it.
Binary libraries should be included in drg/lib if they are platform independent or in the appropriate platform specific subdirectory if they are platform dependent.
See the note for DRG_DONT_CHANGE_PATH if you need to change the path in your local setup.
DRG_DONT_CHANGE_PATH
Setting this variable tells the default script to not change the path or classpath. If you need to set the PATH or CLASSPATH differently from what is done in the default setup you must do 3 things: Set the path and classpath and set the variable DRG_DONT_CHANGE_PATH="TRUE". Otherwise you should set it to nothing.

Run the Makefile

You should now be ready to compile and run the sources (remember to restart your shell if you changed any of the parameters in the scripts).

cd drg/code/src/edu
make world

If this produces errors there is probably something set incorrectly in your environment setup (maybe because I have screwed something up). Feel free to ask me about anything you have trouble with and please tell me if there is something wrong even if you fix it. If you don't someone else will have to suffer as well.

Run TicTacToe

If your environment is set up properly and everything compiled as expected you should just have to type:

java edu.mit.sketch.toolkit.TicTacToe

If this doesn't work one common problem is having an old version of java in your path. See the note for PATH on environment variables for more details.

You first draw the board and then play with yourself or a friend. Feel free to write a computer player at your own leisure.

Developing

Makefiles

Within the DRG system there should be no need to ever write a makefile. If you need a new one, it can be automagically generated for you. (Isn't that nice?)

How to use the makefiles

There are several top level targets for each makefile. Each operation is performed in the current directory and all of the subdirectories listed in the Makefile. Here are the targets (e.g. make all):

Creating and Maintaining

To create a makefile in a new directory simply run: new_drg_makefile. This will create the files "Makefile" and "files" which are:

Makefile
We are using a hierarchical makefile structure with one makefile called "Makefile" in each directory. Makefiles contain a variable (SUBDIRS) that specifies which sub-directories should be made when this directory is made. This is the only variable you should have to edit in this file.
files
Contains a list of all files that should be made in the current directory.
In addition there is a Make.default file: For the most part you should not edit this file. Rather you should set environment variables to do what you want. In general the variables you are likely to want to change will be taken from the environment if they exist already.

Coding standards and conventions

Variable and class names

Spacing, tabs, and braces

Debug output, error handling, error reporting

Documentation

Packages and API's

The local copy (if you have generated them): Local Javadocs

The web copy: Web Javadocs

Mike Oltmans (moltmans@ai.mit.edu)

$Id: index.html,v 1.13 2001/05/09 21:10:30 moltmans Exp $