A Configuration Package

Last Modified: 05/30/2001

Introduction

This package helps programmers write software that is more easily configurable. From the end user's perspective, command line arguments and dialog boxes may be suitable for some tasks, but neither option is satisfactory for most scientific programming. Command line arguments are great for quickly changing a few parameters for each run of a calculation, but they break down when there are a lot of options involved. Dialog boxes look great, but unless they provide an easy way to switch between vastly different configurations and allow the user to express the configuration concisely, they get in the way of work. My personal preference as a user and programmer is file-based configurations. This is the only way I've been able to get the versatility and expressiveness of a command line with the intuitive structure of dialog boxes.

This package provides the user with an easy to use formalism for program configuration and provides the programmer a very simple set of routines for retrieving the user's options. The user fills out a file, the program reads its configuration from that file. One of these days, I may build a GUI for editing these files, but so far, editing them in a text editor has been convenient enough.

For Example

Suppose you are working on a program to find faces in images. There are a lot of options to be set: the name of the input file, the name of a directory containing training examples, various thresholds, iteration counts and tolerances, window sizes, kernel sizes, etc. Suppose you're polishing up and trying to figure out which set of parameters works best. You need to try out different configurations, save old sets of configurations in case you find yourself in a dead end, reuses piece of old configurations, and generally not waste time. Here's a snippet of configuration from my face finder.

section Library {
    section Testing {
    jebfile = skin.jeb
    cachefile = skin.cache
    skindump = skin.pgm
    focal_length = 50.0
    starting_depth = 3.0

    dotest = f # don't run a test unless
    # explicitly overridden in
    # the test's section.

    gmixture = f
    config = f

    section Linalg {
        dotest = f
        ls_solver_1 = f
        ls_solver_2 = f
        semi_ortho_matrix_fname = semi_ortho.pgm
        n_non_ortho = 2
    }

    section Construction {
         dotest = t;
         coeffs = {-170.802 -480.473 -68.5568 336.854 -52.1304 -180.587 420.313 -36.9678 158.834 420.233 -10.6145 149.071 3.03145 -5.2396
6 -77.2672 -195.354 -30.7757 -112.373 -106.244 216.804 -296.789 -91.2857 86.0481 189.719 -76.1543 -7.98662 270.463 24.9486 -262.505}
         outfname = constructed.pgm
   }

...

configuration Brixton {
     Library:Testing:jebfile = brixton.jeb
     Library:Testing:cachefile = brixton.cache
     Library:Testing:colorfaceinput = face50.ppm
     Library:Testing:Eigenfaces:Prefilter:skin_content_thresh = 0.000001
}

Table 1: Part of a configuration file.

The File Format

The configuration file consists of sections and configurations. A section contains a set of key-value pairs, each key corresponding to some option. Sections can be nested for the sake of modularity and have inheritance semantics which allows the user to write complex configurations without being verbose. In Table 1, the "Library" section contains sections "Linalg" and "Construction", for example. "Linalg" inherits all the options of "Library" and overrides the ones it defines explicitly. Configurations are subroutines that can be called from within a section or from the program. Configurations are used to assign different values to a section. For example, when called, the "Brixton" configuration modifies the variable "Library:Testing:jebfile" from "skin.jeb" to "brixton.jeb".

Sections: Aassignments

The user defines a section using the section keyword, followed by the name of the section. The body of the section is enclosed in braces. The body can contain key-value pairs, sub sections, and directives such as inherit and call. In this example, keys in section "Foo" are assigned values:

section Foo {
	Tolerance = 1e-4
	InputFile = foo.data
	UseColor = t
	DownsamplingPyramid = {1 2 4 32}
	TransformationMatrix = {1 0 0 1 3 4}
}

Table 2: Assigning values to keys.

The supported built-in types are integers, floats, strings, bools, and arrays of these. The last section explains how a program can recover these value in the native format from the package. Strings with spaces can be specified by using double quotes. C escape sequences and multiline strings are also supported.

Sections: Nesting and Inheritance

In the following example, "Foo-derived" is a subsection of "Foo".

section Foo {
	opt1 = oldvalue
	opt3 = given
	section Foo-Derived {
		opt1 = 0.1
		opt2 = {4 2}
	}
}

Table 3: Nesting and inheritance.

The fully qualified name of "Foo-Derived" is "Foo::Foo-Derived". Nesting allows configuration options to be organized hierarchically. But they provide much more significant functionality: in the above example, "Foo:Foo-Derived:opt1" takes on the value "newvalue" and "Foo:opt1" is "oldvalue", as expected, but you can also query "Foo:Foo-Derived:opt3" and get "given". Hence, nested sections inherit options from their enclosing section. In addition, you can add another parent to your class using the inherit keyword to inherit from multiple sections:

	section UsefulValues {
		MachineType = Linsux-i386
		Verbose = f
	}
	section Foo {
		opt1 = oldvalue
		opt3 = given
		section Foo-Derived {
			inherit UsefValues
			opt1 = 0.1
			opt2 = {4 2}
		}
	}

Table 4: The inherit keyword.

The inherit keyword makes the keys defined in "UsefulValues" available to "Foo::Foo-Derived". This happens because "UsefulValues" becomes a parent section of "Foo-Derived" in the same way "Foo" is a parent class of "Foo-Derived".

Configurations

A configuration sets the value of individual assignments inside a section. Because it introduces non-local changes, I discourage its use, but I've found it useful on occasion. Here's an example of a configuratin:

configuration ChangeFoo {
	Foo:Foo-Derive:opt3 = 53
	Foo:opt1 = blah
}

Table 5: A configuration

This configuration can be called from another configuration, a section, or from the program. In the first two cases, the syntax is simply: "call ConfigurationName". All the sections being modified must be defined earlier in the configuration file (this may be a restriction that no longer in newer versions).

From the Programmer's End

Reading configuration files is easy. All class are defined in the NTPM namespace, so either use a uses clause in your program or precede all class names in this documentation with NTPM. First create an object of type "Configuration". Pass the constructor of configurator the name of the config file. Alternatively, you can use the open() method and use the default constructor. After that, four methods are useful: lookup<T>(), and list_lookup<T>(), and CallSubroutine().

Lookup() takes a path to a key and returns its value as a string (note that this is a C++ string, not a char*. Specifically, this is std::string which comes standard with C++). All the inheritance and configuration call processing takes place. There is an overload of lookup() which takes a path to a section as one argument and a key name as a second argument.

Lookup<T>() where T is one of bool, int, float, or string is identical to plain lookup(), except it converts the resulting string to the desired type and returns this value instead.

List_lookup<T>() returns a vector<T>. This is the function to use for reading a list from the configuration file. If you want to read "Foo:Foo-derived:opt1" in Table 3, you would use list_lookup<int>("Foo:Foo-derived:opt1") to retrived a vector of int's. Note that vector<> is the C++ std::vector.

CallSubroutine() takes the name of a configuration and executes its body.

Error handling is done through execptions. If there is a syntax error, an exception of type std::runtime_error is thrown. The what() method of this object explains the error as usual.

Downloading

This package is available as a tar file. You can also view the files individually:

config.h

config.cc This file contains a test stub. Look for main().

config.tcc

configuration.y and configuration.l The lex and yacc files.

ntpm.conf A sample config file.

Building

Type gmake. Run ./conf.