Patrick Winston and the Copy Demo

Patrick Winston and the MIT AI Lab Copy Demo (1970)

Patrick H. Winston at ASR 33 teletype

In the second half of 1970, after completing his Ph.D. thesis (and getting Marvin Minsky to actually read it), Patrick pulled together a group of researchers to put together a “closed loop” robotics system. That is, one that encompasses sensing, planning, and actuation that affects the objects being sensed. This was the “Copy Demo”:

Four frames from a copy demo sequence

The group consisted of Patrick Winston, Eugene Freuder, Tim Finin, and Berthold Horn, and used some code and/or help from Thomas Binford, David Waltz, Jerome Lerman, Robert Woodham, Adolfo Guzman-Arenas, Bill Gosper, Steward Nelson, Richard Greenblatt (and possibly some others).

Patrick Winston, Berthold Horn, Eugene Freuder Eugene Freuder Berthold Horn Tim Finin

The first task was to image and analyze images of structures built out of simple polyhedral blocks. Tim Finin recalls:

“Patrick took great care in creating the plaster blocks and used (IIRC) a sanding machine in the inner lab on the 9th floor to carefully sand the blocks to ensure they were square, had sharp edges and were without any surface blemishes.”
Line drawings of the blocks then had to be segmented into separate objects. The physical relationships between these objects had to be determined and used to develop a disassembly plan for the structure (which would ensure stability of intermediate arrangements). Reversing the disassembly plan led to an assembly plan. The robot arm was then commanded to build this structure (mirror image reversed, just for fun) from parts earlier placed into a “warehouse area” that the arm could reach.

Image of stacked blocks Line drawing from Binford-Horn line drawing program

The camera was an electro-optical random access “Image Dissector” from III (Information International Inc.) which had significant radial and tangential distortions that had to be corrected based on calibration obtained using a triangular grid target.

III image dissector III image dissector Calibration target for camera distortion measurement List of relevant references
The transformation between “eye” coordinates and “arm” coordinates had to be calibrated by moving the AMF Versatron hydraulic arm in the work space while the camera tracked a “surveyor's mark” held by the hand.

AMF Versatron arm carrying surveyors mark Hand on Versatron arm Versatron arm carrying surveyors mark Versatron arm carrying surveyors mark Versatron arm carrying surveyors mark
AMF Versatron hydrualic arm Hand on Verstran arm
This was a monocular vision system that relied on knowledge of the working surface relative to the camera and the arm. Touch sensors in the finger tips were used to determine the plane of the work surface in arm coordinates ahead of time.

Fingers touching work surface Hand-eye system for copy demo Fingers touching work surface Cube stacking practice Cube stacking practice Hand-eye system for copy demo

Succesful completion of the task was filmed while Dr. Donald Michie happened to be visiting from Edinburgh University. Fragments of that footage made it into the final cut of “Eye of the Robot” (script by Patrick Winston, narrated by David Waltz), some of which is devoted to Marvin Minsky explaining Patrick's thesis (he had finally read it). An actual sequence of the robot building a copy of a structure may be found about 10 minutes into the 15 minute film.

Animation of copy demo sequence

Sadly, the copy demo project was not documented well because the people working on it dispersed (post Ph.D.) and Patrick was soon roped into managing the Artificial Intelligence Laboratory, with little time for writing papers about past projects (but see references to internal reports below). As a result it is perhaps not as widely known as say Stanford's Shakey project, which also demonstrated visual sensing, planning and actuation. When asked about this in 2018, Patrick responded:

“I think at the time there were no obvious places to publish and Marvin wasn't one to push publication. I can't remember if we even did an AI Memo. Possibly too stupid to realize what a milestone it was. I do remember we sent Marvin a telegram, probably showing off.”

Patrick H. Winston

Some References

  1. Freuder, E.C. (1970, 1971) “The Object Partition Problem,” M.I.T. A.I. Laboratory, Vision Flash #4.
  2. Freuder, E.C. (1971) “Views on Vision,” M.I.T. A.I. Laboratory, Vision Flash #5.
  3. Winston, P.H. (1971) “Heterarchy in the M.I.T. Robot,” M.I.T. A.I. Laboratory, Vision Flash #8.
  4. Winston, P.H. (1971) “What's what,” M.I.T. A.I. Laboratory, Vision Flash #9
    “An outline of the modules used in the copy demonstration, the reasons for doing robotics, and some possible directions for further work.”
  5. Horn, B.K.P. (1971, 1973) “The Binford-Horn Linefinder,” M.I.T. A.I. Laboratory, Vision Flash #16.
    Reprinted in 1973 with additional figures as M.I.T. A.I. Laboratory Memo 285.
  6. Winston, P.H. (1972) “Summary of Selected Vision Topics,” M.I.T. A.I. Laboratory, Working Paper 30.
    “This is an introduction to some of the MIT Al vision work of the last few years.
    The topics discussed are 1) Waltz's work on line drawing semantics, 2) heterarchy, 3) the ancient learning business and 4) copying scenes.”
  7. Horn, B.K.P. (1972) “VISMEM - a bag of 'robotics' formulae,” M.I.T. A.I. Laboratory, Working Paper 34.
  8. Winston, P.H. (1972) “The M.I.T. Robot,” in: Michie, D. (Ed.), Machine Intelligence 7, Edinburgh University Press, Edinburgh, Scotland.
  9. Winston, P.H. (Ed.) (1975) The Psychology of Computer Vision, McGraw-Hill (Front cover and section 1.2, page 3).
  10. Winston, P.H. (1977) Artificial lntelligence, Addison-Wesley, p. 209, and pp. 212-213.
  11. Horn, B.K.P. (1986), Robot Vision, M.I.T. Press, section 15.7 “The Copy Demonstration.”