back

research log

My attempt to log my general progress in building a visual programming environment, called Codeink. Try it here.

# DECEMBER 12, 2013
What and why

I've been imagining an environment where an algorithm can be described visually by sketching and directly manipulating data. Why? Because writing code doesn't always seem like the best way to (a) think through a programming problem, or (b) explain an algorithm to others.

Programming vs. Coding

Writing code consists of two steps: imagining interactions with objects and translating those interactions into a programming language. This translation can make 'programming' seem like a skill that is out of reach for anyone who doesn't speak the language, where syntactic details must be correct or your program doesn't work.

The translation step can be particularly frustrating when you're after some concrete thing. For example, why draw a giraffe using Processing or Javascript? Why fiddle with variables and arbitrarily-named functions to get something painted on the screen, when you could just draw in Photoshop? Well, a big reason is because those variables make the drawing dynamic: you can easily change a giraffe's height or color. Bret Victor's system combines the directness of Photoshop and the dynamism of programming languages: as you take concrete actions towards creating a drawing, the system translates them into parameterizable steps.

Many algorithms are inherently visual

So it would seem that programming should be easier - or at least, the bottleneck should be in demonstrating the drawing steps, not translating them to lines of code. Like creating a drawing, it turns out that many of the algorithms taught in high-school and undergraduate classes are inherently visual: sorting lists, traversing graphs, rotating trees. These data structures are concrete in look-and-feel, and their related algorithms are not taught by writing code on the board. Here are a few examples of how Insertion Sort has been demonstrated:

1. A storyboard showing changes to the list - 6.006 Intro to Algorithms, MIT

2. Cups being rearranged - CS50 Intro to CS, Harvard

3. Romanian folk dance - Algo-rythmics, Sapentia University

The common thread here is that the algorithm is explained by tracing it on a concrete example. In the storyboard (the most common method of illustrating algorithms), the trace is visualized by redrawing the example several times to show changes. Furthermore, after these demonstrations, students dive into problem sets where they have to implement the algorithm in a programming language. They don't have the opportunity to test their conceptual understanding of the algorithm by tracing it on additional concrete examples.

The plan

Following from these observations/gripes/BretVictorianThoughts, I'm building Codeink with this objective: make it easier to communicate and think about algorithms for inherently visual problems. I'm building it in two stages.

Stage 1: Animating Concrete Traces. Make it possible to describe an algorithm's trace on a concrete example by drawing and directly manipulating data. The envisioned usage model is the following:

  1. Setup an example by drawing or dragging objects onto the stage.
  2. Trace the algorithm by directly manipulating the objects.
  3. Codeink translates the trace into parameterizable steps.
  4. Playback the steps as an animation of the algorithm.

The hope is that this environment will make it easier for:

Stage 2: Supporting Abstraction. Make it possible to describe flow control via drawing and direct manipulation, in order to support descriptions of the general algorithm, not just its trace on a concrete example. This is about moving up the ladder of abstraction.

This is the more difficult stage, and while I've thought about how iteration, branching, recursion and functions might be described via direct manipulation, its prerequisite is the completion of Stage 1. So that's what I'm focusing on right now. Still, some ideas that I've discussed with others for Stage 2 are on these slides.

Jeremy Scott (jks@mit.edu)