Organized Research
05 January 2007
Over the last few years, I've developed a way of organizing my research that works quite well for me and may work well for others. I've decided to document it in hopes that it will help other students organize their research. Before I get into the details, here are the fundamental ideas behind my system.
- After some amount of time, you will forget how to run your own code. Is this function the main routine, or is it a subroutine? What parameters to the function give me the result?
- After some amount of time, you will forget what your code does. I see a function named goodfunc1() and goodfunc2(), what’s the difference?
- After spending hours searching for how to do something and then finding the answer, you will think that you will never forget how to do that thing. But you will forget, and sooner than you expect. Complex regular expressions and bash scripting come to mind…
Of course, the obvious answer to all these issues is to write it down. But don’t just write it down anywhere. If you put these things in different files, then you are adding another problem to the list - determing which file has the answer. I put everything in a research journal: a single application for all my research notes, results, important plots, and tons of random tips (like regular expressions and bash scripting…).
In particular, I like MacJournal, which was free when I started using it but is now commercial, though inexpensive. There are plenty of other options, like DevonThink and NoteBook, but I prefer MacJournal for its simplicity.
Here is a screenshot of the journal for one of my projects:
In the screenshot, you’ll notice several things: (1) A typeset equation, dragged-and-dropped into the journal using LaTeXiT, (2) a URL referencing the web page where I saw the equation, (3) some MATLAB code, and (4) the plot generated by that code, captured as a screenshot (command-shift-4), then dragged into the journal.
On the left are the journals, or projects, and on the right are the entries for the selected project. I often name the entries “Experiment N” and let the text and bold headings in the entry describe what the experiment is about. For the most part, I keep my project and entry names consistent with the folders in the Finder that contain the associated code:
In this way, I know exactly where to find the code that produced a plot or given set of results. I also know how to run the code because I’ve saved the commands in the journal, and I’ve saved important equations and plots to remind myself of what I was thinking about during that experiment. I’ve also added version control to this system, as discussed in the previous blog, to keep track of the exact state of everything for a given result - I simply note the svn revision number below plots and results in the journal. Overall, I’ve found this system to be helpful and I hope others will as well.