Progress: November 2005 - December 2005
From LadypackWiki
| Table of contents |
Wednesday, December 21
The XSens MTi IMUs showed up a couple weeks ago. I made a few new pieces of mounting plastic and attached them. One goes with the ladybug camera, the other goes with the Disto Memo laser range finder. Also integrated them into the data logger and took some sample data. I think the platform is now hardware complete (think). Here's the current sensor list:
- Ladybug camera
- Microphone
- Disto Memo laser range finder
- XSens MTi IMU (with ladybug)
- XSens MTi IMU (with disto memo)
 |  |  |  |
Applied SIFT to a video stream to see what it might look like - here (http://people.csail.mit.edu/albert/ladypack/media/lp1-20051205-000956.sift-xvid.avi). Only the locations of the feature points are shown, not the orientation or scale.
Thursday, December 8
Downloaded David Lowe's SIFT demo and started playing around with it. Might be able to just use it as it is without bothering to implement my own version of SIFT. It's slow, though, because you have to launch a new process for each image.
Began parallelizing some code to take advantage of the photon cluster. Larry also suggested that if I really needed computing power, to think about talking to Alan Edelman.
Put a couple videos online of data sets. http://people.csail.mit.edu/albert/ladypack/media/
Tuesday, December 6
Yesterday, Seth and I had a meeting with Prof. Bernt Schiele (http://www.vision.ethz.ch/schiele/schiele.html) from ETH Zurich. Very nice, friendly guy. Gave us a bunch of names to look into regarding people who do work in wearable computing, augmented reality, context awareness, online annotation, etc.
- Andrew Davison (http://www.doc.ic.ac.uk/~ajd/) at Imperial College London (formerly of Oxford), mentioned in a post a few weeks ago.
- Steve Feiner (http://www1.cs.columbia.edu/graphics/top.html) at Columbia works on augmented reality.
- Andrew Zisserman (http://www.robots.ox.ac.uk/~vgg/index.html) at Oxford
- Simon Julier (couldn't find a link)
- Dieter Schmalstieg (http://www.icg.tu-graz.ac.at/Members/schmalstieg) at Graz University
Seth and Bernt noted that in annotating video streams, people don't typically use speech as an annotation method. I suppose that's one reason we're doing what we're doing.
Bernt also suggested we check out Intel's PlaceLab to get indoor wireless localization. I don't actually think that's going to be terribly useful, mostly from my experience with Bluetooth. I'm also not a huge fan of such an instrumented environment.
head mounted cameras suck. The head moves too much.
Wednesday, November 30
Visualizing all the data being processed is slow because of all the number crunching that needs to happen. I looked into ways to create videos of the processed data, but didn't find any satisfactory packages that would easily let me take images and audio data and create movie files. There are actually a number of programs and libraries out there that do it for you, but quick glances through the source code indicated to me that it's just a world of pain. Instead, I just wrote some python code to create an AVI file with uncompressed audio and video data. The resulting files are enormous, but there are tons of easy to use programs that will re-encode AVI streams for you. mencoder has worked well for this purpose.
# mencoder -ovc divx4 -oac mp3lame input.avi -o output-divx.avi
AVI links:
- http://www.jmcgowan.com/avi.html
- http://pvdtools.sourceforge.net/aviformat.txt
- http://en.wikipedia.org/wiki/AVI
Thursday, November 17
Andrew Davison (http://www.doc.ic.ac.uk/~ajd/) has done some very interesting work with MonoSLAM (single camera visual SLAM). Will have to read through some of his papers..
He also has some old linux software (http://www.doc.ic.ac.uk/~ajd/Scene/) for SLAM that looks interesting.
Wednesday, November 16
Keep getting distracted by a hundred different things. Start doing one thing, then get distracted and start another, then get distracted again ad infinitum.
TODO
- calibrate cameras - done
- verify calibration is correct
- re-implement spherical projection - is this really necessary? Useful for image boundaries...
- re-implement image rectification in linux - is this really necessary? no....
- re-implement panorama generation (image stitching) in linux - is this really necessary? no...
- create code for exporting logged data sessions to a movie file format (e.g. AVI or MPEG)
- create code for exporting logged data sessions to Matlab-readable formats. Or at least the images...
Looked at Udo Frese's phd thesis (http://www.informatik.uni-bremen.de/~ufrese/published/fresediss.pdf) because I thought the way he used circular visual markers in his SLAM work was pretty cool. Unfortunately, it turns out that I won't be able do the same thing. One assumption Udo made in his work is that the camera on his robot always be at a fixed height and angle, so that the camera could be calibrated against the floor plane. Since all fiducial markers were on the ground, it would then be easy to predict the appearance of a fiducial marker if it were to appear at a given position. Since the ladybug camera is going to be bobbing up and down and all around, I can't make that assumption.
Tuesday, November 15
Started using the Matlab camera calibration toolbox (http://www.vision.caltech.edu/bouguetj/calib_doc/) to get the intrinsic parameters for the ladybug2 cameras. It's quite tedious, and I've been making mistakes along the way (clicking on the wrong points, not using enough images from different viewpoints), but I think I have the hang of it now. Having to calibrate 6 cameras is annoying.
Monday, November 7
Am now trying to figure out the best way to stitch images from the different cameras together. One option would be to have a postprocessing step in windows that uses ladybug.dll to generate the stitched and spherical images and then saves those for later processing in linux. Another option would be to re-implement them in Linux. I've emailed Point Grey Research asking for source code, but don't really think they'll give it to me.
Added stubs in the analysis code to retrieve disto data.
Ordered two MTi development kits from XSens.
update Paul Cartwright from PGR has written back to say they will not provide source code to the Ladybug2 SDK. This means I'm out of luck unless I take some time to re-implement the panorama stitching and image rectification code in Linux.
Wednesday, November 2
Added data logging support for the Disto Memo laser range finder.
I thought it would be nice to be able to control exactly when the data logging starts and stops. Before, the way it worked was I had to type in the capture command on a text console and then race to put on the ladypack and all the gear before it started recording. One option is to crop the data stream, but then I'd have to crop every one of them by hand. So I added a Bluetooth remote control option and wrote a little client program for my cell phone. Now, I run the capture program, then put the ladypack on. Once I'm ready to start recording, I connect with my cell phone and hit the record button. Much cleaner this way.
Also put together a working battery. Used the 21-cell NiMH that's ORCA surplus. Soldered a cable together to mate the connectors and it works beautifully. Fully charged, the battery is approximately 29.5 volts, which is just within the ladybug's acceptable input range. The only problem now is that I need a charger for the battery - it'll be a major pain in the ass to keep running back to the ORCA shop every time I want to charge the battery. ORCA suggested that if I keep the battery, to buy it somehow from the team.
Tuesday, November 1
Looking into Inertial Measurement Units to use with the ladypack. Ideal is a device that provides 6-DOF angular and positional information. Currently considering some devices from Crossbow, MEMSense, and Xsens.
| Manufacturer | product | pricing | notes |
|---|---|---|---|
| Crossbow | IMU300 (http://www.xbow.com/Products/productsdetails.aspx?sid=54) | $1,995 10% academic discount if using credit card and not purchase order | angular acceleration, X/Y/Z acceleration. Would need to build custom cable. |
| Crossbow | IMU400 (http://www.xbow.com/Products/productsdetails.aspx?sid=27) | $2,995 10% academic discount if using credit card and not purchase order | angular acceleration, X/Y/Z acceleration. Would need to build custom cable. |
| Xsens | MTi development kit (http://www.xsens.com/index.php?mainmenu=products&submenu=machine_motion&subsubmenu=MTi_DK) | EUR 1,990 (~USD 2,388.09) 10% academic discount | angular acceleration, X/Y/Z acceleration, magnetometer. comes with USB adapter |
| MEMSense | AccelRate3D (http://www.memsense.com/content/products/Datasheets/AccelRate3D_SMT_Datasheet_%20revD1.pdf) | $850, 10% academic discount | angular acceleration, X/Y/Z acceleration. analog output, would need to build a custom conversion board to interface with laptop. |
| MEMSense | MAG3 (http://www.memsense.com/content/products/Datasheets/MAG3_SMT_Datasheet_%20revD1.pdf) | $930, 10% academic discount | angular acceleration, X/Y/Z acceleration, magnetometer. analog output, would need to build a custom conversion board to interface with laptop. |
update Intersense says they'll provide info on how to access raw acceleration data if we sign an NDA. I guess this means X,Y,Z acceleration, but he (Brian Calus) didn't say specifically what. Seems silly.
update MEMSense got back to me on pricing and connection details.
update Xsens got back to me on pricing and the difference between MTi and MTx. MTi is a little bigger but easier to mount. Otherwise more or less the same.
update Crossbow got back to me on academic discounts.
Using the DISTO memo 1 start bit, 7 data bits. 1 stop bit, even parity, 9600 baud.
