Final Project for Spring 2005-2006
Visual Code Marker Detection

The project should be done individually or in groups of up to 3 people and should require about 50 hours per person. Each group will develop and implement their own algorithm to detect an unknown number of visual code markers in an image captured by a cell-phone camera.

INTRODUCTION
DOWNLOADS
INSTRUCTIONS
DEADLINES
PROJECT SUBMISSION
PROJECT GROUPS

INTRODUCTION

The project should be done individually or in groups of up to 3 people and should require about 50 hours per person. Each group will develop and implement their own algorithm. The task of the project is to detect and read visual code markers through the cell-phone camera. In the paper titled "Real-World Interaction with Camera-Phones", the author, Michael Rohs, describes the use of visual code markers in interacting with real-world objects. More information can be obtained by following this link. To give you an idea of the utility of visual code markers, the user could, for example, download the data embedded in the marker to his cell-phone, or follow a hyperlink embedded in the marker, or perform item-selection by rotating the camera-phone with respect to the marker, etc.

You will design and test your detection algorithm with a set of training images and the corresponding ground truth data that can be downloaded from the Downloads section. Once you submit your algorithm implementation, we will check its performance with a set of test images, which will be unknown to you while you work on your algorithm.

Visual Code Marker Details:

The visual code markers that we consider are 2-dimensional arrays. The array consists of 11x11 elements. Each element is either black or white. As shown in the figure below, we fix the elements in three of the corners to be black. One vertical guide bar (7 elements long) and one horizontal guide bar (5 elements long) are also included. The immediate neighbors of the corner elements and the guide bar elements are fixed to be white. This leaves us with 83 data elements which can be either black or white.

DOWNLOADS

----------------------------------

Training Images and Ground Truth Data

Training_1.jpg    Training_2.jpg    Training_3.jpg    Training_4.jpg   

Training_5.jpg    Training_6.jpg    Training_7.jpg    Training_8.jpg

Training_9.jpg    Training_10.jpg  Training_11.jpg  Training_12.jpg 

- The training images are regular JPEG images.

- The images can be read by imread( ) in MATLAB.

- The ground truth data are stored in the MATLAB data file called training_ground_truth.mat and can be read using load in MATLAB. The ground truth data consist of code marker data and position of upper left corner of the marker(s) in the given training images.

----------------------------------

Evaluation Program

This is the MATLAB function which will be used to evaluate your code marker reading algorithm. You should use it to evaluate the performance of your algorithm on the training images.

evaluate.m

----------------------------------

Code Generation Program

This MATLAB function generates the code marker for a given input vector of 83 elements.

generate_code.m

INSTRUCTIONS

(1) Note that your detection algorithm has to output

• the number of markers detected in the image
• the coordinates for the upper left corner element of every marker detected in the image (specifically the coordinates of the center of the upper left element)
• the 83 data bits (1=black, 0=white) for every marker detected in the image

Specifically, the interface between your code detection function and the function evaluate( ) is specified in MATLAB notation as follows:

function [data_detected,origin_detected] = detect_code(img)

The input and output are defined as follows
img: height-by-width-by-3 uint8 array representing a 24-bit RGB image obtained through imread( ).

data_detected: N-by-83 double array of 0's and 1's where N is the number of code markers detected. Each row of data_detected corresponds to the 83-bit data embedded in the code marker.
origin_detected: N-by-2 double array where N is the number of code markers detected. Each row of origin_detected corresponds to the [row,column] image-coordinate of the origin of a detected code marker. The origin of a code marker is defined as the top-left element of the marker, i.e., code(1,1) defined in generate_code.m.

(2) The performance metric (for a single input image):

Score = (number of correctly matched elements) - 41.5 x (number of repeats + number of false positives)

A detected code marker is matched for correctness with a ground truth code marker only if the Euclidean distance between the detected upper left corner and the ground-truth upper left corner is below a certain threshold. If multiple detected code markers are within this threshold, then the detected code marker with the least distance is selected. Note that any detected code marker will be matched with at most one ground-truth code marker and vice-versa.

False positive means a detected code marker which is outside the Euclidean distance threshold of any ground-truth code marker.

Repeat means a detected code marker which is within the Euclidean distance threshold of a ground-truth code marker in the image but there exists another detected code marker which is closer to the ground-truth code marker. The detected bits are discarded in this case.

False negative means a code marker that exists in the image but is not detected by the algorithm. Note that a false negative automatically reduces the score by reducing the number of correctly matched elements.

(3) Positive scores will be added for different test images. Negative scores will be counted as zero.

(4) The maximum run time for a single image is 1 minute. We will abort your program if it takes longer.

(5) While still in the training and optimization phase, you can create your own code markers using generate_code.m, paste them on real-world objects, capture images using your humble cell-phone camera, and use them to check your performance.

(6) To register your group, please send an e-mail to ee368-spr0506-staff@lists.stanford.edu with the subject "group registration for project".

DEADLINES

Deadline to register groups: May 12.
Deadline for submission of project programs and report: midnight of June 2 (you have the whole day of June 2 to work on it), no extensions.

PROJECT SUBMISSION

(1) Please mail to ee368-spr0506-staff@lists.stanford.edu (do not c.c. to TA's or Professor's personal e-mail address).

(2) For program, please include ONLY nessessary files in ONE ZIP file named "ee368groupXX.zip" and email it with subject "ee368 groupXX program". Your 2 digit group number XX is listed below.

(3) For report, please prepare in PDF format, name it "ee368groupXX.pdf" and email it with subject "ee368 groupXX report". Your 2 digit group number XX is listed below.

(4) Note that your code will be run on the SCIEN machines by the teaching staff. Hence, it is a good idea to get an account there and make sure that there are no platform-specific problems with your code.

(5) If you are a group, then you need to submit the log of who did what with your report.