Vision-Based Autonomous Security Robot


Note on the Updates:
Presentation and Demo - May 11, 2010

The final presentation was given during the Bradley University Senior Project Conference at 11:00AM, Tuesday May 4.  A PDF version of the slides can be found here.  A PPS version with working animations is also here.  The PDF version does not have animations so several of the slides may not make much sense.  Overall the presentation and questions went very well. 
Yesterday was the faculty demo for VBASR.  The results were presented to four faculty members.
Lastly, the presentation for FCRAR is scheduled for May 20 or May 21 in Jacksonville Florida!   After the notebook is updated along with final updates to the website this project is finished!  Just over 300 hours spent and over 1700 lines of code!  For more details on various parts of the project (example code, files) please contact Joel Schipper.

Finishing Up - May 3, 2010

On April 23 the final version of the FCRAR conference paper was submitted.  Last Friday April 30 all of the seniors presented their various projects to the Industrial Advisory Board.  Each group needed to have a poster for this presentation. 
Lastly, my final presentation for the Bradley University Senior Project Conference is tomorrow morning at 11:00AM.  After the presentation the slides will be uploaded.  The abstract was adjusted for the BU conference.

96.5% Successful! - April 13, 2010

With the addition of the colors algorithm, VBASR has improved dramatically.  The best total system works with 96.6% of the test images!  In addition, the system was tested by attaching the camera to the iRobot and manually driving the iRobot in the direction the algorithm selected.  In this manner, the iRobot identified and navigated down the center of the hallway!  In every sense VBASR is officially a success.
In addition, the first draft of the FCRAR conference paper was submitted last Wednesday April 7.  Upon final updates and revisions, the paper will be published onto this website.
 The next step is to implement the C++ written image processing code into MRDS.  However, after some research, this seems unlikely to be completed in the remaining time left for work.  Updating this website, completing the FCRAR paper and finalizing the presentation are the immediate tasks at hand.

Under the Hood updates - March 27, 2010

Many updates have been made 'behind the scenes.' After reviewing the different successful and unsuccessful parameters, several problems were discovered.  About half of the images in the data set were discarded.  All of the images within a foot of the walls were taken out; VBASR should never be in that situation.  Repetitive images were also discarded to have a collection of unique pictures to test the vision algorithms. There are approximately 300 pictures remaining in the data set.
Other less noticeable changes were made to some of the code:

The next step is to develop a third algorithm in addition to the first two (lines and corners).  The third algorithm will take advantage of the difference in color between the walls and the floor of the hallway.  Take a look at this photo for example.
VBASR - the Vision System has been officially accepted to FCRAR!  The conference takes place on May 20 and 21.  The first draft of the conference paper is coming up soon...
Finally some real numbers! - March 12, 2010

After scattered work the last few weeks (thanks to many other senior year responsibilities) a large data set of about 600 images was collected to be used to evaluate the algorithms for VBASR.  Secondly, a program was developed that will score the algorithms by counting the number of 'correct' and 'almost right' images out of the original 600. 
        The images are split into 7 regions:  Hard Left, Left, Slight Left, Straight, Slight Right, Right and Hard Right.  If the algorithm selects the correct direction then it has evaluated correctly.  If it only misses by one slot (selects Slight Left when it should be Left) then it evaluates as correct for that image as well.  This is necessary to account for the fact that the 'correct' direction in a given image is often a tossup between two regions.
        After much testing and tweaking of my current algorithm, the best result was an algorithm that chose a correct direction 83.39% of the time, meaning it correctly identifies the direction it should go for every 5 out of 6 images!
        Next up is to develop a third algorithm (possibly based on the color of the hallway) to raise the percentage of correctly identified pictures even higher.

Also, please check out the photos page!  Large update completed!

VBASR: The Vision System abstract was submitted for review to the Florida Conference for Recent Advances in Robotics (FCRAR) on February 29.  FCRAR 2010 is hosted by the University of North Florida in Jacksonville Florida. The abstract was a week late due to the Design Review Presentation that was due the day after the abstract for FCRAR was due.  However, with a generous extension from the folks in Florida, the abstract is officially in.  No word yet on official acceptance.
Visit FCRAR.

Making Progress - February 12, 2010

The last few weeks have seen a marked improvement on the computer vision front.  OpenCV has successfully installed and works (finally).  After many different tests and example programs the latest program has shown some good possibilities.  The current algorithm that is being used is rather convoluted (pun intended!). 
1.  Input image via webcam (grayscale or color)
2.  Image Smoothing (Median or Gaussian filter)
3.  Canny Edge Detection
4.  Corner Detection
5.  Vertical Line Strength
6.  Decision

This latest algorithm will be tested next week.  Photos showing OpenCV functionality will be added in the short-term future.

Return to Bradley - January 25, 2009
While classes started a week ago, this is the first updated of the new semester.  The laptop we are using has been successfully re-formatted and re-fitted with Windows 7 and all of the peripherals have been re-installed.  Microsoft Robotics Developer Studio (MRDS) still works as expected.  The major task at hand is getting OpenCV installed correctly and operational within Visual Studio 2008.  After working through many different tutorials unsuccessfully, just this morning a very exhaustive 11 page PDF document was found.  This new tutorial will be attempted tomorrow.  If successful, the document will be uploaded to the website for anyone to use.  As you can probably tell, the installation process for OpenCV is not simple and overly painful.
End of the Semester - December 08, 2009
This will be the last update of the Fall semester. The issues described in the last update have been resolved.  After much discussion, no new hardware will be bought at this point.  If the computer vision programs are up and running, then the department will purchase a small, light netbook computer.  In this way, all the wireless issues can be avoided, along with worrying about drivers and whatnot.  The netbook will be able to run Windows and MRDS.  In this fashion, the hardware will be implemented.
The software implementation will be more difficult.  Using a wrapper to enable the C++ files (OpenCV) to be called in C# files (MRDS).  In such a manner, new programs written in C# will be able to run the OpenCV functions.  This will be a steep learning curve.
Enough of the minor issues.  Now the meat of the project needs to be explored and learned. Winter break will be spent reading, learning and practicing Computer Vision, thanks to the textbooks that Professor Schipper has obtained. 
Presentation - December 03, 2009
As the semester is wrapping up all of the groups need to give their initial presentations. The slides are available on the Deliverables page. The presentation went very well over all and my audience asked difficult questions.  Several people (including my advisor) offered a lot of feedback and the slides posted are actually slightly revised.
New Problems - November 20, 2009
After several weeks of research and documentation, a few major problems have cropped up. The first issue is considering the communications between the iRobot, Webcam and processor. There are several options: other hardware, mounted laptop, or off-board computing. Each of these options introduce new problems. My project is about computer vision so I want to spend as little time as possible to interface the system together.
The second major issue is programming. Microsoft Robotics Developer Studio uses C# for its programs.  OpenCV uses C/C++.  This issue is yet to be resolved also.
First Video - November 12, 2009
First communications with Microsoft Robotics Developers Studio and the iRobot Create were established last week.  Working on the webcams and investigating the image processing services within MRDS. The documentation suggests that it is a poor image processing program but it may be all that the VBASR needs. The first video is uploaded...the download time is very long. My apologies. A few pictures were added also.  The documents are still coming.
Website Created - October 29, 2009
The website is officially up!