HW5 - Annotated Bibliography

Your annotated bibliography must include at least three peer-reviewed conference or journal papers on a single topic. At least one of these papers must contain a reference to some other paper in your bibliography. For each paper, you must provide complete bibliographic information as well as a brief summary of the paper. The summary should describe the key results, note any references to other papers in your bibliography, and explain the connection to those referenced papers. Your paper summaries should be in the ballpark of 2-3 paragraphs each. At least one of your papers should have a publication date within the last four years.

Finding Papers

Here are some possible starting points for finding high quality papers.

Google Scholar is a good starting point. A good way to get started is do some keyword searches related to your topic and take a look at the most highly cited papers that appear relevant. There are many low-quality or uninteresting papers out there. Citation counts provide a good mechanism for focusing attention on noteworthy papers. Once you find an interesting paper you can follow forward and backward citations to get a deeper understanding of the topic.

The ROS wiki contains a list of robotics papers that have publicly available ROS implementations.

I've highlighted a few of the top robotics conferences and journals below. Unfortunately, many of these conferences do not make their proceedings available on-line.

First-tier Robotics Conferences:

• IEEE International Conference on Robotics and Automation (ICRA)
• IEEE International Workshop on Intelligent Robots and Systems (IROS)
• Robotics Science and Systems Conference (RSS)

General AI Conferences that Include Robotics Papers:

• Association for the Advancement of Artificial Intelligence (AAAI)
• International Joint Conference on Artificial Intelligence (IJCAI)

First-tier Journals

• The International Journal of Robotics Research
• Journal of Field Robotics
• IEEE Transactions on Robotics
• Autonomous Robots
• Robotics and Autonomous Systems

Reading Research Papers

Reading a research paper is not like reading a novel or even a textbook. Research papers are usually written under the assumption that the audience will be other researchers in the same field. In addition, papers are often written under strict page limits that restrict amount of background information the authors can provide. The keys to making sense of research papers are patience and perseverance. I suggest the following steps.

• Start by reading the abstract, the introduction and the conclusion. At this point the goal is to figure out the big-picture claims that the authors are making. What have they accomplished? Why does it matter? At this stage you may determine that the paper is not worth reading. If so, move on to a different paper.

• Next, read the whole paper all the way through. Twice. Read carefully, but don't get bogged down by equations or details. Get a sense for the overall organization of the paper and the main points that are being made in each section. Underline or highlight points that seem particularly important or difficult to understand.
• Finally, go back and try to puzzle out the parts of the paper that were unclear on the first few read-throughs. You will probably find that there are terms or concepts that are important, but are not explained in the paper. The Internet/Wikipedia may be helpful. You may also find that the authors are assuming that you are familiar with some other paper or body of work. You obviously won't be able to read every cited paper (and all of the papers cited in those papers), but some extra reading may be unavoidable.

By the time you finish, you should understand the key points that are being made in the paper. You may not understand every sentence and every equation, but you should know what you don't know, and be in a position to discuss it.

Presentations

I suggest the following three step organization for paper presentations:

1. Provide Context - The first step is to make sure we understand why we are reading this paper. Questions you should address:
   • Why did you pick this paper?
   • When was the paper written?
   • What problem is it addressing?
   • What were the previous approaches to the problem?
   (The paper itself should contain the authors' answers to the last two questions)
2. Walk us through the paper - The purpose of this step is to make sure that we understand the content.
   • Describe the algorithm
   • Describe the experiment(s)
   • Explain the results
   • Highlight any aspects of the paper that were unclear to you so that we can try to figure them out as a group.
3. Analysis - The purpose of this step is to help us to think critically about the paper. Everything up until now has been a description of the paper. Here, the goal is to talk about the paper. The best format is probably a guided discussion. General questions to address might include:
   • Did the author's follow through on their promises? Did they successfully address the problem?
   • Are the experimental results convincing?
   • Do the authors discuss the time complexity of their approach? Are there performance bottlenecks that they do not discuss?
   • Do they provide a fair evaluation of the approach?
   • Are the results important? Why should we care?
   • What questions remain unanswered?
   • Did this work lead anywhere? If not, why not?

You will need to distribute at least three discussion questions on Piazza at least two days before your presentation. These questions should help us guide our reading of the paper. The questions could focus on areas of the paper that you found unclear (area 2 above) or on the value and impact of the paper (area 3 above).