Annotated Bibliography

Your annotated bibliography must include at least three peer-reviewed conference or journal papers on a single robotics topic. At least one of these papers must contain a reference to some other paper in your bibliography. For each paper, you must provide complete bibliographic information as well as a brief summary of the paper. The summary should describe the key results, note any references to other papers in your bibliography, and explain the connection to those referenced papers. Your paper summaries should be in the ballpark of 2-3 paragraphs each. At least one of your papers should have a publication date within the last three years.

Finding Papers

As a first step, I would suggest browsing through the programs of recent first-tier robotics conferences: IROS16, IROS17, ICRA16 and ICRA2017. Unfortunately, the programs do not contain direct links to the papers, but all of these papers are available through JMU's institutional subscription to the IEEE digital library.

Google Scholar may also be useful. A good way to get started is do some keyword searches related to your topic and take a look at the most highly cited papers that appear relevant. There are many low-quality or uninteresting papers out there. Citation counts provide a good mechanism for focusing attention on noteworthy papers. Once you find an interesting paper you can follow forward and backward citations to get a deeper understanding of the topic.

I've highlighted a few of the top robotics conferences and journals below. Unfortunately, many of these conferences do not make their proceedings available on-line.

First-tier Robotics Conferences:

• IEEE International Conference on Robotics and Automation (ICRA)
• IEEE International Workshop on Intelligent Robots and Systems (IROS)
• Robotics Science and Systems Conference (RSS)

General AI Conferences that Include Robotics Papers:

• Association for the Advancement of Artificial Intelligence (AAAI)
• International Joint Conference on Artificial Intelligence (IJCAI)

First-tier Journals

• The International Journal of Robotics Research
• Journal of Field Robotics
• IEEE Transactions on Robotics
• Autonomous Robots
• Robotics and Autonomous Systems

Check with me before using any papers that don't come from one of the first-tier conferences or journals listed above. The majority of the papers in your bibliography should have publication dates within the last 4-5 years.

A Note on arXiv.org

The arXiv.org web site provides a popular avenue for quickly disseminating research results that may not have undergone a formal process of peer review. Sometimes arXiv papers are under review for a journal or conference, sometimes they are pre-publication versions of papers that have since appeared in a peer-reviewed publication, sometimes they are more like informal white papers that are not intended for formal peer review.

The arXiv is great, but for the purposes of this project you should only use arXiv papers if they have actually appeared in a peer review publication. This information will often appear in the "comments" section of a paper's arXiv page. The reference information you provide must include the full publication information.

Citation Formatting

I suggest that you follow the citation and reference format outlined in the IEEE Style Manual. I won't be picky about formatting, but you should be consistent and you must provide complete bibliographic information.

Note that a URL is not sufficient. At the least, your references must include the name of the journal or conference along with the original publication date.

Reading Research Papers

Reading a research paper is not like reading a novel or even a textbook. Research papers are usually written under the assumption that the audience will be other researchers in the same field. In addition, papers are often written under strict page limits that restrict the amount of background information the authors can provide. The keys to making sense of research papers are patience and perseverance. I suggest the following steps.

• Start by reading the abstract, the introduction and the conclusion. At this point the goal is to figure out the big-picture claims that the authors are making. What have they accomplished? Why does it matter? At this stage you may determine that the paper is not worth reading. If so, move on to a different paper.

• Next, read the whole paper all the way through. Read carefully, but don't get bogged down by equations or details. Get a sense for the overall organization of the paper and the main points that are being made in each section. Underline or highlight points that seem particularly important or difficult to understand.
• Finally, go back and try to puzzle out the parts of the paper that were unclear on the first read-through. You will probably find that there are terms or concepts that are important, but are not explained in the paper. The Internet/Wikipedia may be helpful. You may also find that the authors are assuming that you are familiar with some other paper or body of work. You obviously won't be able to read every cited paper (and all of the papers cited in those papers), but some extra reading may be unavoidable.

By the time you finish, you should understand the key points that are being made in the paper. You may not understand every sentence and every equation, but you should know what you don't know, and be in a position to discuss it.

Presentations

I suggest the following three step organization for paper presentations:

1. Provide Context - The first step is to make sure we understand why we are reading this paper. Questions you should address:
   • Why did you pick this paper?
   • When was the paper written?
   • What problem is it addressing?
   • What were the previous approaches to the problem?
   (The paper itself should contain the authors' answers to the last two questions)
2. Walk us through the paper - The purpose of this step is to make sure that we understand the content.
   • Describe the algorithm
   • Describe the experiment(s)
   • Explain the results
   • Highlight any aspects of the paper that were unclear to you so that we can try to figure them out as a group.
3. Analysis - The purpose of this step is to help us to think critically about the paper. Everything up until now has been a description of the paper. Here, the goal is to talk about the paper. The best format is probably a guided discussion. General questions to address might include:
   • Did the author's follow through on their promises? Did they successfully address the problem?
   • Are the experimental results convincing?
   • Do the authors discuss the time complexity of their approach? Are there performance bottlenecks that they do not discuss?
   • Do they provide a fair evaluation of the approach?
   • Are the results important? Why should we care?
   • What questions remain unanswered?
   • Did this work lead anywhere? If not, why not?

You will need to distribute at least three discussion questions on Piazza at least two days before your presentation. These questions should help us guide our reading of the paper. The questions could focus on areas of the paper that you found unclear or challenging (area 2 above) or on the value and impact of the paper (area 3 above).