How to write an effective critique of a paper

The following is adapted from Carlo Tomasi's excellent guide to writing paper reviews.

Here are some of the items that should appear in your paper critiques. However, it is up to you to add to these elements anything you deem useful, interesting, or fun. In your critique, provide a set of issues or questions to lead off a discussion. This can be done by asking a series of questions about the paper, or by advocating strong opinions for or against a given method. This item is important for our class discussions. Writing your main points in your critique is a way for both you and all of us to think about the topic ahead of time.

Primary Questions

• Put the complete and correct citation for the paper on the critique.
• Summarize the paper in a few sentences at the beginning of your critique. This should be your own summary.
• Elaborate on your synopsis with a brief summary of the paper, highlighting what is new, what is old, and what is important. Sometimes definitions or brief explanations of difficult or technical aspects of the paper are appropriate.
• State in what way this paper contributes to our understanding of the problem addressed by it. For instance, does the paper describe a mathematical technique from a different field? If so, how can we use it for a task, and what modifications are we likely to have to make? Are these modifications trivial, or are major breakthroughs needed? If the paper describes a system, what is useful about it? How can we improve on it?
• What assumptions need to be made for the techniques described in the paper to work? List both the assumptions explicitly made by the authors and any others you may think of yourself. Give specific examples in which the techniques would and would not work.
• Is the content of the paper good? This question is delicate, and has several facets. When reading a technical paper, we are sometimes dissatisfied with the quality or generality o f the results, the level of mathematical treatment, or the style of the presentation. However, critical reading does not mean finding fault with the paper. On the contrary, it means finding the gold nugget in it, no matter how deeply it may be buried. A brilliant idea poorly presented is still a brilliant idea. If an algorithm requires many assumptions, it may just mean that the author was more careful or more honest than others in making those assumptions explicit. A paper that overlaps 90 percent with something else may still contain 10 percent new material. Please be positive in your comments, but do point out flaws.
• Do the experiments in the paper, if any, back up the claims? Are the results interpreted correctly? Are the experiments representative of the situations for which the proposed me thod is likely to be used? Are the experiments convincing?

Auxiliary questions

These questions are secondary given the purposes of our class, but they will help you understand how to write better papers yourself.

• Does the paper integrate knowledge from other fields? What background did the authors come from, what are the weaknesses of the method, what are its strengths, for its intended goals?
• Is the paper well written? Is its structure satisfactory? Is it clear?
• Is proper reference made to other papers in the field?
• Is the mathematics useful, or does its complexity hide lack of ideas or get in the way of comprehension?
• How does the paper compare or contrast with other papers that we have read so far? Can concepts read somewhere else integrate or strengthen this paper? Are there contradictions or incompatibilities with other papers we have read?