Asking good questions at seminars, or when reading a paper.

How do you ask good questions of research presentations and papers? 

If the following are not answered, they have to be brought out.

The first thing you must do is ask, What is the story here? What is being claimed? 

Second, are there any obvious problems with the work, whether in the data, in the argument, in the theory, that make it hard to credit the work. In professional work, this should be rare. 

Third, is there a proposed mechanism that makes sense of what is being claimed: where a mechanism might be a market, a social interaction, a political process  ...

Fourth, what did the author(s) actually do?

Now, here are some other considerations.

1.  Imagine that you had only one shot, one chance to ask a question, and you had perhaps 100 words in which to phrase it. That might help you focus better. As important, if you put yourself in the shoes of the people who are being studied, say underground residents in Beijing, you might be able to think more clearly about what is interesting in the study of their lives. While we put a lot of emphasis in methods courses on hypothesis testing and the null, much research is about finding out what is going on--even roughly, and perhaps not with the confidence you might get with large N. 

2. External and internal validity are useful notions. In practical work, affecting policy and planning, one is trying to find out something at all. Of course, you want a sense of how reliable is that something. My impression is that your knowledge of the situation is very weak, and so that something is a vast improvement. Better to have an example or two to generalize from than nothing. 

3. Analogy and comparison are powerful strategies. If something is new or foreign to you, look for similar situations in what you know. That may well be biased and unreliable, but at least you are now starting out with something rather ignorance.

4. Most effects have many causes, and most causes have diverse effects. But can you at least connect two phenomena, even if they are not simply connected?