McCloskey "The Insanity of Letters of Recommendation" (also Lynn Conway)

Eastern Economic Journal 28, Winter 2002 by Deirdre McCloskey. (For an interesting parallel, see Lynn Conway, who is one of famous names in VLSI (very large systems integration, on a chip) with collaborator Carver Mead (of Caltech). McCloskey is a distinguished economic historian, Chicago School, and has recently done a series of  books on the rise of bourgeois values and economic development.

I will quote her main point, leaving out her italics:

"The only correct procedure for assessing scholarship in hiring or promotion is for the responsible body to read the candidate's work and discuss its intellectual quality with immediate colleagues in a context of believably disinterested assessments from the outside."

I would add that it might be useful to have a designated devil's advocate (much as there is a designated driver).

------------------------------------
My new book does not take this point of view, but had I read McCloskey when I was putting it together, I would have featured her argument.