Low Grades, Rejected Papers, Tenure Denials--Moving Forward

You have produced a class paper or a scholarly paper which you expect to get you a good grade and will  be published, resp. For whatever reason the paper gets low grade, the solid journals seem uninterested in publishing the paper.

It does you little good to impugn the judgment of your instructor or the editor and referees of the journal, or the quality of the course. It does not help to find that they did not follow the rules, or were prejudiced against you (it may hurt them, but that won't get the article published, or your grade raised)--legally and administratively you may have many legs to stand on, but I am concerned here with your research career. You may win, but you will find that five years down the people will still be wary of working with you, even if they were on you side. I don't like saying this, and I'd rather believe there are no risks here.

What you want to do is to figure out the strongest case for your work, figure out how to acknowledge and deal with reasons for your low grade or rejection--seriously, and with no rancor. If you threaten the editor, the referee, or the instructor--however subtly--you are likely to discover that your weaknesses as a writer/researcher will become more clearly indicated when people evaluate your career so far.  Throwing sand in their eyes will lead to blowback that is likely to disable you.

On the other hand: You could launch a political, bureaucratic, and discipline-wide campaign. You were unfairly dealt with, and others agree with you. But you will surely need legal advice, and and campaign guidance. Stick with the facts you can ascertain. Any speculation needs to be well supported and carefully argued, Usually, this is a long haul. If there is to be bile and insult, find a surrogate to do that work, and stay above the fray.

Do you really want to be a professor in a research university? If so, you must do your research, disseminate it, and train students.

In an earlier post (Academic Dogs), it was pointed out how a large fraction of doctoral recipients never publish anything. If you include just one article in any journal, it must hit 4/5th or more.

If a dissertation is worthy of a PhD, it surely should be worth publishing an article from it, or at least lead to articles from subsequent work.  Now, large research universities produce hundreds of recipients of doctoral degrees that are focused on research (usually, PhDs).

My impression is that the main decent journals in each field would be overwhelmed if every PhD had to be accompanied by one published research article drawn from the research. Perhaps I am wrong.

[I am leaving out professional doctorates, MD, DDS, JD, DSW, EdD, ... , which are not in general serious research degrees.]


If you do not see yourself as contributing to the research literature, for the next 15-30 years, but you need the kind of training provided by the PhD, don't take a job that then requires you to do research and publish. 

Analogy is Destiny

What you want to have in your back pocket are enough relevant examples, so that each time you encounter something novel, you can see it as like something else.An example may be like another example, or exemplary of a theoretical model. So, in general, you know some general theory, and you know a range of examples and models, and you try to find a fit between what you are confronted with and your explanatory methods and your analogous situations.

What's crucial is that nothing is unique, unheard of, new, etc. That does not mean there is no novelty, no inventions, no creativity, etc. Rather, whatever it is, it is like something else.

[In physics, the story of electricity and magnetism, due to Maxwell, is the model for theories of the nuclear and weak forces, and gravity too. In physics the account of the empty universe is the same as the account given of a crystalline lattice.]

You may not realize just what is the analogy, and just how something is analogous to something else, but eventually you figure it out.

Article on "Academic Dogs" from http://www.freebanking.org/2013/06/27/academic-dogs/

Academic dogs

by Kurt SchulerJune 27th, 2013  10:35 pm

A link from Marginal Revolution took me to a paper called "An Empirical Guide to Hiring Assistant Professors in Economics." It is as interesting for what it doesn't say as for what it does. It concludes that "top 30" Ph.D. programs in economics, which accept a bunch of quite bright college graduates every year, do a terrible job in making those who graduate capable of publishing work that academic economists find sufficiently worthwhile to accept for publication in academic journals. From all the top 30 programs combined, the average number of economics Ph.D.s in the period the authors studied was 460. Only 143 (31 percent) had at least one publication in any academic journal ranked by the authors six years after graduating. Even at Harvard, Chicago, or Berkeley, the bottom half of the class essentially published nothing. The paper is talking about graduates, and is excluding students who didn't complete their degrees. Another finding: for graduates who were not at the top of their Ph.D. cohort, Princeton, Rochester, and the University of California-San Diego seem to have provided the best preparation for writing publishable academic papers.. . .

Another interesting thing the paper leaves unsaid is what Ph.D.s who don't write academic journal articles do. The Dutch economist Arjo Klamer once wrote an essay called "Academic Dogs." (It is in David Colander and Reuven Brenner, editors, Educating Economists, Ann Arbor: University of Michigan Press, 1992; I haven't found it online to give a link.) He compared academic publishing to a dog show. Owners spend a lot of time training their dogs to run through some paces to impress the judges at the dog show, but how useful is that outside the arena? Klamer decided that both he and his dog would be happier by staying out of dog shows. It is implausible to me that more than 300 Ph.D.s a year from the top 30 programs have nothing to say. Rather, I strongly suspect many of them have decided that the dog show of academic publishing does not interest them. Some prefer teaching students to publishing. Others go work in nonacademic settings--consulting firms, central banks, government agencies, banks, think tanks, international organizations, etc.--and write for audiences other than the dozen people in the world who care about some narrow academic topic and are unable to make anything come of it in the world.
For recent or future Ph.D.'s who like the academic dog show, fine: you will probably do well at it. A few of you will even do work that is truly important, that changes the world a little bit or at least changes what teachers of economics teach their students about the world. For those who don't like the dog show, take heart: you were wise to have studied for your Ph.D. in economics, not English, and you have many ways of using your knowledge in a worthwhile way outside of academia, at a good salary.
--------------------------------------------------------------------------------------
From the Vanderbilt paper:

"Going further down this table, we see that one would be better off hiring a 95th percentile graduate of a typical non-top 30 department than the 70th percentile graduate of Harvard, Chicago, U. Penn,
Stanford or Yale, or an 80th percentile graduate of Berkeley, Michigan, NYU UCLA or Columbia."

From the Vanderbilt data, average rankings plus minus standard deviation:

Department Rankings based on Graduating Cohorts' Publication Performance (1986-2000) Red is 1-4th, Green is 5-10th, Yellow is 11-15
Department	Coupe Rnk	Ranking	at	Percentile:
Percentile		99th	95th	90th	85th	80th	75th	70th	65th	60th	55th	50th	45th	40th	Av Rank	Stdev
Princeton	11	3	3	1	2	2	2	2	2	1	1	1	2	1	1.769231	0.725011
Rochester	19	10	5	3	3	1	1	1	1	2	2	3	1	2	2.692308	2.496151
MIT	5	1	1	2	1	3	3	4	4	6	6	6	6	6	3.769231	2.087816
UCSD	26	18	9	6	6	4	4	5	3	3	3	2	3	3	5.307692	4.250189
Harvard	1	2	2	4	4	5	8	6	8	8	8	8	11	11	6.538462	2.989297
CMU	30	15	12	11	7	6	5	3	5	5	5	5	5	4	6.769231	3.585941
Northwestern	7	9	6	7	5	7	6	7	7	7	7	7	7	9	7	1.080123
Yale	8	4	4	5	8	8	9	11	11	10	11	11	10	8	8.461538	2.633609
UBC	25	24	16	13	12	9	7	8	6	4	4	4	4	5	8.923077	5.978594
Stanford	4	6	10	9	10	11	11	9	9	9	9	10	9	10	9.384615	1.26085
Toronto	24	8	7	12	11	13	14	13	12	11	10	9	8	7	10.38462	2.399252
Penn	3	7	11	10	13	12	12	10	10	13	13	14	15	14	11.84615	2.192645
Chicago	2	12	8	8	9	10	10	12	13	12	15	17	12	30	12.92308	5.751254
Berkeley	6	17	15	17	16	17	16	16	16	15	14	13	14	12	15.23077	1.589227
Duke	16	25	17	16	14	15	15	15	15	14	12	12	13	19	15.53846	3.430631
Minnesota	21	13	13	14	15	14	13	14	14	16	16	18	17	26	15.61538	3.500916
Madison	15	16	18	18	18	16	17	17	17	17	17	19	16	13	16.84615	1.463224
Columbia	10	11	14	15	17	19	18	18	21	22	23	20	30	21	19.15385	4.705425
Cornell	14	22	23	22	22	21	21	19	18	19	19	15	18	18	19.76923	2.278664
Davis	23	20	22	21	20	22	20	20	19	18	18	16	20	28	20.30769	2.839736
UM	9	21	21	20	19	18	19	21	20	20	20	23	21	23	20.46154	1.450022
UCLA	12	14	19	19	21	20	22	22	22	21	22	26	26	17	20.84615	3.262058
Penn	29	29	24	24	24	23	23	25	23	23	24	22	29	16	23.76923	3.192539
Maryland	18	26	29	25	25	25	24	23	25	25	21	21	19	27	24.23077	2.681848
NYU	13	19	20	23	23	24	26	26	27	27	27	30	27	22	24.69231	3.172397
USC	27	5	30	29	27	31	28	28	28	28	30	25	31	15	25.76923	7.473886
UIUC	22	28	28	26	26	26	25	24	24	26	25	24	28	31	26.23077	2.047513
Ohio	17	31	27	30	29	29	27	27	26	24	26	28	24	25	27.15385	2.192645
Boston	28	23	26	28	28	28	30	29	29	30	31	29	22	24	27.46154	2.8465
Austin	20	30	25	27	31	27	29	31	31	31	28	27	25	20	27.84615	3.236411
Non-top	30	27	31	31	30	30	31	30	30	29	29	31	23	29	29.30769	2.213015

Living Well is the Best Revenge: Tenure Denial, Avoiding and Dealing With

Some immediate remedies for those of you coming up soon. 

1. If you have done joint work, be sure your personal statement says just what you contribution was, and encourage your department to get additional letters from the collaborators to that effect (but not as independent referees). Joint authorship is now the norm in much of social science and departments have to figure out a fair way of dealing with it.

2. As for significance, again in your personal statement, make clear what is the significance of the findings. No need to brag. Just tell it. From what I have seen, departments have definite ideas about significant work, and may well not tenure people whose work is significant but not by their lights.  

3. You do need to publish in well regarded journals. But as important is the importance and seriousness of your work. Everyone can point to articles in the best journals, and wonder how they got through the review process.

4. If you are writing a letter of reference, and want to be influential, you must be fair, you cannot cast aspersions. Otherwise, you will be ignored unless someone wants to use your letter to do a hatchet job, a job that won't hold up on administrative review.

5. Whatever happens, living well is the best revenge. (This seems to come from George Herbert, but may well have been in common usage. Moreover, many point out that revenge is not a good idea, and you are better off looking forward or forgiving. In any case, living well will surely be good for you)
  In other words, go off to another institution (whatever you may do legally about your tenure case), be productive, and realize that your success is what really counts.  Don't worry about prestige--just ask whether the institution will allow you to do your work and support you. In the scholarly world, the connections you have with those people--throughout the country and the world--working in your area and who think well of your work is what counts--this is David Riesman and Christopher Jencks of 40+ years ago.

6. As for departments that deny tenure to an apparently worthy candidate--make sure all your future appointments and promotions are much stronger than that of the candidate you denied.  Otherwise, you set yourself up for legal problems, and it is likely your provost will not be happy discovering that you have left the university open to liability.  

My other blog, This Week's Find in Planning, and the new book out this Fall, has much more on tenure.

Outline for a series of last lectures

DRAFT--The list is too long, and has some missing parts. MK 23June13


You want to say something about what you've learned.

1. Thinking and having ideas you work out is good work
2. Reading widely, but also knowing who is authoritative.
3. Take quantitative work seriously, but also ask how it is vulnerable statistically or in how it was gathered.
4. Models and mechanisms drawn from natural science are often useful, but rarely probative.
5. Discourse on the sacred from religion is often very useful.
6. Concreteness is essential. Abstraction that is cute and wonderful is cute and wonderful. Whether it is essential is up to the facts of the matter.
7. Many if not most people are much smarter than I am. They can follow arguments in philosophy, or in theory, or in mathematics or some such; they know about many concrete situations (they are specialists). Most of the time my eyes glaze over. In any case, I always check my work (that is, my speculations) with experts to be sure I am not missing something big. Usually they tell me: It's ok, but so what? Or, Only you would think of this!
7a. Many people are much more capable than I am of holding in mind a long complex argument.
7a. Most people are experts in their fields, and they can ask decisive questions about research work. All I can do is ask about motivation, general thrust, and provide analogies with other work. I can be readily bamboozled by an expert. On the other hand, I am good at listening to the music and picking out the big themes.
8. Analogy is how I come to understand stuff: Something is like something I do know.
9. Usually, only after I have written a book, and it is published, do I begin to understand what I was up to.
10. I take lots of photographs, but I am not a photographer as that is understood.
11. Everyone needs someone who understands what they are doing, and can mirror them back to themselves. I'm good at this.
12. Students are surely vulnerable, but unless you tell them like it really is, they are likely to make big mistakes. Often they undervalue their strengths. What they need to learn is how to do work, and how to do projects, and also how to lead and contribute. About the latter I have nothing to say, except by analogy to sacred figures.
13. In general references in social science or the humanities, reference meant to model something, to quantum mechanics, relativity, general relativity, quantum field theory, string theory, etc gets you noplace. On the other hand, you can say the features of complex adaptive systems in such a way that a historian would find them very familiar. Chaos Theory, Catastrophe Theory, Nonlinear..., etc usually provide nice ideas but none of their formal apparatus will help you. The Uncertainty Principle, the Second Law of Thermodynamics, "path dependence", The Conservation of Energy,... may well be useful models, but again it's very unlikely that anything substantive will be learned from those subfields of physics that could not be said in ordinary everyday language. On the other hand, the equations of physics or chemistry might well be useful for social science or the humanities, but that does not mean that the physicist's motivations for those equations is apt. In fact this applies to the physicist who borrows an equation from a different subfield.

'Nailing It" in Acting and Teaching

This morning on radio there was an interview with the writer James Patterson, who indicated he had no problems with writing (he loves telling stories). He refers to Morgan Freeman in some movie, where Freeman was fully confident of his acting ability, and Patterson said of Freeman, He nailed it.

I have been teaching for long enough so I have a good sense of my talents. I recall early on, I was struck by the fact that I could talk to a class, having made up and prepared an argument by myself, and it was OK. This was not like teaching physics, nor was it in a course with a textbook. I had to do it all, albeit informed by the reading I had assigned. I was shocked by what I was doing.

But it worked more or less, and as the years went on I discovered other strengths. I am a good consultant and coach, and I can think in front of a class. I can deal with student questions and go someplace with them. There's lots I am not good at, but in effect I have "nailed it" for the roles I have chosen to play

The blurb from Sherry Turkle on the main page of this blog reminds me of what I do well..

What I don't do that other faculty do well leaves me vulnerable to questions about my authority. I can be thrown by such questions, end up writing about them in this blog, but my deeper confidence and my resilience is based on many years of teaching and on the many students who believe that what I do is valuable to them. 

My colleague and friend Bill Alonso (professor of regional planning at Berkeley, and then at Harvard, who died in 1999) said that our salaries are rent paid on our experience set.

It's NOT about You!

Of course, my postings are based on my experience and stories I have been told. In every case I anonymize and also conflate cases. For any particular issue, I have several examples in mind. So if you read a post, and say

Professor Krieger is writing me, and he is distorting who I am.

THAT'S NOT WHAT'S HAPPENING!

I am deliberately mixing in several stories, anonymizing, and also altering my account to make poignant the points I want to make. This is not meant to be journalism or testimony. Rather it is meant to be homiletic, teaching a lesson. I borrow from what people say to me, but combine that from several situations.  So...

You'll recognize yourself in my postings. But it's only partially you, and not necessarily the part you think is you!   Usually what I write about is the pervasive idiocy we all share. The acting out. The vulgarity. The stupidity. Etc. And sometimes the dignity and good sense we have.

MK

Grade Inflation and Real Excellence

In talking to one of our program staff, grade inflation was suggested as a reason for student dissatisfaction with their grades. They had received, say, A's, and now they received A- or B+ or... and could never imagine getting a B-.  I am not savvy about what other professors do, but it seems that often the choice is [A, A-, B+], at least for graduate students. I don't think of myself as de-grading students. Rather, it seems their expectations are not up to the competition.

Grades are a way of telling a student how strong is their work, how reliable is what they do. And if they choose to let others know of their grades, or if they are forced to by the requirements for a job application, presumably those grades should be good indicators of performance. It strikes me as unethical to send students out into the world with grades that do not indicate their true strength.

Of course, one might say that a grade is "earned" by following the rules or a rubric, or that tenure is "earned" by your doing a certain number of articles published in certain venues. As for the latter, at strong universities the question that is asked is, What is the contribution of the work?  As for earning a grade in a course, what you really want to ask is, Does this person know how to think and write about this area?  Of course, you might begin by checking all the boxes, and then asking the italicized questions. [One problem is that students often claim that they have done what they are supposed to have done, but they do not tell you, for example, that their work is plagiarized or poorly sourced. Or, having done what they were supposed to do, their work is weak nonetheless. As for faculty tenure, having done what you were supposed to do, the contribution of the work might be considered insignificant.]

Put differently, there are two issues: reliable indicators of the quality of the work and the person's strengths; and, the conflation of a set of criteria with real excellence.

I do understand the concern that the evaluation process be fair, and hence the desire for objective, almost automatic criteria.  But if you want excellence those objective criteria won't be reliable indicators.

A Basic Grading Rubric for Papers, Undergraduate and Graduate.

What's most important is to go through papers with students, in person, showing them how to make the work stronger. Grades may preoccupy them, but getting stronger should be your focus.

0.       Plagiarism: Is there any violation of academic integrity? If so, send to university committee. Usually obvious to reader, but sometimes confirmed or indicated by Turnitin. Not only unquoted but copied passages, but also features of the argument that are taken from elsewhere. Often harder to detect.

1.       Organization: Is the paper well structured? Clear summary at beginning, divided into sensible parts, each part starts with a topic paragraph.

2.       Scholarship: Is the scholarship reliable? Are the sources sufficiently wide-ranging. Are opposing positions and problems dealt with? Are historical accounts sensitive to the back and forth of “progress”? Is the description of cases and examples likely to be reliable? Are there serious lacunae?

3.       Rhetoric: Is the rhetoric fair. Are you fair to your opponents?

4.       Readability: Does the paper scan? Problems with language and spelling and grammar. Can I read the paper without having to parse sentences? Are spelling errors rare or understood as mistakes?

5.       Thinking: Does the level of thinking and research reflect the demands of this stage in one’s academic career?

6.       Topic: Can I figure out what the paper is about? Does the paper fulfil the assignment?

7.       Excellence: Would I be proud to show this work to a colleague? In general, potentially excellent papers have no #0-6 problems.

In general, a student will ask to have their grade changed because they feel the work is stronger than recognized by their grade. (It is helpful to prepare a memo pointing out the strengths of the paper, in light of the above rubric.) In order to avoid moral hazard, as the economists call it, there is a possibility that a rereading of a paper will result in a lower grade. (Otherwise, if rereadings could result only in an unchanged or an improved grade, the incentive system would not encourage students to carefully think about the problems in their papers.) 

On the other hand, students might feel that the professor could retaliate against them for even asking for a grade change. Hence it is crucial that there be oversight by the department or school. In the background is the fact that in a university, the professors are rationally presumed to have authority and integrity--although this is open to question. The professor can be challenged. But the professor's authoritative judgment is what the student is seeking.  This is very different than most challenges to authority, because often those challenges are meant to negate authority. 

In Organizations: The Structure of Papers for Policy. Bottom Line Up Front=BLUF

If you are writing a memorandum or a study or a white paper or... within an organization, the purpose of which is to inform your boss about what to do, what you must do is make it easy for the boss to get the message.

1. Bottom Line Up Front: In the first page the main points are made. The reader is assumed to know much of the background, so what you want to do is to convey the new information, the new plan, the criticism or evaluation, ...

2. The first main section should be an elaboration of #1. And it had better be written clearly, concisely, and with the needed indications of authority (references to authoritative literature (no websites, in general), summaries of studies undertaken). This is your only chance make your case and to work out the consequences of what you are talking about.

If there are any problems in the writing of #2 or #1, you are toast.

3. Additional materials:
a. background and motivation
b. systematic review of the literature
c. history of how we got here
d. details of case studies, fieldwork, experiments, pilot projects,... (you mention all of this in #2 in a paragraph)
e. alternatives rejected (although you will want to give this very briefly in #1)

Note that this format is the reverse of what you are taught in most writing courses and for most papers and even journal articles.  Your reader is a sophisticated busy person, so much of what would be in #3 is already known, and what they care about is #2. Just be sure that #1 does not allow them to stop reading since it is so poorly written.

Conducting a Department or a University

In talking to a colleague who has experience conducting opera, I realized that a model for managing and leading a university is the orchestra conductor for an opera. Many distinguished performers, a larger vision, the need to keep all the egos in check for the purpose of the larger vision, and a fragile authority.

More to follow...

Is Criticism per se Negative? Forgive and Remember.

In this blog I have tried to address the reader directly, especially about matters where students and faculty get themselves in trouble. My descriptions of pathologies are not meant to address the reader negatively; rather they are meant to describe the problem in such a way that you can avoid it or correct it. Just because you have to admit to yourself and others that you have erred is not a sign of failure. As pointed out in Bosk's Forgive and Remember, about the training of surgeons, the trick is to admit your mistakes, forgive yourself and be forgiven by your superiors, and then go on. Not Remembering is the sure road to future errancy. So "a positive learning environment" is one in which problems are faced directly, and dealt with, and where future judgments are not too much influenced by mistakes. In other words, one focuses on the problems themselves, without making a judgment about those you are addressing. Truly supportive environments make it possible for students and scholars to become more effective. To talk about errors, ones that are made by many scholars and students, is not to damn them: it is to prevent them from continuing to make mistakes.

One other fact: Any problem that I describe is not only or mainly about you. It is recurrent and ubiquitous. I never write about idiosyncratic problems.

The best positive resolution of mistakes is your being able to go forward not making the same mistakes as you have been making. In Bosk's terms, you want to remember your errors, that memory also within  the system as well, correct the errors in the future, and be forgiven as a person.