The Scholar's Creed

Drawing from my new book:

Bottom Line Up Front (=BLUF). 

    Where's the beef (the contribution)? 

    Everything is Like Something Else.

Do what you are supposed to do, now. 

    Perform at your personal best, 

          or the competition may bury you. 

    Force the evolution of your work, 

          and sell it to the world.

Beware of oncoming trucks. 

    Never give weapons to your enemies.

Would you buy a used car from this dean or chair? 

    Deans can count, but they don't read.

Living well is the best revenge.

Excellent Work

When I am reading a promotion dossier or a scholarly paper or a student's paper, I am first asking if the work is good. Does it meet sensible expectations? Is it well written and supported? Is it fishy--plagiarized, unedited, ...

As I am reading, I may discover there is lots more there than I would have expected. The contribution to scholarship is significant. The student paper could have been written by a distinguished colleague. (It has no gaping holes or problems, although that is also demanded for good work.) Perhaps the work violates all the conventional rules, but stands on its own, and is manifestly outstanding--rare but possible. I discover that the work or the paper is excellent, and I want to share it with colleagues.

Excellent student work in my university context is impeccable scholarship and exhibits consistent critical thinking. Excellent faculty work makes a strong contribution to the scholarly enterprise.

In the world of Harry Potter, there are passing grades: Acceptable, Exceeds Expectations, and Outstanding. For me, excellent work really is outstanding. Sometimes excellent work just exceeds my expectations, and I am grateful for it.

Scholarship and Critical Thinking

The scholarly enterprise is a dialectical one, where people make arguments presumably based on evidence and earlier work, and where others take apart those arguments and also make counter-arguments. One refers to the scholarly sources to show that one is in touch with this conversation, and is aware of its subtleties.

While plagiarism is a recurrent problem, the main purpose of references to the scholarly literature is to indicate your being in touch with that literature. Definitions and notions are sophisticated, and it is likely that even an authoritative dictionary won't settle the meaning of terms in scholarly discourse. (My favorite example is the use of "collaboration" as in collaborative planning. Used in that context, its meaning is clear. But all I can think of is Nazi collaborators, highlighted for me by the famous Robert Capa photograph of an alleged French collaborator after WWII, her hair cut off. I should note there is now some criticism of these photograph and what was done. There are many such photographs.)

Critical thinking skills mean that you not only know how to make an argument, but you indicate you are aware of the problems with your argument. That may be explicit, it may be in notes. You know that history is likely to be presented with a certain bias (but professional historians readily acknowledge their orientations, and that is part of their argument), so that simple stories of progress are surely nonsense. You know that an argument that has an uncritical or unscholarly section is likely to fall apart no matter how strong are the other parts.

Moreover, in scholarship and in critical thinking the idea that one might quantify everything, or most things, is generally thought to be a mistake. Judgments can be justified, but not always or often quantified. On the other hand, Kahneman's Thinking Fast and Thinking Slow points out how often quantified judgments might well be powerful, but what is then quantified is very different than what most people pay attention to. (Think of the movie Moneyball.) Clinical judgments might well be replaced by statistical tests, the diagnosing psychologist by an Minnesota Multiphasic Personality Inventory (MMPI) or its more recent successors.

One of the most interesting phenomena these days are automatic grading of essay questions, and the correspondence of those grades with those of readers of the essays. What you need to attend to are the poorly corresponding cases, to find out if you are missing important stuff: very poor essays that pass, and excellent essays that are missed.

More generally, you want to ask;

Who says? Who disagrees? Why?
Are there interesting analogies with other cases?
What are the specific meanings of notions in this field or discipline?

Here is what I wrote to one student:

There are two features of doctoral work that distinguish it from work done earlier in one's education:

     a. Reference to and acknowledgement of the scholarly literature. In other words, when you write a paper, you must deal with those who support your position and those who do not, and in particular you must be fair to all of them. By "fair" I mean that you have to represent their positions adequately, and deal with their support and objections to your own position. You may have strong feelings and an articulated position, but you must be able to deal then and there with objections. 

    b. You must think critically. This is a bit more general than a. Namely, you want to be twice as critical of your own position, so that you can be able to see its flaws and indicate how you might deal with them. 

Most popular writing on a position may present strong arguments, but usually it does not deal adequately with the scholarly literature and often is much less critical than it is advocacy. 

When you want to write a rationale for changing public policy, you would need to deal with the current orthodoxy and why it might make sense. On the other hand, were you being an advocate with no claims to scholarship, you will surely leave gaping holes in your argument but also be much more emotionally convincing.

As for procedures: If you have a problem with a course, your first recourse is to see the instructor. The university rules say that administrators come in afterward, if you are not satisfied with the resolution. Students should be empowered to deal directly with their instructors. (Of course, you are welcome to seek counsel from whoever you wish.) I don't want you to give up a proactive stance.

Is ace-ing a course, getting an A, about real excellence?

1. I just got a call from a student who said more or less, "My goal is to have a 4.0 average in my graduate work. I've always received A's. Why did I get a B+?" He wanted to know what he had needed to do to get an A, and told me he had done everything I had recommended on his draft. What was missing that he did not get an A?

{I have been going through the page proofs of the book drawn from my blog: The Scholar's Survival Manual. What is striking is the recurrence of certain problems. So my description here is much like my earlier description in that book. I think of a blog as a diary, and hence I do not worry much about repetitions in entries. Of course, once you set it up as a book, you try to cut repetition.

{My other discovery was that when I ask colleagues' and staff for advice about the university rules, they look knowingly at me, as if they too have encountered this situation. Now Pride Goeth Before The Fall. But I would rather think of this as a matter of not getting driving tickets and so keeping your insurance rates as low as possible. It's unlikely that your not getting a ticket over a multi-year period is a matter of your flawless driving and scrupulous traffic officers, although I am sure there are such people. You just finally get caught, for something minor, perhaps. Similar issues come up in criminology, where perpetrators get caught because they keep taking risks rather than just stopping. This stopping problem is a nice exercise in probability theory.}

I try to help students get stronger and do better work.  A's are for excellent work, and I take that to mean that I am proud to show the work to my colleagues. B's are for good work. My goal is to make it possible for all students to get at least a B, and when I fail at least a B-.  Only if they are negligent do they earn lower grades.

I have no problem changing a grade if I am incorrect in my assessment of the work, having reread it at the behest of the student. I try not to give a lower grade if the work is worse than I originally thought.

2. As to what is missing: If you choose a topic that won't allow you to excel, it's hard to do A work. If you have a mechanical notion of school work, it is unlikely that you will do well. Put differently, if you do not have a critical sense, you are unlikely to do a excellent work. For example, if you give an account of the history of participation in urban planning, and treat it as a story of progress and positive "evolution," you miss just what's interesting, including backtracking, and you mistake the notion of evolution for the notion of gradual change. Evolution does not go anywhere, it just goes--at least if we are Darwinians.

3. As in life: Talent matters. Inventiveness matters. Thoughtfulness matters. But you can learn to be a critical thinker. It's just what you don't learn when you do well in high school and often in undergraduate school (this is terrible, but it seems true).

4. By the way, I can't figure out what grades have to do with anything in doctoral work. People want to see your thesis work, and figure out how you can help them. Grades are the last thing on their minds. For master's candidates, who want to enter a doctoral program, or enter a professional school, I can see how they might be concerned about their grades. Most crucially, they should be concerned about the letters of reference that will be written by their teachers, and one might go:

     "Mr. Potter did B+ work in my class, but I was impressed by his inventiveness and industry, the quality of his thinking, and his knowledge. I would choose him over most of the A students because what is needed in our discipline is that inventiveness, industry, and thinking."

OR

      "Mr. Malfoy did A work in my class. He writes well. However, I would be reluctant to have him join my research group, his formal excellence does not extend to independent research work. I am sure he will graduate with a 4.0 average. I am sure that he will thrive in the right doctoral program. But I doubt that he would thrive in ours given its demands for independence and inventiveness."

5. I have been thinking back to my college life, 1960-1964 at Columbia. I was a klutz, and in effect I half sleepwalked through my education. There was no way I would get a 4.0 average: I did not know how to use Cliff's Notes and their equivalents, I took courses that I thought were important and knew that it was quite unlikely I would get an A, and some of the time I really did not know how to study (surely in the humanities, but sometimes in science and mathematics).  The legal scholar Richard Epstein was in one of my classes and he was sophisticated and smart and interesting. And one of my classmates who did not stand out in physics, as far as I knew, went on to a very distinguished career as an experimental physicist. So I had a sense of where I stood, and it was good but nowhere at the top. Of course, everyone knew that "the top" predicted nothing about your future, at least when compared to slightly below the top.

I had some idea that I wanted a liberal education, so study of political theory, social theory, great books, literature, etc, all seemed appropriate. I had to figure out what I would major in (mathematics, philosophy, physics), and I studied physics because it was something I could do well, and liked--one learned how the world worked!, while mathematics was "too hard" and seemed to require a gift I did not have. (In retrospect, my teachers could have given me the training so that I could have had more of that gift, but it was sink or swim. It was a matter of election.)

I am sure some students had 4.0's. We were allowed to get A+, so it was possible to be imperfect yet have a 4.0. I have no idea if they turned out to have great futures as professionals or whatever. I had this crazy idea that I wanted an education.

Workloads in the Research University

In order to maintain a vigorous research program, faculty need time. They might buy out some of their courses through research grants. Equivalently, they may have professorial chairs that allow them to have smaller teaching loads. If they want to have a number of doctoral students, they probably need to be able to support them. What's happening in the background is often there are fewer regular faculty than traditionally, and so students in didactic courses are taught by adjunct faculty.

1. In many universities, the rules are that we can buy out a course for, say, 10-15% of 9-month salaries depending whether the sources of funding comes with federal overhead, little overhead, or no overhead. If you add in fringes, that is 13% to 19%, before overhead. Again, large research projects, or writing a book, demand that we have more time than is usually available in a regular teaching load.

2. The supervision of doctoral students is part of our regular workload as faculty members, and for many of us our research would not be possible without graduate students. What seems crucial is whether we support our graduate students with external grants, and for how many years.(In an engineering school, the teaching load might be smaller, but the demands for externally funded research are more substantial I believe. They might still allow course buyouts, but may have credits for PhD supervision and funding students. These credit systems can become byzantine.) 

3. In general, regular faculty numbers are now much fewer than traditionally, given size and budget.

    The question is the quality of instruction we provide and who provides it. For undergraduates, coming to a research university should mean they are exposed to regular faculty (including research and teaching as well as tenure-track) for the most part. We ask tough questions when we appoint tenure-track and teaching and research faculty, and those questions mean that the research university gives our students the best. 
        Adjuncts are always valuable for fieldwork or studio courses, but for didactic courses in the usual subjects it would seem that students should expect regular faculty. At the doctoral level, it is crucial that research and tenure track faculty teach the courses, since mentoring and modeling are what counts here.

        But, often, adjuncts teach many of the regular didactic courses at undergraduate and master's levels. As the university moves up, this will not be sustainable.  

        Now, adjuncts are much less expensive than regular faculty (tenure track, teaching, research) per course, so if we were to staff our didactic courses with regular faculty we would need a larger faculty and that would have budgetary impact.(Princeton uses regular faculty for all didactic and seminar courses, Chicago does not. Princeton's endowment/student is much larger than almost any other institution.)
        I have not addressed the issue of whether adjuncts are good instructors for these didactic courses. All the newspapers need to note is that the adjunct costs $6,000 to teach a course, and a professor teaches four courses (lets be nice and say half of time is research) and costs at least $15-30,000/course including fringes. If you don't include research time, and the newspapers won't, the disparity is much greater.
        I gather from some students there are some adjuncts who are not adequate teachers. I have no idea if this is the case. I am sure deans do the best they can. But, again, this is a potential time-bomb. Regular faculty might also be poor teachers, but then it is incumbent upon the office of the dean to have them get help.

Quality, Grades, Excellence, Compensation--vs. Working to Rule

In every course I have taught, in every class, at least one person, and usually more than one, has done excellent work. Their contribution to discussion, their papers, and their thoughtfulness are exemplary. And, usually, some students do much weaker work, often embarrassingly so--I would not want to share their papers, for example, with any colleague.

This is likely true in any institution. In any case, when I give a low grade to a student, the work itself is not good, but it is as well manifestly much less meritorious than those who do excellent work. If I give a B- to a graduate student, the work is more than deserving of such a grade. I work hard with students to avoid C's and D's, with drafts, etc, but in the end they do not deliver good work.

In a university, this is also true of faculty who are at the probationary stage. When the university tenure committee meets, some candidates stand out. They are spectacular. They've done what we would want, and done it excellently. (These are not external lateral hires, but those who were hired perhaps 6 years ago as new assistant professors.) And others, with all the help we might give them, do not deliver.

In between are all the intermediate cases. In classes, we can give intermediate grades. In tenure decisions, I have been told that if you have doubts about quality, vote NO.

There is an interesting problem when we consider compensation. If we have a bureaucratic system of salary determination, with movement between levels bureaucratically defined, then the issue does not come up much. Surely there are different scales for those in some fields-- business and law for example. The argument here is that we want to attract the best, and the comparative external salaries are much higher than we offer most faculty. (There is little consideration that perhaps teaching and research offer advantages over practice that are especially valuable.) But what is irksome is that some second-rate scholar in one of these higher-paying fields is paid vastly more than the strongest scholars in more conventional fields--often new assistant professors are paid more than distinguished full professors in the conventional fields. Perhaps the engineering and science faculties also have much higher salaries. If salaries are secret, one might not worry about this, but in general that information leaks out.

The perception of the excellence of the institution as a whole is almost always a matter of its conventional fields. So a university's first-rate history and mathematics departments, as evaluated by peers, may have faculty salaries that are much lower than their second-rate business or law schools. A provost might want to invest in conventional departments if they were concerned with the perception of excellence.

Of course there are other issues. Professors in "hot" fields are likely to be able to bid up their salaries, again their excellence as scholars being a secondary factor. Professors who work hard to generate external offers, will also be able to generate higher salaries for themselves (although the provost or dean might wish them well in their new institution!). On the other hand, professors who are doing what they really want to do, in teaching and research, might view all of what I have discussed here as interesting but missing the point. In pursuing their scholarly life, they are getting paid decently and they are getting away with murder.

Always Put Forth Your Best Work . . .

You are being judged all the time. All the time. So you want to be sure that you are showing yourself at your best. Of course, you will make mistakes, perhaps for "Freudian" reasons, perhaps for other reasons, perhaps because you are overworked or lazy. But in so far as you can do so, always put forth your best.

If you are going to submit work to your boss or teacher or a journal, be sure to read over the final version to catch any mistakes. If you are setting up a website, check that all the links work, and that they link to the right files or media. If you are giving a presentation, you want to practice and be sure what you say is both clear and within the time limit.

I have made just about every mistake one can imagine. That's why I write about these issues. I am not saying that I or anyone is a paragon here. Rather, you just want to keep in mind what you must do if you want to do something and go someplace.

Everyone knows just about everything I write about in this blog or in my other blog. I hope it is worth stating even what I am saying is a commonplace or moral cliche.

Dementors and Tormentors

As in Harry Potter: "Dementors are among the foulest creatures that walk this earth. They infest the darkest, filthiest places, they glory in decay and despair, they drain peace, hope, and happiness out of the air around them... Get too near a Dementor and every good feeling, every happy memory will be sucked out of you. If it can, the Dementor will feed on you long enough to reduce you to something like itself...soulless and evil. You will be left with nothing but the worst experiences of your life."

—Remus Lupin to Harry Potter



In any organization, some people are dangerous--

"Of course they won't let go. They are enjoying it.

"There are vicious pitbulls that attack little dogs even after the little dogs show submission, unlike normal dogs. There are coaches who want to humiliate their opponents, such as Red Auerbach's smoking a cigar in sign of victory in the 3rd quarter.  There are murderers who will stab their victims dozens or hundreds of times even after their victim stopped breathing.

"They might just be getting started. I'm telling you this not to scare you, but so you can prepare yourself mentally and withstand even worse stuff that might be coming."

Keep in mind that some people are Dementors and Tormentors (a term invented by the above speaker).

On Escalation (vs. De-escalation): Writing and Your Teacher

Many years ago, Herman Kahn wrote a book on nuclear war spelling out a series of steps on the road to war, On Escalation. I have stolen his title.

I have offered students the chance to go through their final papers, but after Commencement when things quiet down and I have time to see them.  Many of the papers were not up to par and had writing problems--from grammar and diction to organization and focus. (See my earlier post.) The only way to help students is to go through bits and pieces of such a paper and show them, then and there, how to do better. Editing in front of them works. Just editing so that what they experience is a marked up paper allows them to escape their own problems. And many final papers are never picked up by students, and rarely do they come see you afterwards, and then usually for a grade concern, not to improve their writing.

In my experience, some or many students do not like hearing that their writing is execrable. They tell me that it works on their job or others have never complained.  Perhaps they are wonderful on the job, and others have never noticed the manifest problems I find.

For whatever reason, maybe one or two or maybe a larger group believe this was a requirement and that in particular it is after the end of the semester. Perhaps my notice to them should have said, "You don't have to let me help you. Pick up your papers, and you don't ever have to see me again."  I do not hear directly from any of those students, while the large number who have no problem with seeing me about their papers just tell me when they want to see me.

Escalation: Rather, the one or two or more who take my offer as a requirement go to a staff member, and that is how I hear of their concern. And that concern is cc'd, by the staff member, to the office of the dean. The staff members has further escalated their concern to the point where the staff member is in the middle. It makes sense for the staff to send the students to the professor to see if the professor might resolve the problem.

Whatever else, the first thing to do is to see your teacher first: email, phone, office hours, an appointment. Staff have little influence on professors' behaviors; deans sometimes wish the dean had some influence (especially on tenured full professors). Moreover, this puts the staff in an unenviable position between student and instructor--what a dean does as a matter of course should not be loaded onto a staff member or another professor. [In the case of mandatory reporting, as in sexual harassment, the university provides intermediates such as the Office of Diversity, to let the staff member off the hook.] If the staff member gets directly involved at this point, the students' concerns take a backseat to bureaucratic matters.

Of course, the staff member might convey the students' concerns informally.  But once the dean or the university bureaucracy is involved the staff member is in an untenable position and the students' work is not the focus. The most effective advocate for a student is the student, unless things must rise to the attention of the office of the dean or the university (unresolved matters of fairness, grading, mandatory reporting, retaliation).

Professors, like parents, almost always have in mind the best interests of their charges.

Reading Papers for "What's Going On": Where's the Beef? Is it Horsemeat?

I have been reading student term papers. At this point, having seen earlier drafts and given my best advice, I am humbled. Whatever I say seems not to work for most students, at whatever point in their school careers.

0. Whatever you learned in composition classes gets in the way of writing effective scholarly work and surely in the way of policy memoranda. Bottom Line Up Front.

1. I want to know the main points by the end of page 2 if not page 1.  At the beginning of each section, the main point of that section should be the first sentence. See #2. What's the BIG IDEA or Ideas? Lots of introductory material often hides the big ideas.
2. I don't want to discover the point of the paper in the conclusion. It should be on page 1.
3. Executive summaries are not essays. They need to be short, assert the main points.
4. Early on it would be helpful to have a roadmap of what is to come.

5. I don't want an account of the research and reading experience. I want something that shows how it all holds together, or does not.

6. Sections need to be headed  by a paragraph that lays out the argument to follow.
7. Subheads and divisions of the paper: If you are double spacing, put the next section heading on the next double-spaced line, in the same font as the text but underlined. Don't use bold or italics. If the paper has bigger divisions, using roman numerals: I, II, III, ...

8. I want sentences that scan so that I can just read rather than decode.
9. Make lists explicit and obvious. If there are, say, seven features of a phenomenon, I expect each one's name to be in italics or maybe numbered.

9. References should be authoritative. I don't need references for minor points unless they are quite special. The references to the main sources are important. In any case, unless you are surveying a literature, where the various writers are prominent, it's best to leave their names and works to the notes and just present the various positions.
.
11. Copyedit, spellcheck, etc.
12. Don't right justify. Don't use colorful devices to set off sections of your papers. Simple and clear.

      Namely, I want to be able to read the paper and not get lost, wondering "Why am I here?" I want to be able to consider the argument or case, rather than worry about how it is put together in terms of sentences and paragraphs. (I feel that it is not fair to have students in a graduate course needing basic writing skills. I want is to engage them in their analysis and argument.)

      I have told students that once they have what they consider a decent draft, they should go back and write the first two pages or so to fulfil my #1. And each section needs an introductory paragraph (#6).

      What's wrong with me? I have never had a semester where these concerns about writing were not first and foremost. Maybe this problem has always been present, everywhere, at all institutions. I recall that Bill Leuchtenburg (Columbia and UNC-CH, American history) used to edit his students' dissertations. But I wonder how bad the writing was to start out with.

My job is to help you do better and I take you as you are, at whatever level you are. Still, it would be good to be a clear writer before you enter graduate school.

My colleague David Sloane wrote me a propos of what I said above:
       Personally I think students write better now than in the past, but that may be my selected set. As I grow more experienced (ie older), I do find that I have less patience with the types of things you point out -- but in reality I am different not them. They have always viewed papers as mystery novels where the most important point is revealed in the conclusion, and really have no idea why they should source one thing over another. I do believe that as you read more and more of these things, one forgets the students are actually at the same place they always were, we just aren't. I would be happier if they would somehow accept the learning I have gained, but I also know that they will not!
        I like your number 1, and I think you should emphasize it, but you would have to decide. Then, when you read a draft, and they don't do it, take off 5 to 20 points. I would guess they would do it the second time. But the struggle is long and hard.

         Last, I do think that one problem is that few of us ever see them twice. I once had the experience of teaching students I had had in a freshmen class a Dartmouth a senior class in history. I must say the difference was pretty spectacular, and a few of them were kind enough to suggest that I had a small part in that difference.

Stevenson/Wolfers on Statistics

Interpreting Statistical Evidence

by Alex Tabarrok on May 2, 2013 at 7:31 am in Data Source, Economics, Science | Permalink

Betsey Stevenson & Justin Wolfers offer six principles to separate lies from statistics:

1. Focus on how robust a finding is, meaning that different ways of looking at the evidence point to the same conclusion.

In Why Most Published Research Findings are False I offered a slightly different version of the same idea

Evaluate literatures not individual papers.

SWs second principle:

2. Data mavens often make a big deal of their results being statistically significant, which is a statement that it’s unlikely their findings simply reflect chance. Don’t confuse this with something actually mattering. With huge data sets, almost everything is statistically significant. On the flip side, tests of statistical significance sometimes tell us that the evidence is weak, rather than that an effect is nonexistent.

That’s correct but there is another point worth making. Tests of statistical significance are all conditional on the estimated model being the correct model. Results that should happen only 5% of the time by chance can happen much more often once we take into account model uncertainty not just parameter uncertainty.

3. Be wary of scholars using high-powered statistical techniques as a bludgeon to silence critics who are not specialists. If the author can’t explain what they’re doing in terms you can understand, then you shouldn’t be convinced.

I am mostly in agreement but SW and I are partial to natural experiments and similar methods which generally can be explained to the lay public while other econometricians (say of the Heckman school) do work that is much more difficult to follow without significant background and while being wary I also wouldn’t reject that kind of work out of hand.

4. Don’t fall into the trap of thinking about an empirical finding as “right” or “wrong.” At best, data provide an imperfect guide. Evidence should always shift your thinking on an issue; the question is how far.

Yes, be Bayesian. See Bryan Caplan’s post on the Card-Krueger minimum wage study for a nice example.

5. Don’t mistake correlation for causation.

Does anyone still do this? I know the answer is yes. I often find, however, that the opposite problem is more common among relatively sophisticated readers–they know that correlation isn’t causation but they don’t always appreciate that economists know this and have developed sophisticated approaches to disentangling the two. Most of the effort in a typical empirical paper in economics is spent on this issue.

6. Always ask “so what?” …The “so what” question is about moving beyond the internal validity of a finding to asking about its external usefulness.

Good advice although I also run across the opposite problem frequently, thinking that a study done in 2001 doesn’t tell us anything about 2013, for example.
Here, from my earlier post, are my rules for evaluating statistical studies:

1) In evaluating any study try to take into account the amount of background noise. That is, remember that the more hypotheses which are tested and the less selection which goes into choosing hypotheses the more likely it is that you are looking at noise.
2) Bigger samples are better. (But note that even big samples won’t help to solve the problems of observational studies which is a whole other problem).
3) Small effects are to be distrusted.
4) Multiple sources and types of evidence are desirable.
5) Evaluate literatures not individual papers.
6) Trust empirical papers which test other people’s theories more than empirical papers which test the author’s theory.
7) As an editor or referee, don’t reject papers that fail to reject the null.