A Good Academic Life: A Manifesto

I believe that teaching and research are synergistic activities. Excellence in one does not come at the cost of another, it improves it. Any institution short changing one is suffering a loss of intellectual capital. 

I believe that we do not have the right tools to quantitatively measure the impact of an academic. This does not mean that we shouldn’t try, but it does mean that we should take our tools less seriously.

I believe that excellence in the classroom cannot be scripted.

I believe that academic freedom means the ability to follow the path of knowledge for its own sake. We do not yet know which bits of fundamental knowledge will yield applied advances. I believe in useless knowledge.

I believe that the goals of being a good academic and being a good scientist should be in alignment. Academic freedom is the ability to do this in the face of broken incentive structures.

I believe that if you are writing a paper merely for increasing the length of your CV, you are doing it wrong.

I believe that science only works when it is open. I believe in making my papers openly available on the web, in sharing data, and treating publication as communication first, accolade second

I believe in open access. Science that is locked up behind a paywall isn’t science. 

I believe that learning is doing. Education is not “infotainment” and it’s better to demonstrate learning through projects than tests.

I believe that pride comes from hard work. Very few people can point to an “easy A” class as a life changing one. Similarly, the papers that ultimately have the most impact are those that stretch you the most.

I believe there is a difference between hard work and burnout. Burnout is antithetical to creativity.

I agree that teaching is a radical act of hope

A new professional chapter

I am thrilled to announce that starting in the fall, I will be working as an assistant professor for the Minerva Schools! I want to resurrect this blog with a post about my decision, and what it means for my career as well as for higher education more broadly.

What is Minerva? Good question. While they don’t yet have the name recognition of some of our older institutions, they aim to be "the first elite American university to be launched in a century". Starting from the question of “what does it mean to be an educated person in our time?” they designed a new type of university, stripped of unnecessary sports teams or facilities. In fact the only physical buildings are rented dormitory buildings in San Francisco and other world cities. Classes are held online, but unlike the MOOC model of video lectures, the pedagogical philosophy is geared towards fully active learning. The teaching platform allows instructors to rapidly give polls and quizzes, and to create small groups with a touch of a button. The students live together and move to a different international city each year.

Why I’m excited about this:
If you’ve read this blog before, you will know that I have been uncomfortable with a number of aspects of a Typical Academic Career. I worry about the consequences of grade inflation, and of how to maintain academic rigor without getting crushed in teaching evaluations. I think about what I call “the ever-accelerating hamster wheel” problem: our research impact is often measured in terms of number of publications, and we are putting out more work than can be read, forcing us to aggressively market our own research just to be heard through the noise. In my field, about 1/3 of papers are never cited! And we hire far more graduate students and postdocs to do this ever increasing amount of work while the number of academic positions for them is dwindling through the adjunctification of the professorship. Funding levels are so low that economists have questions whether spending time on grant proposals is even worth it.

So, is it better to change an institution from within or to blaze a different path? This is the question I have been wrestling with over the course of this job season. Academia (though not academics, generally) is conservative, and its wheels turn slowly. It’s deeply hierarchical, and can lull people with the sweet siren song of the status quo. That said, it’s what I’ve been working single-mindedly towards for the last 15 years. The argument for being at a traditional university can be best summed up by one of my mentors, who argues that while R1 life is not perfect, it's the best of the available alternatives. But what if it can be made better?

In the end, the question I kept asking myself is "what do I want out of an academic career? What makes for a good academic life?" For a long time, I've been uncomfortable with institutions that do not value excellence in teaching. Any one professor's research program, no matter how high profile, is still a small slice in the big pie of human knowledge, while the impact that one can have in the life of a student through teaching and mentorship can last a lifetime. But what about research? Am I shutting myself out from this world? I don't think so. I am testing the bold hypothesis that I can do great research outside of the normal paradigm.

Paradoxically, I think that my research might have more impact when freed from the pressures of "bean counting". The academics whose work I read most are not the ones publishing the most papers, but the ones publishing papers with the most depth of thought. I hope to maintain and develop a number of collaborations, tapping into the best minds and free from the need to be in any one location. And I am working to have a home base where I can mentor the research projects of my Minerva students, sparking the same passion for research in them as I had as an undergraduate.

Stay tuned for the future, I think it's going to be bright.

Soundbites: Presidential edition

I highly recommend this article on "animates" in the French Enlightenment. One can think of them as an early type of robot. After a couple of centuries of thinking about this stuff, you would think we would have progressed more in our thinking!


"I think the educational system has become a major factor stopping people from thinking about the future."


and


"You can’t just write checks to the thirty smartest scientists in the United States. Instead there are bureaucratic processes, and I think the politicization of science—where a lot of scientists have to write grant applications, be subject to peer review, and have to get all these people to buy in—all this has been toxic, because the skills that make a great scientist and the skills that make a great politician are radically different. " Peter Thiel

A great article on the future of cognitive enhancement in The Atlantic.

Speaking of enhancements, Slate wonders how life will change when, through drugs or engineering, our memories are perfect.

How to translate academic-ese.

Increased undergraduate debt increases the probability of attending graduate school.

Research Works Act - seriously?

I am not a fan of the academic publishing industry, and have written before on the need for more openness in the publishing process. My position is very simple: it is not ethical for taxpayers to be forced to buy access to scientific articles whose research was funded by the taxpayer.

I am very dismayed at the introduction of the Research Works Act, a piece of legislation designed to end the NIH Open Access policy and other future openness initiatives.

Sigh... even in academic publishing, we're socializing the risks and privatizing the gains. Here, I agree completely with Michael Eisen's statement in the New York Times:
 "But the latest effort to overturn the N.I.H.’s public access policy should dispel any remaining illusions that commercial publishers are serving the interests of the scientific community and public."

As this bill was written by representatives taking money from the publishing industry, perhaps we should include lawmakers in that group as well.

Soundbites: Surfin' Santa edition

Cool tool: Cross Validation is the Stack Overflow for statistics questions.

The utility of using neuroscience to understand art (e.g. "neuroaesthetics") is considered.

Cool: the Wounded Warrior Bill mandates cognitive testing of soliders before and after deployment. This can help screen for brain injuries and (in theory) get wounded soliders needed treatment. Not cool: this is not actually happening.

Ireland is considering a proposal to add lithium to the drinking water as a method for reducing crime. Lithium is a standard treatment for bipolar disorder. The proposal cites studies in Texas and Japan showing reduced crime in locations where lithium is present in drinking water (though it's not clear whether it was added on purpose, like fluoride).

Mind Hacks follows a new amendment to the US Controlled Substances act, adding a number of chemically synthesized cannabinoids.

"...the raison d’être of a college is to nourish a world of intellectual culture; that is, a world of ideas, dedicated to what we can know scientifically, understand humanistically, or express artistically. "


A beautiful critique of statistical cut-corners in the Freakonomics empire.


The Neurocritic discusses an interesting case: a child with a malformation in the prefrontal cortex and extreme behavioral problems. Can we assign a causal relation?

Who takes the responsibility for quality higher education?

This gives me chills: a professor denied tenure for using the Socratic method of teaching. Of course, there are two sides to every story and this article is rather one sided - I have been in classes where so-called Socratic methods are thinly veiled excuses for hurling insults at students - but if we are to take the article at face value, this is another story in a disturbing educational trend.

The Socratic method is challenging for students and requires preparation and engagement with the material. It requires being able to effectively communicate under pressure. However, I feel that learning involves a certain amount of discomfort. Learning means pushing past the boundaries of what we already know, and what we can already do. Most undergraduate courses I took were of lecture-style, teaching students to expect to be a passive audience in class. It's a much easier route and the student can hide lack of preparation, misunderstanding or having a bad day. However, these students cannot hide forever, and this under-preparation often comes back to haunt them at exam time.

As a TA in graduate school, I saw many freshmen having harsh wake-up calls when the first midterms came back. The typical story was "But I came to all the classes, and I read the book chapters twice! How could I have gotten a C on the exam???" The unfortunate answer is that the student mistakes being able to parrot back a section of textbook or lecture for understanding the material. When an exam forces the student to use this information in an analytic or synthetic way, the facade of learning crumbles. 

I don't know any instructor who wants to give a student a poor grade, but the integrity of the educational system depends on accurate assessment of mastery. If an instructor is fired, demoted or denied tenure due to the rigors of his/her course, this could spell the end of education. Sadly, this story is reminiscent of this case: a professor denied tenure for not passing enough students. I highly recommend reading this page because, if we are to take the author at his words, he took every reasonable action to enable his students to succeed.

Who is responsible for student success in higher education? Professors, of course need to be responsible for presenting learning opportunities to students in a clear manner, and to be available for advice and guidance at office hours. However, university students are adults and need to take responsibility for the ultimate learning outcomes. I am concerned by a culture of entitlement that has conditioned students to expect top marks for simply showing up. The expectations of the "self-esteem generation" and the incentives of professors to earn high student evaluations both play a role, I suspect.

I wonder sometimes whether the cost of attendance at American colleges and universities partially drives this phenomenon. Paying for education turns students and their families into customers, and "customers are always right". Perhaps subsidizing higher education would create a culture that divorces education from "service", leading to more honest evaluations and better learning.

Soundbites: Thanksgiving edition

It was my birthday this week. Turns out my brain is just about done maturing. (Tell that to my behavior...)

Not really news: tenured faculty members bring in twice as much money to universities as they cost.

Funny titles for scientific studies.

You know how I'm often griping about the lack of published negative results and replication attempts? Some people are trying to solve both. Very cool.

From Nature: improving understanding of statistical arguments.

EEG can be used to predict conscious awareness in vegetative patients.

Wanna take some computer science courses at Stanford? Several courses will be open to the public virtually next quarter!

The results of another self-reported survey on the use of cognitive enhancing drugs.

Soundbites: Spooky edition

Should we give all surgeons cognitive enhancers to improve performance?

The "smart phone brain scanner". This could either be the best thing ever, or the worst thing.... I still have not decided.

The Royal Society has made 60,000 articles freely accessible. W00t!

... happy birthday, dear fMRI. Happy 20th birthday to you!

New ADHD guidelines allow for diagnosis in children as young as 4. (Which to me begs the question of what a "normal" 4 year old is supposed to act like).

What counts as a person in an era where some states are proposing laws to define a fertilized egg as a person? Neuroethics Canada has some intelligent commentary.

Is the academic publishing industry evil?

Like most people, I didn't think much about the profit model for academic journals until I was publishing in them. Even after going through the process a few times, I am still struck by a feeling that academic journals are the toll trolls on the road of knowledge dissemination.

While a non-academic journal such as The Atlantic or the New Yorker pays its authors for content, academic journals get massive amounts of content volunteered to them. While non-academic journals pay an editor to hone and perfect the content, academic journals have volunteer peer reviewers and volunteer action editors doing this work for the cost of a line on the academic CV. Both types of journals offset some publication costs with advertising, but while non-academic journals sell for ~$5 per issue and under $50 for a year's subscription, an academic journal will charge $30-40 per article and thousands for a subscription. This means that the tax payer who funds this research is not able to afford to read the research.

Let's say you're an author, and you're submitting your article to a scientific journal. It gets reviewed and edited, and is accepted for publication by the action editor. Great! Your excitement gets diminished somewhat from two documents that get sent to you: one that signs over your copyright to the journal, and a publishing bill based on the number of pages and color figures in your work (often a few hundred dollars). Now, if you want to use a figure from this article again (say, for your doctoral dissertation), you must write the journal to get permission to use your own figure. Seriously. Other points against academic journals can be found in this entertainingly inflammatory piece.

But what about open access journals? Good question. These journals exist online, and anyone can read them, which is great for small libraries struggling to afford journal costs and citizens wishing to check claims at the source. They're not so great for the academic, who gets slapped with a $1000-2000 fee for publishing in them. As inexpensive as online infrastructure is these days, I would love for someone to explain to me how it costs the journal so much just to host a paper.

I was excited to read this interview with academic publishers Wiley and Elsevier on these issues. However, I find most of the responses to be non-answer run-arounds. A telling exception to this is in the first question "what is your position on Open Access databases?". Wiley responded:

"The decision to submit a manuscript for publication in a peer-review journal reflects the researcher’s desire to obtain credentialing for the work described. The publishing process, from peer review through distribution and enabling discovery, adds value, which is manifest in the final version of the article and formally validates the research and the researcher."

(Emphasis mine).
In other words, we do this because there is a demand for our journal as a brand. You, researcher are creating the demand. However, I do hold out hope that as more publishing moves online, more researchers and librarians realize that there are both diamonds and rough in all journals, and this will wear away at brand prestige, allowing the illusion of "publisher added value" to wear away.

Insula-gate

For those just tuning into this week's latest installment of NeuroNonsense brought to you by the New York Times, let me being you up to date:

The New York Times allowed (nonscientist) Martin Lindstrom to once again use its Op-Ed space to "publish" non-peer reviewed "science".

Scientists, disgusted struck out at this perversion of science throughout the blogosphere (here, here, here, here though I'm sure I'm missing others). Dozens of prominent cognitive neuroscientists wrote a counter op-ed denouncing this practice (heavily edited by NYT staff).

At work the other day, a graduate student asked me why our field has a lower bar for press shenanigans and wildly implausible claims. I think there are several possible answers to this question (the fact that folk psychology seems to provide causally satisfactory explanations, the allure of pretty pictures of the brain "lighting up", or the intrinsic interest people take in their own brains all come to mind easily. However, I'm afraid that there's also a capitalist component to this one as well: Lindstrom makes his money convincing companies that his "science" will lead to better marketing outcomes. I can't think of a single case where someone impersonates a particle physicist or an inorganic chemist to sell snake oil.

The interest people take in their brains unfortunately creates this market for NeuroNonsense.

Soundbites, profoundly late

The Economist describes a fascinating new study that shows that people attribute less mind to vegetative patients than to dead persons.

Nature takes on the issue of work-life balance from both sides.

Over at The Atlantic, a nice piece on how deeply held cultural beliefs can kill.

Some hard numbers on prescription drug use for attention deficit hyperactivity disorder.

Jonah Lehrer provacatively asks "is corporate research more reliable than academic research?"

Interesting interview on the future of psychoactive pharmaceuticals.

Not the new truth serum.

Magnetic Pulses to the Brain Make it Impossible to Lie: Study

Zapping the brain with magnets makes it IMPOSSIBLE to lie, claim scientists

Holy crap! Hold on to your civil liberties...get your tin foil hat.... Something really exciting must be going on in neuroscience.

Right?

So it turns out that these articles refer to the following study:

Here, participants were shown red and blue circles and asked to name the color of the circle. At will, the participant could choose to lie or tell the truth about the color of the circle. However, while they were performing this task, repeated transcranial magnetic stimulation (rTMS) was applied to one of four brain areas (right or left dorsolateral prefrontal cortex (DLPFC), or right or left parietal cortex (PC)). TMS produces a transient magnetic field that produces electrical activity in the brain. As it is causing the brain to have different firing behavior, TMS allows researchers to gain insight into how certain brain areas cause behavior. Previously, the dorsolateral prefrontal cortex has been implicated in generated lies. Here, the authors sought to assess whether this area has a causal role in deciding whether or not to tell a lie. Here, the parietal cortex served as a control area as it is not generally implicated in the generation of lies.

So, is TMS to DLPFC the new truth serum? Here, I've re-plotted their results:

When TMS was applied to the left DFPFC (compared with the left PC), participants were less likely to choose to tell the truth whereas they were slightly more likely to be truthful when stimulation was applied to the right DLPFC. As you can see from the graph, the effect, although significant, is pretty tiny. The stimulation changes your propensity to lie or truth-tell about 5% in either direction. This cannot be farther from the "impossible to lie" headlines.

Interesting? Yes. Useful for law enforcement? Probably not.

Karton I, & Bachmann T (2011). Effect of prefrontal transcranial magnetic stimulation on spontaneous truth-telling. Behavioural brain research, 225 (1), 209-14 PMID: 21807030

Sunday soundbites: 8-21

Over at Rationally Speaking, there is a proposal to move towards using odds instead of probabilities when speaking of uncertainty.

I highly recommend this lucid discussion of the brain, free will and criminal responsibility from David Eagleman in The Atlantic.

Running experiments is sometimes a pain, but I'm not sure what to think of this site where one can outsource one's experiments. It's called 'EBay for science', but I worry about how one would trust that the experiments were done well.

What does your writing say about you? Quite a bit, it turns out.

Predictive policing - using statistics to determine the times and locations of future crimes.

Bayesian truth serum, grading and student evaluations

In one of my last posts, I examined some proposals for making university grading more equitable and less prone to grade inflation. Currently, professors are motivated to inflate grades because high grades correlate with high student evaluations, and these are often the only metrics of teaching effectiveness available. Is there a way to assess professors' teaching abilities independent of the subjective views of students? Similarly, is there a way to get students to provide more objective evaluation responses?

It turns out that one technique may be able to do both. Drazen Prelec, a behavioral economist at MIT, has a very interesting proposal for motivating a person to give truthful opinions in face of knowledge that his opinion is a minority view. In this technique, awesomely named "Bayesian truth serum"*, people give two pieces of information: the first is their honest opinion on the issue at hand, and the second is an estimate of how the respondent thinks other people will answer the first question.

How can this method tell if you are giving a truthful response? The algorithm assigns more points to responses to answers that are "surprisingly common", that is, answers that are more common that collectively predicted. For example, let's say you are being asked about which political candidate you support. A candidate who is chosen (in the first question) by 10% of the respondents, but only predicted as being chosen (the second question) by 5% of the respondents is a surprisingly common answer. This technique gets more true opinions because it is believed that people systematically believe that their own views are unique, and hence will underestimate the degree to which other people will predict their own true views.

But, you might reasonably say, people also believe that they represent reasonable and popular views. They are narcissists and believe that people will tend to believe what they themselves believe. It turns out that this is a corollary to the Bayesian truth serum. Let's say that you are evaluating beer (as I like to do), and let's also say that you're a big fan of Coors (I don't know why you would be, but for the sake of argument....) As a lover of Coors, you believe that most people like Coors, but feel you also recognize that you like Coors more than most people. Therefore, you adjust your actual estimate of Coors' popularity according to this belief, therefore underestimating the popularity of Coors in the population.

It also turns out that this same method can be used to identify experts. It turns out that people who have more meta-knowledge are also the people who provide the most reliable, unbiased ratings. Let's again go back to the beer tasting example. Let's say that there are certain characteristics of beer that might taste very good, but show poor beer brewing technique, say a lot of sweetness. Conversely, there can be some properties of a beer that are normal for a particular process, but seem strange to a novice, such as yeast sediment. An expert will know that too much sweetness is bad and the sediment is fine, and will also know that a novice won't know this. Hence, while the novice will believe that most people agree with his opinion, the expert will accurately predict the novice opinion.

So, what does this all have to do with grades and grade inflation? Glad you asked. Here, I propose two independent uses of BTS to help the grading problem:

1. Student work is evaluated by multiple graders, and the grade the student gets is the "surprisingly common" answer. This motivates graders to be more objective about the piece of work. We can also find the graders who are most expert by sorting them according to meta-knowledge. Of course, this is throwing more resources after grading in an already strained system.

2. When students evaluate the professor, they are also given BTS in an attempt to elicit an objective evaluation.

* When I become a rock star, this will be my band name.

Soundbites!

Mmmmm.... useful math.

It's unfortunately easy to get people to falsely confess to crimes, reviewed in The Economist.

Retractions are up in scientific journals. Now there is a blog devoted to 'em.

Completely awesome matrix of how people on various rungs of the scientific ladder view one another.

Only 4% of the public can name a living scientist. Time for a scientific re-branding.

A solution to grade inflation?

In the somewhat limited teaching experience I've had, I have found grading to be particularly difficult. The grade a student receives in my class can determine whether he'll get or keep scholarships and will play a role in determining what kinds of opportunities he'll have after my class. This is a huge responsibility. As a psychophysicist, I worry about my grade-regrade reliability (will I grade the same paper the same way twice), order effects in my grading (if I read a particularly good paper, do all papers after it seem to not measure up?), and whether personal bias is affecting my scoring (Sally is always attentive and asks good questions in class, while Jane, if present, is pugnacious and disruptive).

Of course, the easiest thing is to give everyone generally good grades. The students won't argue that they don't deserve them, and in fact, there is evidence that they'll evaluate me better for it in the end.



And while many institutions have (implicitly or explicitly) adopted this strategy, the problem with grade inflation is that it hurts students who are performing at the top level, and removes accountability from our educational system. So, what do we do about grading?



The Chronicle of Higher Education has an interesting article showing two possible solutions. The second solution involves AI-based grading, which sounds intriguing. Unfortunately, no details were provided for how (or how well) it works, so I remain skeptical. However, the first proposed solution merits some discussion: outsource grading to adjunct professors who are independent of the course, professor and students. The article follows an online university that has enacted this strategy.


Pros of this idea:
- As the grader is not attached to either the professor or the student, bias based on personal feelings towards a student can be eliminated.
- In this instantiation, graders are required to submit detailed justifications for their grades, are provided extensive training and are periodically calibrated for consistency. This can provide far more objective grading than what we do in the traditional classroom.


However, the idea is not perfect. Here are some cons that I see:
- The graders' grades get translated into pass or fail. A pass/fail system does not encourage excellence, original thinking, or going beyond the material given.
- Much of traditional grading is based on improvement and growth over a semester, and this is necessarily absent in this system. Honestly, I only passed the second semester of introductory chemistry in college (after failing the first test) because the professor made an agreement with me that if I improved on subsequent tests, she would drop the first grade.
- Similarly, the relationship between professor and student is made personal through individualized feedback on assignments. Outsourcing grading means that there cannot be a deep, intellectual relationship between parties, which I believe is essential to learning and personal growth.


While not perfect, this is an interesting idea. What are your ideas for improving on it (or grading in general)?

What is it about music?

I can't get enough of this song. I saw Gillian perform this in 2005 or 2006, and I'm so happy that she's finally put it on an album. Why have I played this song over 20 times in one day? Of course, there are technical aspects of it that are neat (I particularly like the frequent dissonances that resolve in Rawlings' guitar line), and the lyrics remind me of the time in my life when I first heard the song, but are these alone enough to produce such strong emotional reactions? Why does music give us chills? Why does it freak us out?

Indeed, music seems to activate the neural reward system, and certainly there are no lack of hand-wavy evolutionary psychology theories on music's emotional pull. But why does music make us feel things?

We have several touch/feeling metaphors for music: a person's voice can be "rough" or "velvet", musical passages may be "light" or "heavy, and pitches can be "rising" or "falling". Given this mapping, can we find cross-modal effects of music and feeling? One type of cross-modal effect is synaesthesia, where two senses are correlated in the same person. To a synaesthete, letters can have color, tastes can have shape, etc. The most common form of synaesthesia related to music is "colored music". Can we find evidence for "touched music"?

This paper is the reason I would love to be a psychologist in the 1950s. Here, the goal was to see which combinations of senses could be combined in synaesthesia, either in naturally occurring synaesthesia or in (I can't make this up) mescaline-induced synaesthesia. Here is the summary matrix of their results:


(An "N" in a cell represents a naturally occurring synaesthesia, and an "E" in a cell represents an "experimentally induced" synaestesia through mescaline).

So, who were the participants in this study?? The two authors and their two friends, one of whom was a natural music-tactile synaesthete. (See, I told you psychology was fun in the 1950s!)

This synaesthete described her experiences: "A trumpet sound is the feel of some sort of plastics; like touching certain sorts of stiffish plastic cloth - smooth and shiny - I felt it slipping." Sounds interesting, but does not sound like the "chilling", emotional experience.

It turns out that musical chills, while having a strong physiological basis, are not automatic but rather require attention. This implies that music and somatosensory (touch) systems are not necessarily linked.

So, no real answers in this post, but there's one other cool piece of data that I'll throw into the mix.  So, if you ask people to assign colors to both emotions and music, people (normal, non-synaesthetes) are ridiculously similar in the colors chosen. Of course, this was presented at a conference and is not in final, peer-reviewed form, but it is certainly interesting.