The first time I heard the term "experimental philosophy," part way through my master's degree, it sounded like such an obvious oxymoron that I couldn't help but think it was a terrible idea. I shared, and continue to share, many of the worries that Seth pointed to in his recent post on the subject. Although I can certainly see the appeal in having some kind of respectable yardstick against which to measure any philosopher's intuitions (I share Dan Dennett's suspicion of arguments overly dependent on the word 'surely'), I think that many have probably underestimated the difficulty in obtaining empirical evidence actually capable of serving that purpose.
Some of the reasons why I think this go right to the heart of the theory and practice of the scientific method. Even the most tightly controlled experimental results admit of multiple interpretations, and the nature of the subject matter of much x-phi means that the highest standards of scientific enquiry simply cannot be met. Suppose we are running an experiment to test the assumption that people, in general, are moral objectivists. It turns out that, when given a certain scenario, people respond in the way that the designers of the experiment intended to indicate that they are moral objectivists. So what? Does this mean that most people are moral objectivists, or only think they are? What kind of moral objectivist are they? One of the implausible kinds, perhaps? Are people intuitively intuitionists for instance? Does this make intuitionism more plausible or just show that most people are in the grip of an implausible theory? It is no shame on x-phi that no experiment will be able to give us a watertight answer to whether the assumption they set out to test is true. What is more likely, as with the legion x-phi insights on various "trolley problems," is that the results from x-phi will force philosophers into potentially interesting theoretical acrobatics in order to accommodate the results into theories that they already prefer for other reasons.
I can confidently predict that no x-phi result will ever, by itself, settle a significant philosophical debate, but you know what? I think that's fine. I think it's fine from the point of view of armchair philosophy, because the theoretical acrobatics I just mentioned are a vitally important part of philosophical practice, and the more things that force us into engage in them the better. I also think it's fine for x-phi. This is partly because I think conducting intuition-hunting experiments are valuable to philosophy for the reasons I just mentioned, but also because I really don't think experimental philosophers are trying to do anything more. X-phi sceptics frequently get all hot under the collar about the supposed claims of x-phi that it can usurp armchair philosophy. That x-phi-ers somehow think that the right survey, with the right population, under the right conditions, will tell us once and for all whether or not we should pull the trolley-switching lever, or whether we survive being teleported, or whatever the hell else. Not only is this clearly rubbish, but it is highly doubtful that any philosopher, experimental or otherwise, thinks any such thing.
Those who say that x-phi has got too big for its boots fall into two camps. Those who think that x-phi has pretensions to do away with armchair philosophy, or instill itself as the new dominant philosophical method, and those who think that it cannot even do the intuition-hunting that so much of it claims to. The former camp are, to my mind, scared of a straw man, though I don't think there are as many of them around as the blogosphere (on both sides of the x-phi debate) would have us believe. No-one but the most foamy-mouthed of hardcore intuitionists would so much as hint at the claim that "if most people think that P, then P is true." The worry that x-phi cannot even reliably source intuitions is more problematic for those philosophers who embrace it, but I don't think this is a criticism that holds against all x-phi necessarily, rather it is a reminder that philosophers might perhaps not be the best people at designing experiments. Maybe we'll get better at determining which questions can be answered using x-phi and which can't, at how to isolate which variables and how to formulate a null hypothesis.
Or maybe these are problems distinctive of the kind of x-phi we are most used to reading about: the kind that asks people what they think of a morally tricky situation. Should we kill one to save five? Did the CEO intentionally hurt the environment? etc. etc. If what we're testing for is intuitions on intractable philosophical issues, we shouldn't expect the answers to be much less difficult to interpret than the questions themselves. Interestingly, and thankfully, intuitions about difficult problems are far from the only things that x-phi can test that might be useful to philosophers generally.
What things besides intuitions can x-phi help us establish? One piece of experimental philosophy recently reported in the British Journal of Aesthetics, Meskin et. al's "Mere Exposure to Bad Art" (the article is behind a paywall, but the piece is nicely summarised here), attempts to establish not intuitions but preferences: specifically, preferences for artworks.
The problem the paper seeks to address is this: studies have shown that "mere exposure" to particular artworks leads to increased preference for them over others to which one has not been exposed. One implication of this has been taken to be that the canon of great western art is maintained not by virtue of any special artistic merit, but rather just because those canonical works are the ones to which we are most exposed. Is the main reason The Beatles will live on in the canon of western art but New People probably won't that we all get played The Beatles a hell of a lot more than we hear New People? If so, this poses a problem for philosophers of art, since so much of our work is predicated on the fact that there are real artistic and aesthetic qualities that works have, and by virtue of which they can be judged as better or worse works of their kind.
Meskin and co called this contention into question with their study by showing that the "mere exposure" effect was not the same in all cases. Students were "merely exposed" to some good art (Millais landscapes), and some bad art (Kinkade "cottage porn"), in tightly controlled, randomised experimental conditions. It was found that, far from the exposure having a positive effect on the viewers for Kinkade's work, after only one exposure there was a statistically significant drop in preferences for his daubings. So, the paper claims, we can't maintain the claim that "mere exposure" is the principal determining factor in our artistic judgments, or at least the principal determining factor in what is included in the canon of great western art.
What's interesting about this paper as an example of x-phi is that, in contrast to the intuition hunting projects we see discussed more frequently, the result here can't even be mistaken for an attempt to answer the philosophically interesting question ("what determines our artistic preferences?") with an experiment. Rather, this experiment is making room for philosophy to happen. This is similar to the role I suggested that ethics x-phi actually plays in philosophical discourse - it forces proponents of certain views into interesting acrobatics - but in this case the experiment allows for the acrobatics to happen at all, rather than dictating the size of the trampoline we have to bounce on (if I might torture the metaphor a little). This experiment shows not that a certain philosophical position is the most widely held, but rather that a philosophical discussion of the subject is a worthwhile pursuit.
I guess the final thought is this: X-Phi is great if it is making space for philosophy to happen, which is what most x-phi does in actuality (even if not in intent). In many ways it's a shame that the most prominent examples of x-phi have to do with testing intuitions on philosophical problems. When the things being tested for are precisely those things at issue there's bound to be some deep design and interpretation problems to be uncovered in analysing the results. However, when our x-phi is designed to test not "folk" intuitions about philosophical problems, but philosophically significant features of the "folk" that are not in themselves philosophical views (like their artistic preferences in certain conditions, for instance), the data gathered is guaranteed to be far less mysterious, and probably far more useful.
Image Note: The image is a Thomas Kinkade parody found here, referred to there as "Kinkade Road" by Akadajet.
We spent a little time making fun of Kinkade back on ep. 28.