Over at Ecology for a Crowded Planet, Philip Martin wrote a nice summary of a great workshop we both attended a few weeks ago. The workshop was about using new methods to increase the accuracy of expert judgements and was presented by Mark Burgman. Please check out the original post for some context.

In the comments of that post, I made a remark on the potential utility of adapting Burgman’s methods in order to get a more accurate representation of self-reported values when doing sociological studies in conservation biology. I was not as clear as I should have been, so I’ve decided to expand my comment into a more detailed post.

First some background. My initial idea was sparked by something written by Malcolm Gladwell in his book, Blink. The video clip  briefly captures his main message (EDIT: Skip ahead to 09:40 – 11:15 for the main message; or if you can download the full version of this talk).

In short, Gladwell reports that traditional market researchers have found that survey-based assessments are poor indicators of peoples buying behaviour. This is because our value-judgements expressed in surveys come straight from our thinking, logical neocortex. Our actual behaviours, however, are due to a muddled combination of our conscious and sub-conscious thinking processes. In short what we say we like, and what we really like might not always be the same thing.

Conservation biology can learn from this. In the past few decades, conservation biology has embraced the social sciences because most of us understand that any conservation action will be jeopardised unless it has the collective buy-in of local stakeholders. So, in order to assess the values and perceptions of local stakeholder, researchers might let them complete questionnaires to measure their values in an attempt forecast their responses to potential changes. But these kinds of self-reporting methods can potentially suffer from the same biases as the market research. Maybe we can reduce these biases by modifying tools designed for group decision making?

(Click on image to see view higher resolution)

For example, a local community might have a tradition of subsisting off the gathering of firewood and trapping of small antelope in a patch of forest. A large logging company might want to purchase adjacent land for their timber works. A researcher might ask the local community this: “On a scale of increasing likelihood from 0 to 10, how likely are you to sell your patch of land to the logging company?” Assuming that Participant A answered three  – this would represent that person’s logical appraisal of his own behaviour (Red point on the left). But we cannot be certain that selling behaviour is primarily logical, because many unrecognised and subconscious aspects also determine our actions.

Here is where we could modify Burgman’s methods to get a more-accurate value judgement.

In a second step, the researcher can ask the same member of the local community, “On a scale of increasing likelihood from 0 to 10, how likely are other people in your community to sell their patch of land to the logging company?” Let us assume that Participant A responded that the community-likelihood was seven (Red point on the right). This value has two aspects: first, it is determined by the actual likelihood of a community response and, second, it also captures the cognitive bias of the respondent who might be projecting his own subconscious values onto other members of the community.

These two-part questions are then asked to the entire community and, let’s assume, it shows that the community-averaged response to the second question is 5.5 (the likelihood that others will sell; blue dot). Because the community was valuing the same thing, we can assume that this average is a good estimate of reality (if the sample size is large enough) and that the difference between the individual responses and the average is due to projection biases of the individual respondents. We can then modify the original response by Participant A to the first question by incorporating the sub-conscious projection bias (this correction can even include a weighing factor if necessary).

I realise that this whole post is very speculative and that it is based on the very weak assumption that group appraisal is less biased than personal appraisal, but I still think it is an interesting way to try and improve the accuracy of sociological assessments. Furthermore, I am sure that social scientists have been hard at work testing similar methods, so I am very eager to hear the opinions of someone who is more knowledgeable than I am.