Mekong Region Futures Institute
John Ward and Alex Smajgl
Leads of MK28: Implementing cross-sectoral negotiations to coordinate Nam Xong livelihoods, ecosystem services and agricultural intensification
What is your race?
The immigration officer processing my visa application had just questioned my answer to “what is your race”. I had written “scientist,” believing this to be a valid descriptor, although grandfather or bloke would have been equally apt. Scientist was hastily redacted and replaced with “Caucasian”. The visa episode sparked my curiosity: how is it that I can be bureaucratically described as Caucasian, a term derived from the Caucasus, the remote mountainous region of Azerbaijan and Georgia? Recollections of a Stephen Jay Gould essay “The Mis-measure of Man” (1996) and some additional inquiry uncovered Christoph Meiners and Baron Johann Friedrich Blumenbach, both proto taxonomists of the late 18th Century as the source of the racial classification. Meiners coined the term Caucasian based solely on the aesthetic observation that the Caucasian peoples had pale skin, compared to the Mongolian race he had also defined in a racial typology. Caucasian was later expanded to typify a more sensitive, beautiful, morally virtuous and intelligent person. As a contemporary of Linnaeus and living in the epoch of classification, Blumenbach expanded the racial neotypes to five and gave the terms scientific credibility by grounding them in the embryonic disciplines of taxonomy and craniology.
This anecdotal episode is an enduring example from the family of simplifying abstractions and approximations of the world that we necessarily employ in the practice of day to day living. We cannot think without abstractions. The rules of thumb, classifications, social norms, conventions, rituals, customs, beliefs, values, habits, theories, ideologies and dogmas (and more recently memes) that guide and structure our daily lives both as individuals and members of civil society. We all think in abstractions and necessarily sample, classify and simplify a complex world according to our biological, cultural and historical pedigrees. Through the accretion of personal experience and shared knowledge, the majority of abstractions enable us to simplify our decision making, develop a semblance of social coherence and minimize the costs of information processing that help us avoid cognitive paralysis.
The body of knowledge we call science is also not immune to abstractions. Like all bodies of knowledge, science is a constellation of approximations of truth/reality with one essential and primary distinction: the scientific pattern of enquiry is predicated on falsification. Scientific claims must be testable; we must, in principal, be able to envision a set of observations that would render them false. Some abstractions, however, are less benign and impervious to change, such as Blumenbach’s racial archetypes and the traumatic interpretations that have manifested from these, such as discrimination, bigotry and immigration quotas. These require a more robust challenge and rigorous scrutiny.
In 1987 Carl Sagan stated that “In science it often happens that scientists say, ‘You know that’s a really good argument; my position is mistaken,’ and then they would actually change their minds and you never hear that old view from them again. They really do it. It doesn’t happen as often as it should, because scientists are human and change is sometimes painful. But it happens every day. I cannot recall the last time something like that happened in politics or religion”. Sagan suggests that as a science community we are more inclined to accept new critiqued knowledge and explanation and amend our beliefs accordingly. However, Stephen Jay Gould reminds both the science and policy maker communities that “the most erroneous stories are those we think we know best – and therefore never scrutinize or question” (Gould, 1996 b).
Where is the evidence for evidence-based policy?
For our own projects with WLE Greater Mekong, one possibility of scientific assumption comes in the form of the concept of evidence based policy and its sub species, research-for-development. We are guided by various theories of change that articulate clearly defined pathways to policy outcomes and beneficiary impacts where policy decisions can be attributable to associated research and science knowledge. After many years’ experience, the common mantra of the frustrated scientist (and policy maker) has been “why doesn’t the policy reflect the research results we were commissioned to conduct”. We examine this conundrum a little further on.
Attribution may be feasible with uncomplicated programs typified by relatively certain and accepted knowledge, single agency decisions, clearly defined, standardized research methods and analysis and general consensus amongst affected people. The underlying premise is that relevant scientific knowledge readily and passively diffuses into policy decision making: “good” science impels “good” policy. It is unlikely projects with this set of characteristics are currently being funded by WLE.
Our WLE Nam Xong project (and the majority of WLE projects) is characterized by a high degree of factual uncertainty, decisions concern multiple agencies with competing and conflicting objectives and a general lack of agreed analytical resolutions. We argue that claims of attribution become increasingly speculative when we are faced with this set of circumstances and there is a general paucity of evidence supporting evidence based policy. It is more likely that there are multiple factors of attribution (mostly unobserved) and modest incremental changes observed during the project lifecycle, will yield more substantial post project dividends and outcomes. If we apply the principle of falsification, what set of observations would we use when the evidence (or the paucity of evidence) suggests there is unlikely to be evidence to support evidence based natural resource policy???
Following Einstein’s maxim that “everything should be made as simple as possible, but not simpler”, we can simplify our problem of evidence based policy into two main components: evidence (here we mean WLE science outputs) and natural resource based policy.
Science outputs as Evidence
Richard Feynman (1998) nominates explanation as the primary and most important yield of science (see Van den Hove 2007 for an incisive review and Smajgl and Ward 2013, 2015): that is, science systematically produces controllable knowledge, classified to “explain whatever strikes us of being in need of explanation” (Popper 1989). Our proposition is this: science explanation challenges firmly held causal beliefs (or heuristics) currently held by policy decision makers, which initiates a cycle of learning which potentially yields a modified set of beliefs aligned with the explanation. Heuristics are simplifications or rules of thumb to help understand the dynamics of complex systems, in our case the cross sectoral interactions of policies to manage the Nam Xong River basin. These rules of thumb are also the most erroneous stories that we think we know best that are rarely challenged and receive limited scrutiny.
Richard Feynman (1955), “Of course if we make good things, it is not only to the credit of science; it is also to the credit of the moral choice which led us to good work. Scientific knowledge is an enabling power to do either good or bad — but it does not carry instructions on how to use it. Such power has evident value — even though the power may be negated by what one does with it”. The fate of scientific knowledge is often determined by a prescribed set of instructions that guide decision makers; and in the domain of sustainable development the instruction set often includes ethical, sectoral and political claims that often trump scientific knowledge and explanation.
Winter (1966) proposes that “policy has to do with man’s problems with coping with his future …..policy brings to statement what is judged to be possible, desirable and meaningful for the human enterprise. In this sense policy is the nexus of fact, value and ultimate meaning in which scientific, ethical and theological-philosophical reflections meet.” In practice, however Clay and Schaffer (1984) propose: “the whole life of policy is a chaos of purposes and accident. It is not at all a matter of the rational implementation of the so-called decisions through selected strategies”. Our observations suggest decision making for the Nam Xong is an amalgam of both.
We propose that the presentation of science knowledge into cross sectoral and complex decision making requires 1) a systematic method to challenge and reconstruct learning by decision makers and 2) metrics to detect learning. Based on a vast cognitive and social psychology literature we use life guiding values, causal beliefs and attitudes as our learning psychometrics (see Smajgl and Ward 2013, 2015, Smajgl et al. 2014).
We did not present any new science evidence in the first two of the Nam Xong workshop series.
Decision makers from Lao PDR agencies were invited to identify and prioritize the most plausible and important future changes in the Nam Xong sub-basin, as well as the likely policy responses to meet those challenges and the important indicators of success in tackling those challenges. We formulated a co-designed research programme based on the workshop outcomes. The same set of participants in the second workshop constructed a future vision for the Nam Xong; a shared vision that addressed the multiple and competing objectives of agencies, not just single sector objectives and mandates. They articulated the social, economic, environmental and policy characteristics (and the metrics of success) of a desirable future, and those of a likely and an undesirable future. The visions created for the Nam Xong are the topic for our next blog piece.
In the two workshops we measured five different aspects of life guiding values by having participants fill out surveys before and after the workshops. The questions reveal the importance individuals place on environmental preservation, individual fulfilment, altruism and concern for others, preservation of current social structures, and openness to change. The expectation was that detected changes in the orientation of life guiding values indicates a high probability of changed casual beliefs and learning.
In the absence of science evidence we argue it is reasonable to expect that without learning, the inertia of prevailing beliefs and heuristics will result in unmodified values, decision making and policy outcomes. Using the values psychometrics only we proposed two Null hypotheses:
- Null Hypothesis 1: in the absence of challenging explanations the value orientations held by decision makers will be stable and unchanged.
- Null Hypothesis 2: the values orientations of policy makers (in the absence of challenging explanations) are not significantly different to those held by individual household members in the Nam Xong.
What we found during the first two workshops:
- Without the introduction of science evidence we did not detect learning amongst the participating decision makers. No evidence that participating in the inception and future visioning workshops resulted in altered personal values, and unlikely that changes would occur in beliefs and decision making behaviours (see Foran et al. 2013 for details on the future visioning process).
- Based on our previous work in the Mekong (Smajgl and Ward 2013 and 2015) we expect systems learning will occur during our next round of workshops and we will observe value and belief changes when science evidence is presented as a device to challenge existing beliefs. That will be a subject of future blog.
- The decision makers did not have value orientations representative of their household constituents, instead they typically displayed higher values than individual household members. Thus we reject Null hypothesis 2.
We can summarize our argument thus: We necessarily think in abstractions and heuristics, and some of these are erroneous and receive limited or no scrutiny. We have nominated evidence based policy (when the decision space is multi sectoral, contested and facts are uncertain) as a likely candidate of an erroneous story or abstraction worthy of further scrutiny. The primary yield of science is explanation, but explanations do not come with an instruction set as how they should be used. Explanation has the potential to initiate a cycle of learning, leading to modified value orientations, beliefs and decision making behaviour. We propose that if the explanation-learning nexus is the primary purpose of presenting science evidence, we need a set of learning metrics. We used the values that guide peoples’ lives based on tested theoretical frameworks of cognitive and social psychology. New science explanation/evidence was absent during the first of our workshop series, where participants were encouraged to understand the contrasting perspectives and objectives of other agencies with a mandate to manage the Nam Xong. We treat this an informal experimental control. We did not anticipate learning to occur. If we did detect modified values, that would suggest that the process of debate and discussion may be a sufficient learning device. Our results supported the null hypothesis that in the absence of explanation, values held by decision makers would be stable and remain unchanged. The results rejected the null hypothesis that decision maker values are aligned with those held by surveyed Nam Xong households. That is, decision makers’ values do not reflect the households for whom decisions are actually being made.
We anticipate higher correlation between values held by decision makers and those of households when we introduce new evidence in future workshops. We will keep you posted on conclusion of the workshop series.
In our next blog post, we will explore our own testing of the evidence behind evidenced-based policy, through our work with natural resources decision-making at Nam Xong. For now we ask you to ponder:
1) In the decision space we described, what evidence do you have of evidence based policy? And 2) how will you fill out your next visa application?
Feynman R. (1955) “The Value of Science,” public address at the National Academy of Sciences (Autumn 1955); published in Feynman, R. and Leighton, R. (2001) What Do You Care What Other People Think: further adventures of curious character. Norton and Co., NY.
Feynman, R. (1998) The Meaning of It All. Perseus, Reading, MA
Foran T, Kemp-Benedict E, Ward J, Smajgl A, (2013). A technique…foresight… Ecology and Society, 18(4).6.
Gould S.J. (1996) The mismeasure of man. Norton and Co., N.Y.
Gould, S.J. (1996 b) Full House: The Spread of Excellence from Plato to Darwin, Harmony Books. p 57
- Popper, (1989) Objective Knowledge, An Evolutionary Approach, second ed., Clarendon, Oxford.
Smajgl A, Ward J, 2013. A framework to bridge science & policy… Futures, 52(8), 52-58.
Smajgl A, Xu, J, Egan, S., YI, Z.-F., Su, Y.,Ward J, (2015). Assessing …PES…China Environmental Modelling and Software, 69, 187-195.
Smajgl, A., & Ward, J. (2015). A design … research impact evaluation…Journal of Environmental Management, 157, 311-319.
Smajgl, A., Foran, T., Dore, J., Ward, J., & Larson, S. (2015). Visions, beliefs… Ecology and Society, 20(2):15.
Smajgl, A., Toan, T.Q., Nhan, D.K., Ward, J., Trung, N.H. , Tri, L.Q., Tri, V.P.D., Vu, P.T. (2015). Responding to rising sea-levels in Vietnam’s Mekong Delta. Nature Climate Change, 5, 167-174.
Stern, P.C., Dietz, T. and Guagnano, G.A., (1998). A brief inventory of values. Education and Psychology Measures (58): pp 984–990.
Sybille van den Hove (2007) A rationale for science–policy interfaces. Futures 39; 807–826