Not by Behaviour Alone: In Defence of Self-Reports and ‘Finger Movements’

Dariusz Doliński (2018, this issue) addresses the important issue of social psychology’s failure to study the behaviour of humans. He is right to observe that social psychologists claim to study real behaviour but rarely do. We not only fully agree with this point, but also appreciate the author’s analysis and arguments about why it is important for researchers to try harder and do better when it comes to studying significant and meaningful forms of human behaviour.

In this commentary, however, we do not want to go over this ground again. Instead, we would like to use the available space to draw attention to a few other considerations that—in our view at least—need to be part of a broad debate about the nature of the dependent variables that social psychologists should investigate. In particular, in an attempt to extend this discussion, we would like to elaborate three points. First, it is important to be clear about what we mean when arguing for the importance of studying behaviour. Here debate often seems to centre on a firm distinction between behaviour and everything else (what we might call “non-behaviour”). In our view, we need to be careful because there are a range of contexts in which this distinction is more apparent than real and therefore proves unhelpful.

Second, while there are good reasons to be concerned that studies relying on self-reports and surveys far outnumber studies that examine actual behaviour, there are also potential problems associated with an aspiration to study behaviour exclusively. We should be mindful of these problems not least because studies that only examine behaviour do not necessarily produce more important, valid and true findings than studies that exclude behavioural variables.

Underpinning these two points is a final, larger, point. This is that while debate about the need to measure behaviour is important, there is another debate that we also need to engage in—a debate that should ideally precede this one. This relates to the question of what it is that social psychologists are trying to achieve in their research and what kind of independent as well as dependent variables they need to study in order to do this. Indeed, while we fully agree that it is time “to sound the alarm” when it comes to our failure to study human behaviour, we would also sound an alarm to draw attention to the neglect of theory when it comes to the necessary task of giving direction to various calls for change. In what follows, we will unpack these points in greater detail.

The Distinction Between Behaviour and Non-Behaviour Is Often Unclear and Unhelpful

When it comes to defining social psychology many researchers in the field still endorse the definition put forward by Gordon Allport in 1954 (Allport, 1954). Here Allport famously defined social psychology as the “scientific study of how people’s thoughts, feelings, and behaviours are influenced by the actual, imagined, or implied presence of others” (p. 1). This definition is interesting for a number of reasons. First, the very fact that three classes of dependent variables relating to thoughts, feelings, and behaviours are identified in the same breath, suggests that Allport did not see them as disconnected and independent, but as complementary. Indeed, each appears to provide a part of the puzzle needed to understand how humans are influenced by others.

So while it is true that the three dependent variables are not always correlated (i.e., people may indeed feel and think one thing but do the opposite), this does not mean that we should just ignore feelings and thoughts because behaviour is the only thing that matters in the end. To ask the question of how what happens internally is distinct from what happens externally is also not particularly productive for two reasons. The first is that human psychology cannot be neatly dissected into different components that are independently controlled by different motivations and drivers. The second is that even if such demarcation were clear, the bigger question is how these elements are woven together within human experience. And here, just as it is a mistake to psychologize and think that psychology is all that matters, so too it is mistake to behaviouralize and think that behaviour is all that matters.

Behaviour on Its Own Is Often Uninformative

It is also the case that when Allport defined social psychology as the scientific study of thoughts, feelings, and behaviours, he did not seem to see any of the three classes of variables as more important than the other two. Indeed, even though everyone seems to be in awe when we can show that our manipulations affect actual behaviour, behavioural evidence does not necessarily provide the ultimate proof of the validity of our theory. This view stands in contrast to the way in which behaviour is often characterised in the debate about behaviour versus self-reports and finger movements in which the reasoning often implies that behaviour is the ultimate dependent variable— the final and definite response that necessarily trumps both thoughts and feelings (see Baumeister, Vohs, & Funder, 2007; Kruglanski, 2017). Appealing as this position might be, it is important not to fall into the trap of arguing that emotions, beliefs and attitudes are unimportant and meaningless adjuncts to the study of ‘real’ behaviour. To do so risks recreating the psychological void and theoretical dead-end of radical behaviourism.

As behaviourism has shown us, one reason why we neglect cognition and emotion at our peril is that behaviour itself is never unambiguous and so it is never clear from a person’s behaviour exactly what it is that they are actually doing. Is writing this article an act of aggression, or of scholarship, or of self-expression? The significance of this point becomes very clear when we reflect on the lessons that have been handed down from classic studies in social psychology (e.g., see Jetten & Hornsey, 2017; Reicher & Haslam, 2017).

Turning first to the work of Stanley Milgram, as is well known, several variants of his ‘Obedience to Authority’ paradigm provided compelling evidence that a high proportion of participants would be willing to deliver ostensibly lethal shocks to a Learner who was performing poorly on a memory task. In this, we would all agree that the experiments captured some very compelling behaviour (i.e., participants were either willing to inflict harm on the Learner or they were not). Yet once one starts to engage closely with the studies, it turns out that the nature of this behaviour and its implications for our understanding of Milgram’s research — and its broader relevance to society — are no longer that obvious (or even that compelling). This is seen in recent debate about precisely how the behaviour of Milgram’s participants should be interpreted (e.g., see Brannigan, Nicholson, & Cherry, 2015; Haslam, Miller, Reicher, & Bettencourt, 2014). Should their willingness to administer shocks be seen as obedience to authority (as Milgram, 1974, argued); or as willingness to cooperate (Lutsky, 1995); or as engaged followership (Haslam & Reicher, 2017); or as a manifestation of trust (reflecting belief that the shocks were not real; Hollander & Turowetz, 2018)? Likewise, is unwillingness to continue administering shocks a sign of disobedience and willingness to challenge authority; or the result of identification with the victim; or a reflection of failure to trust the experimenter? The fact is, by looking only at behaviour, we can never answer such questions.

More generally, this example makes the point that behaviour whose meaning might seem at first blush to be self-evident is rarely — in fact never — self-evident at all. We see this in the case of Milgram where, given the ambiguity about what participants were actually doing when they administered shocks, researchers have necessarily focused increasingly on how participants felt and thought about their experience (e.g., Haslam, Reicher, Millard, & McDonald, 2015; Hollander & Turowetz, 2018) and used an array of methods to try to tap into these emotions and cognitions (e.g., see Burger, Girgis, & Manning, 2011; Haslam, Reicher, & Birney, 2014; Slater et al., 2006). Importantly too, while this research has augmented our understanding of what is going on in the paradigm, most of these recent studies have explored behaviour which (for ethical reasons) is less obnoxious than that of Milgram’s original research. But while this may make it less impressive and newsworthy it is no less useful scientifically.

Furthermore, the fact that the meaning of behaviour in the Milgram paradigm is ambiguous has meant that efforts to build theory around the studies has been notoriously difficult. In particular, because it was not clear what participants were actually doing when they complied with his experimenter’s requests, it is clear from Milgram’s experimental notebooks that he struggled to find a good explanation for his findings (see Haslam & Reicher, 2017). This is also apparent from various writings in which he provides a wide range of theoretical explanations of his results—explanations that are at times inconsistent with one other. For example, in 1963, Milgram leant towards a dispositional explanation of participants’ behaviour: “Obedience is the psychological mechanism that links individual action to political purpose. It is the dispositional cement that binds people to systems of authority” (Milgram, 1963, p. 371). A year later, though, he favoured a situational explanation: “The disposition a person brings to the experiment is probably less important a cause of his/her behaviour than most readers assume. For the social psychology of this century reveals a major lesson: Often, it is not so much the kind of person one is as the kind of situation in which they find him or herself that determines how (s)he will act” (Milgram, 1974, p. 205). To this we might add that Milgram’s own struggle itself reveals another major lesson: That the study of impactful behaviour does not on its own ensure theoretical progress. Indeed, in this regard, Blass (2004) makes the point that Milgram was actually handicapped as a theorist by the very power of the behaviour he had unleashed (see also Ross, 1988).

Much the same point also emerges from Asch’s famous line judgement studies. Again, as is well known, Asch provided compelling behavioural evidence that a significant proportion of his participants would be happy to conform to a majority that made judgements of line length that were clearly wrong. But was this really conformity? Or was it instead an attempt to avoid embarrassment or an act of politeness (see Jetten & Hornsey, 2017)?

Thankfully, to help answer such questions, Asch developed elaborate debriefing procedures to try to understand why his participants behaved as they did. Even though much of the richness of these self-reports — and of Asch’s findings more generally — is ignored in most textbook accounts of the phenomena (Griggs, 2015; Swann & Jetten, 2017), it is clear that the development of Asch’s theoretical analysis was primarily informed by attention to participants’ self-reports during this debriefing (Asch, 1955, 1956). Moreover, the fact that he was more successful in this theoretical endeavour than Milgram can be attributed in no small part to this fact. Indeed, where Asch attended closely to the self-reports that he garnered, it was left for later researchers to do this in the case of Milgram’s work (Haslam & Reicher, 2017; Hollander & Turowetz, 2017).

In the context of various points made by Doliński (2018, this issue), it is thus rather ironic that when it comes to understanding the psychological underpinnings of the findings of Milgram and Asch’s research, this task is rendered much more difficult (and is perhaps impossible) without recourse to self-reports. The key point here, then, is that when it comes to understanding psychology, behaviour never speaks for itself. In particular, this is because behaviour is typically silent about the motivations, beliefs and feelings that drive human behaviour. And because they help to break this silence, finger movements and self-reports are an indispensable weapon in psychologists’ arsenal (see also Swann & Jetten, 2017). Surely, the pen (or finger tap) is not always mightier than the sword, but it would be foolish to imagine that it never could be.

Behaviour Alone Cannot Inform Us About Psychological Process

Let us return one final time to Allport’s definition of social psychology as the “scientific study of how people’s thoughts, feelings, and behaviours are influenced by the actual, imagined, or implied presence of others” because this definition is relevant for one further reason. This relates to the fact that it is clear that the focus of this definition is not so much on the different types of dependent measures that are included to gain insight into processes that are of interest to social psychologists (i.e., thoughts, feelings, and behaviours), but rather on determining how these outcomes are “influenced by the actual, imagined, or implied presence of others”. That is, social psychology is a science that is first and foremost interested in theorising about social influence (Turner, 1991).

To achieve that goal, we therefore need to study observable outcomes of such influence. These outcomes include behaviour, but also quite reasonably encompass cognitions and emotions. Whatever outcome we focus on, though, we see that studying this is not a goal in and of itself, but stands in the service of helping us to develop better theories of this influence process. In the Milgram paradigm, for example, this means that we are not primarily interested in the fact that people are prepared to punish others, but in the fact that in doing so (or not doing so) they show how far they are prepared to go to enact the instructions of another person (the experimenter). What we are actually interested in here, then, is not the shocking per se but rather the obedience (or, depending on how one theorises this, the co-operation, the followership, the trust). In other words, we— like Milgram—are less interested in the behaviour per se than in the process that underlies it.

This suggests that the study of behaviour (or thoughts and feelings for that matter) can never be the sole focus of our research endeavours. In these too our focus should be on developing sound theorising about the process rather than privileging the behaviour itself (fascinating as this may be). To do otherwise is to put the cart before the horse.

This, then, is perhaps the most important point to get across in this commentary and that has most relevance to the practice of contemporary social psychology. Here we applaud recent attempts to grapple with the ‘replication crisis’ and improve the way we conduct our research. In discussion of how we might do this, though, we see that energy has been focused much more on considering ways to improve our empirical work than on improving the theory that needs to underpin and inform that empirical work. This is unfortunate because for psychology to become the science of action we need to first and foremost have theories in place that help us to make sense of important aspects of human action and behaviour.

Again, the Milgram case is instructive here. For while, by most standards, Milgram did ‘everything right’ as an experimenter (his methods were reproducible, his findings were replicable, the behaviour he studied was impactful, his data were made publicly available), his limitations as a theorist would mean that if the whole of psychology looked like this it would be an impoverished discipline.

Concluding Comment

In making the above points our goal is not to question the validity of the eminently reasonable points that Doliński makes. Rather it seeks to enrich his analysis by drawing attention to a broader debate that we need to be having when we reflect on the value of behavioural measures. In particular, we need to be aware that studying impactful behaviour can be a very useful way of shedding light on important psychological processes of social influence. But the fact that it is impactful does not necessarily make it useful in this regard. Moreover, it is equally legitimate, and can be just as useful to study social influence in the realm of thoughts and feelings (accessed via self-reports or finger movements). Indeed, rather that quarantining the domains of cognition, emotion and action there is much to be said for examining the complex interplay between these inter-related classes of outcomes. More importantly though, whatever it is that we study, we should avoid fetishizing the dependent variable above all else. For, if we do, we will end up with a science of impressive-looking bricks when what we really need is a solid house.