This is an open access article distributed under the terms of the Creative Commons
Attribution License (“Kazdy wiatrak mysli ze grunt skrzydlami machac”
[Every windmill thinks that flailing of wings is what counts]
– Pan Zagloba in “Potop [Deluge]” by Henryk Sienkiewicz (1886/2017)
There is much to like, and agree with in the target article by Dariusz Doliński (2018, this issue; “Is psychology still the science of behavior”) that follows up and amplifies an earlier piece by Baumeister, Vohs, and Funder (2007). Both papers take to task social and personality psychologies for having abandoned “actual behavior” as a subject of study. Doliński analyzed a recent issue of the Journal of Personality and Social Psychology and concluded that only 9% addressed ‘real behavior’. Moreover, “not one of the studies involved exploration of behaviours associated with something other than a sitting posture and the movement of fingers on the part of the participants!” (p. 5). The implication of this statement and the similar one by Baumeister et al. (2007, p. 397) about “finger movements, keystrokes and pencil marks” seems to be that “real behavior” must entail gross bodily movements, and dramatic gestures as in the “wing flailing” of Pan Zagloba’s comment. A moment’s thought, however, reveals that keystrokes and pencil marks can have immense impact; they can, in fact represent “important things that people do” (of pivotal concern to Baumeister et al., 2007, p. 396).
President Trump or North Korea’s Supreme Leader Kim Jong-un have bragged about their ability to set off nuclear havoc by pressing their respective buttons1 (Gambino, 2018). In one of social psychology’s most impactful studies, Stanley Milgram (1974) had participants believe that they were shocking another person by the mere press of a switch. In an equally important study by Solomon Asch (1951), conformity was indexed by participants’ judgment concerning the relative length of lines, etc. If Doliński’s interest is in “important things that people do,” keystrokes, and pencil marks are not to be excluded as they can have truly momentous consequences.
What is Behavior?
Even though Doliński decries the absence in social and personality psychology of true measures of “behavior” he does not actually stop to define behavior. Doing so is instructive, in fact, because this concept is rather broad and it clearly encompasses keystrokes, marks on paper, and responses to questionnaires as bona fide behaviors. Consider the following definition by B.F. Skinner (1938, p. 6): “behavior is that part of the functioning of an organism which is engaged in acting upon or having a commerce with the outside world.” Or, take another definition by Johnston and Pennypacker (1993, p. 23) whereby “behavior . . . is that portion of an organism’s interaction with its environment that is characterized by detectable displacement in space through time of some part of the organism and that results in a measurable change in at least one aspect of the environment (p. 23). The breadth (and vagueness) of the concept of behavior is best captured by Malott and Trojan Suarez’s (2004, p. 9) striking “dead man test”: “If a dead man can do it, it ain’t behavior. And if a dead man can’t do, then it is behavior.”
In short, the majority of what personality and social psychologists study is in fact behavior; if so, Doliński misdiagnoses the ills of our discipline by attributing them to behavioral deficits in our work. This mischaracterization obfuscates the real issues to which this author importantly alludes and that bedevil contemporary research in personality and social psychology.
The Real Issues
Social and personality psychologies address the critical interface bridging individual and society; as such they should have important things to say about vast issues that people care and form policies about: problems of war and peace, oppression and dissent, extremism and moderation, self control and addiction, among others. Despite this potential, there is widespread feeling these days that research in our field hasn’t lived up to its promise and that our real world impact is less than could be expected (Kruglanski, Chernikova, & Jaśko, 2017; Motyl et al., 2017; Open Science Collaboration, 2015). Three inter-related problems account for this state of affairs, all implicit in Doliński’s (2018, this issue) and Baumeister et al.’s (2007) critiques. We label them as issues of validity, generalizability and consequentiality.
Validity
Our published studies, though highly controlled and ‘bristling with statistics’ as Doliński aptly put it, often lack construct validity: simply, their results do not bear out the constructs they are deemed to represent. It is not that self reports of behavior are not behavior. They are. The problem is that they are the wrong kind of behavior, not the kind the experimenter was interested in, and commented on. Stating how much one would donate is demonstrably different (and for well known reasons) from actually donating, and indicating on a questionnaire how one would behave and feel (while representing behavior) is a poor indicator of how one would feel and act in the circumstances at issue. After many decades of research on the fallibility and limitations of self reports we should be much more careful about how we interpret them, and about the claims we make on their basis.
Generalizability
A veritable crisis currently roiling our field is that of non-replicability (Open Science Collaboration, 2015). The findings of many of our studies fail to generalize to other contexts. Why should it be so? An important reason in our opinion is that social psychological effects are typically multiply determined. Moreover, the determinants that we study are often highly esoteric and unusual; in fact that is why we study them and the popular press (including textbook writers) touts them as “hot” and newsworthy. Often these have weak effects whose demonstration requires very special conditions, in which the “signal” they produce overcomes the “noise” produced by multiple alternative determinants of the phenomenon under study. This may be patently hard to do thus creating a problem of non-reproducibility even though the original demonstration might have been valid and replicable in the appropriate contexts. Consider the power posing effect (Carney, Cuddy, & Yap, 2010; Carney, Cuddy, & Yap, 2015). This research addresses a miniscule (though interesting) factor that could, under some restricted circumstances, produce a detectable effect. Power posing could affect one’s sense of confidence, but so could many other factors (prior experiences, one’s degree of fatigue, status differences between oneself and others). For the power posing effect to replicate, all those other (arguably more important) determinants of confidence should be held at a relatively low level, so that the ceiling on the DV isn’t reached. By privileging esoteric variables (that elicit a “Wow” effect) we create a problem of non reproducibility that undermines the credibility of our science.
Consequentiality
To be taken seriously, our findings must not only generalize to other labs but also have significant consequences in real world contexts. Esoteric variables are inconsequential almost by definition. So are hypothetical, uninvolving, scenario studies lacking both “experimental” and “mundane” realism (Aronson & Carlsmith, 1968). So are studies conducted almost exclusively with college students (e.g., see Sears, 1986) and with MTurkers paid mere cents for their participation. And, so are studies conducted almost exclusively with Westerners (Gelfand & Kashima, 2016), if the generalization targets include members of non-Western cultures. We thus agree with Doliński (2018, this issue) and Cialdini (2009) about the need to reinstate field research (i.e., work in situ) to its proper place and to “break up” as they put it.
The Publication Scene in Social and Personality Psychology: Between a Rock and a Hard Place
Doliński (2018, this issue) and Baumeister et al. (2007) insightfully identify the reasons for the current problems that challenge our science. We are caught between the Scylla of (occasionally unwarranted) IRB constraints (cf. Schweder & Nisbett, 2017) and the Charybdis of publication requirements of our major journals. Research on meaningful social psychological phenomena often breaks on the shoals of IRB rulings that have been known to disqualify innocuous albeit important research for insufficient reasons.
Unrealistic standards of precision and ever rising standards of productivity further delimit our ability to carry out consequential research. It is difficult to publish work on consequential social psychological phenomena studied in real life contexts in major social and personality journals because such research is often messy and multiple studies with large samples are often well nigh impossible to obtain.
Yet multiple studies are the standard social/personality researchers are expected to uphold. We (Kruglanski et al., 2017) recently sampled 40 articles published in each of three time periods in Journal of Personality and Social Psychology: 1965-66, 1990-91, and 2015-16 (120 papers in total). While in the 1965-66 period a psychological article published in JPSP contained an average of 1.25 studies, in the 1990-91 period that number climbed to 1.75 studies, and the 2015-16 average is 4.43 studies per paper, with some papers including as many as eleven studies (!). While recent Ph.Ds straight from graduate school hired to academic positions in developmental psychology had 2-3 publications to their name, and their counterparts in cognitive psychology had 5, those hired for social psychology positions had a whopping 10 with about 50% first authored publications (Valla, 2010).
Partly due to the ethical difficulties of investigating high impact social psychological phenomena, partly due to the emphasis in our major journals on mediating processes, multiple studies and perfect results, social psychologists are all but forced to rely on low hanging fruit, as Doliński notes, easy to execute studies carried out with readily available samples. These constraints contribute to the superficiality of our findings and the tendency to “learn more and more about less and less” as it were.
Conclusion
Doliński (2018, this issue) as well as Baumeister et al. (2007) make the plea to “put a bit more behavior into the science of behavior” (Baumeister et al., 2007, p. 401) while conceding “the need and the sense of studies concerning things other than real human behaviours” (Doliński, 2018, this issue, p. 12). We would like to believe that what these authors really mean is the need to make social and personality research more valid, generalizable, and consequential. Ultimately, the issue is not really about “behavior.”