This is an open access article distributed under the terms of the Creative Commons
Attribution License (In the wake of the replication crisis, there have been numerous calls to increase the clarity and precision of theory in the social sciences. A popular solution calls for the ‘formalization’ of theory, expressing theory in the language of mathematics (Borsboom, 2013; Borsboom et al., 2021; Frankenhuis et al., 2013; Fried, 2020; Guest & Martin, 2021; Muthukrishna & Henrich, 2019; Navarro, 2021; Robinaugh et al., 2021; Smaldino, 2020a; van Rooij & Baggio, 2021). Formalizing theory can promote clarity and precision, for instance, by requiring the scientist to specify all constructs and their relations, make assumptions explicit, and logically deduce predictions. Of course, formal models are no panacea, and may be of limited use when they rely upon questionable assumptions. In some cases, a well-articulated verbal theory may be sufficient. The key point is that scientists should strive for more rigorous, clear, and precise theory.
Such calls for increased theoretical rigor are not new. Similar calls, in fact, predate the replication crisis (Epstein, 2008; Farrell & Lewandowsky, 2010; Gibbs, 1987; Harris, 1976), as do more general manifestos cataloguing the immature state of theory in the social sciences (Gigerenzer, 1998; Meehl, 1978; Mischel, 2008; Rozin, 2001). It is unknown whether the application of formal theory is actually increasing in response to these calls. Regardless, formal and precise theory remains rare in the social sciences—disciplines studying human behavior, including but not limited to psychology, anthropology, criminology, and sociology—though exceptions exist in several subfields. And where formalization is absent, ambiguity can flourish (Smaldino, 2017).
The persistence of ambiguous theory raises uncomfortable questions: Are social scientists aware of the problem? If so, do they see this as only a minor issue? Could it be that some researchers in some cases prefer ambiguous theory? A naïve explanation of the prevalence of ambiguous theory might be that producing formal and precise theory requires advanced mathematical and programming techniques that are beyond the skillset of the typical social scientist. But this cannot be the whole story. After all, social scientists routinely use equally advanced methodological and statistical techniques. Might there be structural and systemic factors—rules, norms, and institutions—limiting the spread of formal and precise theory?
The Incentives Made Me Do it
“I have 3 recent experiences where I am publishing counter-evidence and the editor sends the ms directly to the author of the theory I am addressing and that person says, no, not counter-evidence at all because my theory can be stretched to cover that data.”
Susan Rvachew (2021, emphasis added)
In this paper, we argue that current incentives favor strategic ambiguity in the development and presentation of theory. By ‘strategic ambiguity’ we mean the use of ambiguity to achieve self-interested goals (Eisenberg, 1984). Any statement or set of statements, including a theory or hypothesis, is ‘ambiguous’ to the degree that it is open to multiple interpretations (Eisenberg, 1984; Gambetta, 2011; Lee & Pinker, 2010). Like everyone else, scientists sometimes strategically deploy ambiguity for the flexibility it affords—flexibility which can be used to obscure weaknesses, deny specific interpretations, and accommodate unexpected data. In these and other ways, ambiguous theories are harder to falsify than their precise counterparts (Popper, 1963; Smaldino, 2017). Although previous research has discussed the difficulty of falsifying flexible theory (Szollosi & Donkin, 2019, 2021), there has been less research on how scientists employ ambiguity strategically in formulating theory and on the incentives that make this possible. Analyzing strategic ambiguity in communication, Eisenberg (1984) noted: “The more ambiguous the message, the greater the room for projection. When an individual projects, he or she fills in the meaning of a message in a way which is consistent with his or her own beliefs” (p. 233). In science, ambiguous theories are more prone to confirmation bias—the tendency to search for, favor, interpret, and remember information in a way that confirms or supports one’s prior beliefs (Nickerson, 1998)—and are therefore more resistant to falsification when compared to clear and precise theories. As a result, the use of strategic ambiguity by scientists may come at the cost of scientific progress.
The “motte-and-bailey fallacy” (Shackel, 2005), a pernicious rhetorical trick named after a medieval castle design, illustrates the use and usefulness of strategic ambiguity. Here, the speaker first advances an indefensible claim (the ‘bailey’). When challenged, the speaker retreats to a more modest claim (the ‘motte’) that shares some similarities with the indefensible claim. In this way, the speaker can argue that the original (and indefensible) claim has not been refuted without ever having to defend it! Worse, the speaker can accuse the challenger of being unreasonable. As Boudry and Braeckman (2011) note, “A skilled pseudoscientist switches back and forth between different versions of his theory, and may even exploit his own equivocations to accuse his critics of misrepresenting his position” (p. 150).
Ambiguity at any stage of the research process can impede scientific progress (Rohrer, 2021). For instance, during statistical hypothesis testing, ambiguity in preregistrations affords degrees of freedom that researchers might exploit by running different analyses and selectively reporting those that yield desired outcomes. One solution for removing this ambiguity is to make preregistrations machine readable (Lakens & DeBruine, 2021; van Lissa, 2022). Here, we focus on ambiguity in theory formation and communication. We build on a previous blogpost on this topic (Frankenhuis et al., 2021), an earlier book chapter (Smaldino, 2017), and a recent talk on the properties of theories that promote their proliferation (Hussey, 2022). Though we draw our examples primarily from psychology, we believe the problem of strategic ambiguity is widespread across much of the social sciences (for a recent discussion of ambiguity in criminological theory, see Niemeyer et al., 2022).
The Natural Selection of Ambiguous Theories
Scientists are, of course, motivated to discover the truth and thereby help the scientific enterprise succeed. But, being humans, scientists are also motivated to have impact and thereby help themselves succeed. Impact results from having ideas widely cited and discussed. One way of achieving impact is to propose theories that accurately describe known phenomena and explain and predict novel phenomena. In this case, the interests of scientists and the scientific enterprise converge. Another way is to propose theories that can be interpreted in many different ways and thereby appear to describe known phenomena and predict novel phenomena. The ability of ambiguous verbal models to accommodate a wide range of findings and their resistance to falsification might increase their impact on the scientific community. If the motivation for impact is strong enough, scientists might prefer the flexibility afforded by ambiguous verbal models over the rigidity imposed by precise formal models. Intentionally or not, scientists can use this flexibility to create the appearance of consistent support and progressive research programs, which may be rewarded by journals, funding agencies, hiring committees, and the press.
The problem of detecting ambiguous theories masquerading as clear ones is made worse as gatekeepers, including scholarly journals, funding agencies, and university departments, crave impact, too. These gatekeepers also benefit from ambiguous theories if those theories produce research that raises a journal’s impact factor, increases a funder’s media visibility, or boosts a department’s ranking. Intentionally or not, scholarly journals, funding agencies, and university departments may be contributing to the proliferation of ambiguous theory. And so there may be many potential levers for improving incentives. In this paper, we focus on the incentives that push scientists, rather than scientific institutions, toward strategically ambiguous theory.
We should note that our discussion of strategic ambiguity does not require intentional ambiguity. Although intentional ambiguity certainly occurs, incentives can favor scientists who use ambiguous theories over scientists who use precise, well-specified theories without scientists being aware of the incentive structures nor consciously choosing to be more ambiguous (Hussey, 2022; Smaldino & McElreath, 2016; Stewart & Plotkin, 2021). For example, consider that some scientists criticize the use of formal models for the unrealistic assumptions they demand. This is a valid criticism in some cases. But it is also possible that this criticism rationalizes a preference for verbal models and the ambiguity they afford. Perhaps there is also the concern that formal models will reduce the impact of—or even replace—familiar verbal models, especially a researcher’s own. Some researchers may well be aware of their goals, but not the extent to which these goals end up shaping their beliefs. Our claim is that ambiguous theory proliferates especially widely in disciplines where verbal models predominate due to ambiguities in natural language, regardless of whether scientists act in good faith. We will also argue that reshaping incentives to favor transparency about ambiguity will require intentional, goal-directed action by scientists. These actions need to be supported throughout the scientific system, including scholarly journals, funding agencies, and university departments.
Outline of the Paper
We begin by arguing that there is nothing inherently wrong with theoretical ambiguity. In fact, in the early stages of theory construction, ambiguity may be inevitable—and may even be useful. But we also argue that this ambiguity should be transparent—and that scientific institutions should be designed to promote this kind of transparency. And so, in the subsequent section we provide recommendations for authors, reviewers, and journals to bring about transparency in the presentation of ambiguous theory. For instance, we provide a set of questions that can help to evaluate the extent to which a theory is ambiguous, akin to empirically-oriented checklists (Aczel et al., 2020; Flake & Fried, 2020). We conclude with suggestions for research on how scientists use strategic ambiguity to advance their careers and which norms, incentives, and practices favor strategic ambiguity. Our paper ends with a simple mathematical model exploring the conditions in which high-ambiguity theories are favored over low-ambiguity theories. This model illustrates that incentives for credit can favor low-quality, high-ambiguity science, and provides a basis for future formal analyses.
Clarity and Ambiguity Each Have Their Place
“Preliminary operationalizations and fuzzy inferences are not a crime, but a normal starting point of scientific discovery. Yet to progress toward precise claims, the initial vagueness must be recognized and tackled in subsequent studies.”
Anne Scheel (2022, p. 3)
At this point, some readers might complain, “But psychology [or sociology, or anthropology, or …] is a young science!” or “Human behavior is too complex to ever have precise theories like physics!” Depending on the details of the argument, we may be sympathetic to these complaints. There are deep and fundamental differences between physics and the social sciences (Fodor, 1974). But we think these kinds of objections, in general, miss the mark. We are not arguing that ambiguity has no place in scientific inquiry. Instead, we argue that while the transparent use of ambiguity can benefit scientific inquiry, we should not tolerate ambiguity used strategically to benefit the scientist at the expense of the scientific enterprise.
Of course, the study of human behavior faces special challenges. There is often a yawning gap between theoretical concept and empirical measurement, especially when compared to a science like physics. Physicists largely agree on how to define and measure the motion and mass of matter, while psychologists often disagree on how to define, let alone measure, the mental states and motivations of minds. Realistically, these kinds of challenges are unlikely to be resolved any time soon. Perhaps they never will. Nevertheless, at the level of theory—concepts and their relations—social scientists can and should aim for clarity.
Despite its benefit to the scientific enterprise, clear theory will not settle every dispute. For instance, scientists may disagree over which empirical unit (estimated from observed data) best captures a given theoretical unit (Lundberg et al., 2021; Rohrer, 2021). Nevertheless, this kind of debate will be more productive when scientists operate within a shared framework of transparent ideas and logic, rather than a wild west world of ambiguous intuition. We highlight two specific ways in which strategically ambiguous theory can be detrimental to scientific inquiry. There are certainly others.
Theory Stretching and Post-Hoc Precision
Intentionally or not, scientists might leverage theoretical ambiguity to serve their own strategic ends. Figure 1 illustrates two different ways in which scientists might exploit the wiggle room afforded by ambiguity to make the evidence for a theoretical claim appear stronger than is warranted.
First, a scientist might engage in ‘theory stretching’, interpreting an ambiguous claim more expansively to absorb data outside the scope of the original claim. Susan Rvachew (2021, April 17) describes this phenomenon in the epigraph above. Theory stretching may become a repeated pattern, with each revised and expanded claim serving as the basis from which to further revise and expand, thereby swallowing up more and more data that was outside the scope of the original claim. Second, a scientist might use ‘post-hoc precision’, interpreting an ambiguous theoretical claim narrowly so that it appears more precisely aligned with the data. For instance, a scientist might initially, and ambiguously, claim that people living in neighborhoods characterized by resource “variability” act more impulsively. Now, suppose the data support this claim only for temporal variability (fluctuating resources across time), but not spatial variability (different city blocks having different levels of resources, which remain stable over time). They might redraw their claim to apply only to temporal variability—crucially, without being explicit about this shift.
Figure 1
Illustration of how Ambiguity Affords Inferential Wiggle Room
Note. The rectangle represents a hypothetical space. The medium-sized, solid ellipse represents an initial theoretical claim. Encompassing a set of different hypotheses rather than just one, this claim is ‘ambiguous’ as it is open to multiple interpretations. Now, suppose data supports a hypothesis outside of the initial claim, represented by the dashed square. The large, dashed ellipse represents ‘theory stretching’ as the scientist swallows up data that was outside the scope of the original claim. Suppose, instead, the data supports one specific hypothesis within the scope of the original claim, represented by the dotted square. The small, dotted ellipse represents ‘post-hoc precision,’ in which the scientist narrows the original claim to more precisely align with the data. Both theory stretching and post-hoc precision lead to the overestimation of evidence for the original theoretical claim.
Theory stretching and post-hoc precision both lead to the overestimation of evidence for the original claim. Note that these practices differ from HARKing—Hypothesizing After Results are Known (Kerr, 1998)—which does not start with a theoretical claim, but rather with a search for statistically significant results. One way theory stretching and post-hoc precision can be kept on a leash is when, prior to conducting a study, researchers make precommitments (e.g., in a Registered Report) about which specific outcomes count as support for and against a theory or hypothesis (Nosek & Errington, 2020).
There is nothing inherently wrong with theory stretching and post-hoc precision, so long as they happen transparently. After all, what is learning if not revising beliefs based on observations? But if we allow ambiguous theory to masquerade as its clear counterpart, surreptitiously shapeshifting to match the data, we end up overestimating the evidentiary support for theoretical claims. Strategic ambiguity allows researchers to observe different or even contradictory data patterns, and nevertheless interpret both patterns as being consistent with the same theory (Fried, 2020; Robinaugh et al., 2021). One particularly pernicious practice is making different theoretical claims in different papers using the same label for each claim. By making empirical evidence seem to support a claim more strongly than is warranted, strategic ambiguity distorts the scientific record. To prevent this, scientists should be transparent about their use of ambiguity, a form of intellectual humility that benefits the scientific enterprise (Bringmann et al., 2022; Frankenhuis et al., 2021; Hoekstra & Vazire, 2021).
Ambiguity Might Be Inevitable—and Perhaps Even Useful
Ambiguity might be unavoidable in the early stages of theory construction. When scientists explore new territories, they encounter novel phenomena. They may face uncertainty about how to categorize or even conceptualize these new phenomena—and yet develop concepts and explanations they must (Scheel et al., 2021). And so they present candidate explanations with coarse-grain mappings between the observed parts of the system and the parts involved in their explanation. This can become an iterative process of tinkering as it may not even be initially clear what one’s hypothesis even is (Kauffman, 1971). It would be counterproductive to dismiss nascent theories for being ambiguous. How could they be otherwise? We cannot expect a useful map until we have charted the territory! As Paul Rozin (2001) notes, “we would do well to open our eyes more widely before we dig too deep a hole at one place in the broad and varied terrain of human social life” (p. 13).
Rozin’s (2001) warning of premature excavation can be interpreted in at least two different ways, either about uncertainty reduction or about surveying the problem space. Armed with a good map of the problem space, we can reduce uncertainty without having to ask ambiguous questions. Consider the way young children often play the game 20 Questions:
-
Is it Ringo Starr? No.
-
Is it George Harrison? No.
-
Is it Paul McCartney? No.
It would be most impressive if this strategy correctly guessed the target, but extremely unlikely as the overwhelming majority of possible solutions are not members of The Beatles. As adults, we know it is better to start with broad questions:
-
Is it a human? Yes.
-
Is this person still living? Yes.
-
Is this person an artist? Yes.
In the game of 20 Questions, broad questions can be just as precise as specific ones. In fact, the whole point of broad and precise questions is to quickly reduce the search space. Knowing that the target is a ‘human’ dramatically narrows the space of possible solutions. Whether broad or specific, there is little room to interpret what a question means so long as it is clear. Playing this game is similar to how Platt (1964) discussed ‘strong inference’ as a guide to scientific discovery, in which definitive experiments carve up the possibility space in a finer and finer grain. When the scientific terrain has already been well-mapped, definitive experiments present a powerful tool for learning about how the world works.
But what happens when we do not already know the problem space? Trying to reduce our uncertainty about a specific hypothesis would be premature. Here, we need to map the problem space. This is precisely the situation in which scientists often find themselves during the early stages of inquiry: Unclear about basic concepts and categories and unsure about the space of possible, let alone probable, hypotheses. During these early stages of scientific inquiry, ambiguity may not only be inevitable—it might actually be beneficial. Ambiguity affords more interpretive room, which might lead different scientists to think about the problem in different ways, ask different types of questions, and consider different kinds of evidence. The more diverse the pool of scientists studying the phenomenon, the more hypothetical space they are likely to cover (Hofstra et al., 2020). Some will hit upon fruitful lines of inquiry, leading others to follow. In the early stages of scientific inquiry, we should heed Daniel Dennett’s (1991) warning of the ‘heartbreak of premature definition,’ especially when studying elusive concepts like consciousness, which remain mysterious despite sustained scientific inquiry spanning decades, if not centuries.
To be clear, we are not calling for the abolition of ambiguity in science. That would be both counterproductive and futile. Instead, we argue that scientists should be as clear as they can be about the ambiguities lurking in their theories, whether studying novel phenomena or mysterious ones, and everything in between. And, as we will discuss, incentives should encourage transparent ambiguity and discourage its strategic counterpart.
Towards Transparency About Ambiguity
“Giving a bad thing a name can help to raise awareness. For example “p-hacking”. What should we call it when: detailed methods & explicit models are heavily criticized while thin methods & vague verbal models pass freely? Used to see this constantly on grant panels.”
Richard McElreath (2021)
We have argued that social scientists should strive for theoretical clarity and be transparent about ambiguity when it is either inevitable or useful. One way to do this is to construct formal models of hypotheses and theories that clearly state their meaning and scope (Kauffman, 1971; Smaldino, 2017). Several tutorials exist to support non-modelers in developing formalizations of theories (Frankenhuis et al., 2013; Fogarty et al., 2022; Smaldino, 2020a; van Rooij & Blokpoel, 2020; Wilson & Collins, 2019). But, while modeling is one way to reduce ambiguity, the two issues are hardly isomorphic. So rather than rehashing advice for model-building here, we instead focus on actions that authors, reviewers, and journal editors can take to increase transparency about ambiguity in the scientific literature. Clarity should be the goal, even if some ambiguity is inevitable and perhaps useful in the early stages of theory construction. By this we mean that one should be as precise as possible, given one’s current understanding, as to the scope of a theory or hypothesis: the set of conditions under which it is or is not expected to hold (Walker & Cohen, 1985). Clarity holds interpretative wiggle room on a leash, while ambiguity liberates it. By making assumptions, concepts and their relations, and the derivation chain from assumptions to predictions explicit, it becomes easier to decouple a theory from the scientist who proposed it, making the theory a public good for all to evaluate and use (Epstein, 2008; Guest & Martin, 2021; Meehl, 1990; Smaldino, 2017). Clarity also makes it easier for a hypothesis to be falsified, and the rate at which incorrect hypotheses are falsified influences the growth of scientific knowledge (even if this growth also depends on other factors, such as the discovery of anomalies, Kuhn, 1970, and the development of new theory that better accounts for the data, Lakatos, 1970).
Rewarding Ambiguity and Penalizing Precision
The epigraph above points out that, all too often, grant panels, hiring and promotion committees, and journal editorial boards more harshly criticize formal and precise models and methods than verbal and ambiguous models and methods. Responding to McElreath’s tweet, Smaldino (2020b, November 30) proposed the term ‘RAPPing’ to denote the practice of “rewarding ambiguity and penalizing precision”. The term RAPPing is general and could be used in the context of empirical work as well—for instance, when grant proposals are criticized for the planned sample sizes they provide, whereas other proposals that do not even state their planned sample sizes sail through. Here, our focus is on theory.
Why would a scientist evaluating another’s research reward theoretical ambiguity? One could argue that evaluators, being human, are imperfect, despite being motivated by good intentions. This might explain noise in evaluations, but not a bias toward ambiguity. We think there is something else going on. The mapping between theory and measurement is often inexact, especially in the social sciences. When scientists write clearly about their models and methods, this inexactness is brought into stark relief. As Julia Rohrer (2021) notes: “It is the curse of transparency that the more you disclose about your research process, the more there is to criticize” (Rohrer, 2021, December 8). Clarity can have the perverse effect of making it easier for evaluators to identify flaws that might have remained hidden in a more ambiguous description. This is especially detrimental if, as suggested by empirical research, grant reviewers weigh negative information more heavily than positive information (Teplitskiy et al., 2022). Ambiguous theory does not, of course, eliminate flaws, it merely conceals them. We hope that labelling this phenomenon draws attention to it and, hopefully, encourages remediation.
To help researchers avoid RAPPing, we list six questions evaluators can ask of any hypothesis or theory to help identify ambiguities (Table 1). For each question, we have suggested a few response options, but of course it is possible to obtain more graded estimates by rating those questions on a more finely grained scale. It is not essential that all these questions be answered in the affirmative for a hypothesis or theory to be deemed clear. For instance, while formal modeling makes it easier to satisfy questions 1–3 and 5–6, some verbal theories may be clear and precise without requiring a formal model. Furthermore, clarity is but one desirable feature of a scientific theory. However, each question answered in the negative should reduce the clarity assigned to the hypothesis or theory being evaluated. We also note that the clarity of a theory is not synonymous with its empirical testability. For example, the Hawk-Dove model has been influential in the study of human and non-human conflict but was never intended to be directly testable (Maynard Smith & Price, 1973). The Hawk-Dove model does, however, involve clear assumptions that can be directly compared with empirical data to assess its applicability. Thus, Table 1’s question about scope directly implies questions about the specification of conditions for validation or falsification. Finally, our list is not meant to be exhaustive, but merely suggestive. Future work could explore its effectiveness and improve the items for clarifying theoretical ambiguities through empirical research. Some journals and funding agencies could even train reviewers in how to use these types of questions effectively, aiming to increase standardization in evaluations of theoretical transparency.
Table 1
Six Questions to Help Evaluate Theoretical Transparency
| Question | Examples of responses |
|---|---|
| 1. Is each term clearly defined? | No / some / all terms are defined |
| 2. Are all relations between terms specified? | No / some / all relations between terms are specified |
| 3. Are all assumptions explicitly described? | No / some / all key assumptions are discussed |
| 4. Has the theory been formalized? | The theory does / does not have a formal basis |
| 5. Is the scope of the theory well specified? | The conditions in which the theory does and does not apply are unstated / coarsely described / fully explicit |
| 6. Is the theory consistent across papers? | The theory is consistent / inconsistent across papers in terms of its assumptions, scope, and predictions |
Evaluating Theoretical Transparency: A Worked Example
We illustrate how asking these questions can help identify key ambiguities by considering a specific example. We use this example not because it is a unique outlier, but because it is high-profile and representative of a widespread trend across the social sciences.
In a target article in the journal Behavioral and Brain Sciences, Baumeister, Ainsworth, and Vohs (2016) develop an argument for the following hypothesis: “Groups will produce better results if the members are individuated than if their selves blend into the group.” Some scientists might use this hypothesis as the basis for their empirical research. However, doing so would be problematic, because as it currently stands the proposal is not falsifiable (Smaldino, 2016). Evaluating this hypothesis—as it is presented in the paper—in light of Table 1 yields an answer of “no” to every single question. The hypothesis contains undefined terms and undefined relationships; it is laden with hidden assumptions and lacks clear scope; and the hypothesis is presented as a verbal rather than a formal model. Let us interrogate the hypothesis with Questions 1 and 5.
The hypothesis takes the form of a standard conditional statement (B occurs if condition A holds). The antecedent clause, “the members are individuated [instead of] their selves [being] blend[ed] into the group,” involves some ambiguity, but for simplicity we will focus most of our attention on the consequent: “groups will produce better results.”
First, the terms are not clearly defined. To what sort of groups does the hypothesis apply? The target article discusses a very wide range of groups, including track athletes in relays, families, universities, corporations, military organizations, and governments. Does the hypothesis apply to all groups? Kindergarten classrooms? Dutch soccer players? Bob Dylan fans? Or only to groups that have particular features? If so, which features? Should these features occur in combination or is any one of them sufficient?
Second, the scope of the hypothesis is never specified. What constitutes ‘better’ results? For some groups, there is a relatively straightforward answer: a sports team performs better when they win more competitions (though even here, not all competitions are equivalent). For other groups, it is harder to pin down a single measure of success; particularly as groups often exist for multiple reasons. An intervention could increase performance on one metric while decreasing performance on another (e.g., a corporation increases its profits but employee morale decreases, leading to attrition). What happens when groups are nested within larger groups, or when individuals belong to multiple groups with non-aligned interests? These and other ambiguities of the consequent clause must be addressed before one can test whether a particular group outcome is within the scope of the theory. In its stated form, the hypothesis is suggestive and can productively drive research to answer these disambiguating questions but cannot be directly tested. It is too ambiguous to falsify.
To reiterate, ambiguity is not necessarily a bad thing; it might be inevitable or even useful in the early stages of theory construction. Not all research can or should be driven by precise hypotheses. Instead, sometimes the purpose of research must be disambiguation rather than confirmation or falsification. Transparency about a current lack of precision can and should motivate future research. However, if ambiguous theoretical and empirical work is consistently supported at the expense of more precise research proposals—if there is excessive RAPPing by grant panels, editorial boards, and hiring and promotion committees—then science is the worse for it. We need transparency about ambiguity, which helps us decide for any case whether the ambiguity is inevitable, desirable, or harmful.
Recommendations and Future Directions
“[T]he effectiveness and the ethics of any particular communicative strategy are relative to the goals and values of the communicators in the situation.”
Eric M. Eisenberg (1984, p. 238)
The authors of this paper often use formal modeling. We have often heard critics argue that formal models are not useful tools in theory construction. These critics often defend their position by arguing that formal models make assumptions that are too simplistic, too unrealistic, and too arbitrary. Instead, these critics advocate for the use of verbal models, which they argue are more complex and more complicated, and therefore more realistic. To this, we might respond by highlighting the ambiguities inherent in verbal models, including imprecise constructs, implicit assumptions, and predictions based on intuitive rather than deductive reasoning. There are, of course, merits to each side of this debate (for a historically informed discussion of the mathematization of nature, see Eronen & Romeijn, 2020). Our hope is that partisans in debates like this can find common ground in our proposal to strive for transparency in the face of ambiguity, in much the same way that proponents of more exploratory research and proponents privileging confirmatory research can find common ground in the proposal to clearly delineate these two types of research (Chambers & Tzavella, 2022; Frankenhuis & Nettle, 2018; Nosek et al., 2018; Wagenmakers et al., 2012). How can authors, reviewers, and journals promote transparency about theoretical ambiguity?
Recommendations
First and foremost, the burden should be on authors to strive for transparency about ambiguities lurking in their theories. For instance, authors could write: ‘our definition [of some term] is not without problems’, followed by an explanation. Or ‘there is friction between one of our assumptions [about the relationship between two terms] and another assumption’, followed by an explanation. If authors use ambiguity deliberately, they should signal their intentions. If they do not, they may be building a Potemkin village, presenting a façade of clarity that collapses on closer inspection—and takes down anyone who was lured into working on its construction. Nguyen (2021) argues that epistemic manipulators strategically imbue their belief system with an exaggerated sense of clarity to avoid closer inspection. Whereas a sense of confusion invites us to think more, a sense of clarity, whether real or imagined, encourages us to terminate our inquiries, protecting the belief system. If authors believe their theories are clear and precise, they should welcome scrutiny rather than assume a defensive posture. The use of transparent ambiguity is fine. It is opaque ambiguity that poses risks precisely because it can be used opportunistically, whether intentional or not. The goal should be transparency in the face of theoretical ambiguity.
Reviewers should be mindful to not penalize authors for being transparent about ambiguities—just as they should not penalize authors for being explicit about exploratory aspects of their research. Reviewers should appreciate that transparency about ambiguity is a much-needed form of intellectual humility (Hoekstra & Vazire, 2021), a move that will only benefit the scientific enterprise. And just as a culture that licenses us to be more open and explorative in empirical research can feel liberating (Frankenhuis & Nettle, 2018), so too can a culture that acknowledges and embraces transparency about ambiguity in the development and presentation of theory. In addition, reviewers may consider penalizing opaque submissions—perhaps identified using Table 1—as these compete with transparent submissions.
Journals can help to reduce the harms associated with ambiguities by encouraging the formalization of theory and transparent communication about known ambiguities (van Rooij, 2022; Jamieson & Pexman, 2020). They might consider publishing articles or special issues in which empirical researchers collaborate with modelers to formalize theories, or in which many modelers independently develop models of an influential verbal theory (van Dongen et al., 2022). In addition, journals can be explicit that they value transparency about ambiguity and encourage authors to be transparent about ambiguity and reviewers not to penalize this kind of transparency. Additionally, funding agencies can help by supporting the development of formal theory.
We have argued that theoretical ambiguity proliferates in disciplines in which verbal models predominate. Though we have strived for clarity, we are certain that some of this paper’s content is open to multiple interpretations. We welcome criticism and hope for improved clarity as we develop and test these ideas. Toward this end, we propose two future directions, one empirical and one theoretical.
Empirical Research on Strategic Ambiguity
Empirical research could examine which norms, incentives, and practices favor the use of strategic ambiguity. This kind of research would benefit from qualitative and quantitative measures of ambiguity. A first step might be to have human reviewers evaluate the degree of ambiguity in theoretical claims, perhaps by using our Table 1. Another measure of ambiguity could be the number of distinct interpretations that different researchers provide when reading a theoretical claim. Such distinct interpretations appear to explain at least part of the variation in results obtained by the different teams involved in the various “Many Analysts” projects (Scheel, 2022). Specifically, a recent reanalysis of one “Many Analysts” study (Silberzahn et al., 2018) suggests that teams answered different versions of the underspecified research question (Auspurg & Brüderl, 2021). A more quantitative approach might involve the use of a machine learning algorithm that ‘reads’ a verbal theory and quantifies the degree of ambiguity. We are aware of one study that used Flesch’s reading ease score to explore whether the readability of abstracts of scientific papers is associated with the evaluation of their quality. Specifically, in an analysis of the Research Excellence Framework, a research impact evaluation of British higher education institutions, machine learning models rated harder-to-understand abstracts better than easier-to-understand abstracts; these models had been trained using mock ratings provided by scientists (Thompson et al., 2022). This finding is consistent with incentives for strategic ambiguity. We look forward to future empirical work exploring which norms, incentives, and practices favor strategic ambiguity in high-stakes, real-world settings. Some of this work should be qualitative, investigating how grant panels, hiring and promotion committees, and journal editorial boards interpret, discuss, and evaluate formal and precise models versus verbal and ambiguous models.
More broadly, the social sciences would benefit from identifying, cataloguing, and understanding the practices of theory stretching and post-hoc precision. For instance, as has already been done for HARKing (John et al., 2012), survey studies could examine how commonly researchers self-report engaging in theory stretching and post-hoc precision (self-admission rates), what they believe the percentage of other researchers who had engaged in each behavior to be (prevalence estimate), and among those researchers who had, the percentage that would admit to having done so (admission estimate) (John et al., 2012). In addition to providing baselines, this tool could be used to compare across social science disciplines (e.g., psychology and economics) and between subfields within a discipline (e.g., cognitive and developmental psychology). Such comparisons could be used to identify factors which contribute to theoretical ambiguity. Tracking these measures over time, and measuring their responses to interventions (e.g., changes in journal policies), can provide insight into perceived theoretical progress.
Finally, we have not covered all the reasons why theoretical ambiguity may be favored. For example, another reason might be the so-called ‘Guru effect’ (Sperber, 2010): people judge profound that which they fail to grasp. If ambiguity leads people to feel that they do not understand, this feeling of ignorance evokes awe, and this awe increases a theory’s dissemination and success, then ambiguous theory may be favored. Such a process could be studied empirically, for instance by examining whether (a) people feel that they understand ambiguous theories less well than clear ones, (b) ambiguous theories evoke more awe than clear ones, and (c) theories that evoke more awe are more likely to proliferate. Though these ideas are interesting and perhaps worth exploring, we have restricted our scope to falsifiability (rather than awe) as the mediating pathway for a theory’s success.
Modeling Strategic Ambiguity: A Worked Example
We believe theoretical modeling would be useful in exploring the conditions that favor and disfavor the strategic use of ambiguity. This kind of work could follow in the tradition of treating the scientific enterprise as a cultural phenomenon and applying the logic of cultural evolutionary theory (McElreath & Smaldino, 2015; Smaldino & McElreath, 2016). Such an approach can help identify conditions in which strategic ambiguity flourishes as well as interventions that might reduce it. Of course, this kind of research would also benefit from formal definitions of strategic ambiguity in the context of scientific theory. Such definitions could build on related developments in other fields, such as political science and communication (e.g., Aragonès & Neeman, 2000; Jarzabkowski et al., 2010; Pinker et al., 2008).
As a starting point, we develop a formal model to illustrate how and why theoretical ambiguity might be favored. This is a toy model, intentionally simple, designed only for the purpose of making clear an otherwise complex point. Like all models, this model makes specific assumptions. Any conclusions drawn from this analysis only apply to scenarios in which these assumptions apply.
Suppose scientists can choose one of two strategies determining how they produce and disseminate research: A low-ambiguity strategy (L), in which scientists derive and test precise hypotheses from clearly specified formal models, and a high-ambiguity strategy (H), in which scientists use ambiguous language to shroud the vagueness and imprecision of their theories and hypotheses.
Scientists receive credit for their work based on value conferred by their research community. We assume that scientific inquiry involves risk, so that any individual study may fail to generate credit. A low-ambiguity researcher produces useful and repeatable results a proportion p of the time. When they do, they receive a payoff of 1. And a proportion 1–p of the time, the low-ambiguity researcher produces less useful results. In these cases, they receive a payoff of ε (where ε < 1), which without loss of generality can be set to zero. Overall, low-ambiguity researchers have an expected payoff of p. The real-world consequences of low payoffs may be severe, especially for early-career researchers, as they can result in failures to get grants, promotions, or jobs.
We assume that high-ambiguity researchers produce research that is of lower quality in terms of usefulness and repeatability when compared to research produced by low-ambiguity researchers. However, by its very nature, highly ambiguous research offers flexibility in its interpretation, meaning there is a lower risk that the researcher fails to receive credit compared with the low-ambiguity strategy. At the extreme, there is little risk when a theory can accommodate any finding and get away with it. We assume that high-ambiguity researchers receive an expected payoff of q, such that 0 < q < 1. This payoff range captures two ideas. First, that ambiguous work is less valuable than low-ambiguity research that produces useful and precise results (i.e., q < 1). Second, because it can accommodate any finding, high-ambiguity research will be perceived as more valuable than low-ambiguity research when the latter fails to produce useful results (i.e., q > 0). Note that if p < q, the expected payoff (that is, the average payoff over a large number of studies) of high-ambiguity research is larger than the expected payoff of low-ambiguity research, even if useful and repeatable results obtained from low-ambiguity research are preferable to high-ambiguity research.
Finally, we assume that research is costly in terms of time and resources. We assign a separate cost for the low and the high-ambiguity strategy: cL and cH, respectively. Further, we assume that producing low-ambiguity research requires greater effort than high-ambiguity research (i.e., cL > cH).
We can use these costs and benefits to calculate the expected payoffs for each strategy. The expected payoff of a strategy is the credit received minus the time and resources spent. The expected payoff for a low-ambiguity researcher is therefore:
1
Similarly, the expected payoff for a high-ambiguity researcher is:
2
We are now able to ask when the low-ambiguity strategy is favored over the high-ambiguity strategy. This happens when UL > UH. In other words, a low-ambiguity research strategy is favored when:
3
which can be rewritten as follows:
4
The left side of the inequality is the extent to which the expected credit advantage of low-ambiguity research offsets the larger cost. Two conclusions follow from this analysis. First, because the denominator is always positive, this inequality is never true if the expected credit payoff of low-ambiguity research is less than that of high-ambiguity research (i.e., p < q). Second, and perhaps more troubling, highly ambiguous research can be favored even if, on average, it yields a lower payoff than less ambiguous research, so long as producing the latter is sufficiently more costly. Figure 2 illustrates this relationship. Note that when low-ambiguity research is much costlier than high-ambiguity research, high-ambiguity is always incentivized.
Figure 2
Illustration of the Mathematical Model
Note. The plot shows the credit advantage to low-ambiguity research (left side of Equation 4) as a function of its expected payoff, p. The differently colored lines represent different values for the added cost to low-ambiguity research. The low-ambiguity strategy is favored only when the solid lines are above 1 on the y-axis. This requires low-ambiguity research to pay off more reliably when such research is more costly. In this example, q = 0.3.
Based on the logic of this simple model, we have two levers to promote greater clarity and less ambiguity. First, we should decrease the credit accruing to high-ambiguity research, perhaps through increased standards of scrutiny within a research community. Second, we should be concerned if the cost to produce low-ambiguity research in a scientific discipline is significantly higher than the cost to produce high-ambiguity research. Otherwise, low-cost, high-ambiguity research becomes incentivized and will proliferate.
Conclusion
We are confident that increased transparency about ambiguity will benefit science, but we are less clear on the value of any specific protocols for detection or enforcement. For this reason, we have avoided a more specific set of guidelines for dealing with problems related to strategic ambiguity. Systemic problems often require sustained scrutiny and systemic solutions that focus on the system in which individuals operate, rather than simply nudging individual behavior (Chater & Loewenstein, 2022). We have a long way to go. A start would be to acknowledge, as some other disciplines do (e.g., physics, biology), that developing theory is not something that empirical researchers can just do ‘on the side’, but rather is a professional skill that requires specialized training. Where are the jobs for theoreticians in our field? Where is funding for them? In our view, such factors contribute to the current incentives not favoring rigorous theory. Ideally, proposed solutions operate in concert, targeting multiple components of a system, as is done by funder and journal partnerships rewarding transparency in empirical research (Chambers & Tzavella, 2022; Munafò, 2017). The challenge of how to improve incentives for better theory in the social sciences is underexplored. We hope metascience researchers will make progress on this challenge in the coming years.