Keeping secrets in large organizations
Lately I’ve found myself thinking about one of the classic defenses against conspiracy theories. The moon landing couldn’t have been faked, so the logic goes, because you can’t expect all of the huge number of people involved to have kept quiet. Intuitively, this makes sense. Eventually, someone is going to divulge the secret—and the more people who are involved, the more someones are out there to do it.
But the logic isn’t airtight. Two problems come to mind immediately.
Large organizations ask their members to keep secrets, and sometimes they even succeed. The CIA’s MKUltra project went on for two decades before the public found out about it. Or more mundanely, all the time there are new tech and entertainment releases whose creation requires a large team—movies, video games, hardware—but whose details are kept under wraps until the public announcement. (Plenty leak too, but not all of them!)
Leaks are not necessary or sufficient to shape public beliefs. A nontrivial number of Americans believe the moon landing was faked (as do an even greater fraction of Europeans), despite the seeming improbability of keeping such a huge conspiracy under wraps. No one would have had to point out the absurdity of the conspiracy theory if the theory weren’t out there!
Conversely, supposed whistleblower revelations don’t always dispel public skepticism, especially in the absence of a verifiable paper trail. We know there are people who will say anything to get attention, or who are nursing a grudge, or who are just trolling. In the realm of the much more mundane, I can’t even count how many supposed “leaks” of the Switch 2 hardware were posted on r/nintendoswitch that turned out to be total bunk.
These points notwithstanding, I think it’s typically going to be impossible to keep a secret in a large organization in the circumstances we care about. But I’d like to know when secrecy might be maintainable. The best way I know how to sort that out is to set up a model. As I don’t have the bandwidth to actually write and solve a proper model,1 this blog post is me thinking out loud about the ingredients that would go into it.
Cooking up a game-theoretical model
Here are the minimal components I think we’d need to study the ability to keep secrets in a large organization.
There’s a large organization. If we want to make claims about the likelihood of maintaining secrecy as the organization size increases, clearly there must be some sense of an organization size. So we’ll want there to be \(N\) “members” of the organization, who have access to secrets and the opportunity to leak them. Speaking of which…
There’s (potentially) something to keep secret. Breaking this down further—
There is some random variable whose realization is revealed to the organization members. The simplest way to model this would be a “good news”/“bad news” binary signal.
There is another player who wishes to know the state of the world, but who is not a member of the organization and thus does not have direct access to it. You can think of this as “the media” or “the public”. I’ll just call it the outsider. You could imagine there being some bias toward good or bad news, but that seems like a second-order consideration—for now, assume the outsider is truth-seeking (e.g., the outsider wants to select an action that matches the underlying state of the world).
The organization wants to prevent bad news disclosure. This is one of the things that’s elided (or at best only treated in reduced form) in purely mechanical models of leaks. We’ll need to introduce one last player, say a “manager”, who wants the outsider to take the “good news” action regardless of the true state of the world.
Members may leak (and may want to do so). After learning the state of the world, each of the \(N\) members of the organization may send a message to the outsider. A key question is whether these messages are verifiable, i.e., is it possible to send the “bad news” signal when in fact the news is good? Or are the messages just cheap talk? More on this below.
Another question is why members would want to leak. There must be some divergence in preferences between the manager and the members.2 You could imagine “noble” members who want the outsider to take the action matching the state of the world, or “dissident” members who always want the outsider to take the “bad news” action. My guess is the distinction doesn’t matter when leaks are verifiable, but matters much more so when leaks are cheap talk.
The organization can punish leaks. Without the threat of punishment, any noble or dissident member can be expected to leak bad news. Realistically, though, there are formal mechanisms (e.g., NDAs) as well as informal ones (e.g., implicit threat of firing or reassignment) through which organizations try to prevent harmful leaks.
The problem is uninteresting if leaking behavior is public, so assume the messages from the members to the outsider are unobserved to the manager. The manager just sees the choice the outsider makes, and then chooses which members to punish, after which the game ends. The manager’s incentive here is to punish a member iff that member has indeed sent a “bad news” message to the outsider. All else equal, members want to avoid punishment.
At baseline, there must be some heterogeneity among members in order for the punishment problem to be interesting. You could imagine there being some parameter that describes each member’s placement along the sycophant \(\to\) noble \(\to\) dissident scale, and the manager having different prior beliefs about the distribution of this parameter for each member.
At least to begin with, we’d want to assume that each member only cares about her own punishment or lack thereof, and that the outsider is indifferent to all punishments—there could be some interesting second-order effects from these utility externalities, but not worth the additional complication until the main interaction is nailed down.
Verifiable leaks
I think that if leaked information is directly verifiable, then the model I’ve described here essentially collapses into the Baliga, Bueno de Mesquita, and Wolitzky (2020) framework. In this setting, it only takes one “bad news” message to get the outsider to take the “bad news” action, so there’s more-or-less no persuasion problem vis-a-vis the outsider. So we can think of sending the “bad news” message as akin to committing an attack in the Baliga, Bueno de Mesquita, and Wolitzky (2020) model, and then the manager’s punishment choice is akin to the deterrence problem in that setting.
If I’m correct that we can port over the logic from Baliga, Bueno de Mesquita, and Wolitzky (2020), then keeping secrets indeed becomes more and more difficult as an organization gets larger, holding fixed the distribution of member types. A key result in Baliga, Bueno de Mesquita, and Wolitzky (2020) is the strategic complementarity among prospective attackers: an exogenous increase in one attacker’s likelihood of attacking leads to an increase in all other attackers’ equilibrium probabilities of attacking. In the model here, you could think of an increase in organization size as being like taking an prospective attacker’s “opportunity” probability in the Baliga, Bueno de Mesquita, and Wolitzky (2020) model from zero to some positive value. The underlying logic lines up with basic intuition—it would be close to impossible to pinpoint the source of a leak in a large organization, giving each individual member maximum license to leak.
One unsurprising lesson here is that if a large organization wants to keep bad news secret, it is necessary—though most likely insufficient—not to leave any kind of verifiable paper trail. Interestingly, as generative AI makes it easier to create fake visual and even video content, large organizations might be better able to keep secrets as information becomes harder to verify.
Unverifiable leaks
If leaks are akin to cheap talk, the strategic problem becomes more difficult, as now it is nontrivial to persuade the outsider about the state of the world. See my Vanderbilt colleague Chris Li’s work with Brian Libgober on “Group Talk” for a general model of cheap talk persuasion by members of an organization.
An important aspect of the unverifiable environment is how much information the outsider has about the potential biases of the organization members. First imagine a world where the organization members appear identical to the outsider (e.g., leaks to the media from low-level bureaucrats whose preferences vis-a-vis institutional priorities aren’t publicly discernible). Then the outsider’s decision rule will presumably be some kind of threshold: choose the “bad news” policy after receiving sufficiently many “bad news” messages, and the “good news” policy otherwise. Ex ante the manager would want to commit to a punishment policy that targets the members who are most likely to be near whatever threshold the outsider sets, but I suspect this wouldn’t be credible ex post. There are many moving pieces, but my bet—less certain than for the verifiable information case, to be clear—is that this also ends up looking like the Baliga, Bueno de Mesquita, and Wolitzky (2020) model.
The most interesting case, but also the most complicated to analyze, is when the outsider has enough insight into organizational politics to know which leakers are likely to be “noble” versus “dissident” (e.g., when the relevant organization includes political appointees or other people with known records). Unlike in the verifiable information world or the one where the outsider is just counting beans, now the dissidents have effectively no credibility. It’s really hard, albeit not impossible (Seidmann 1990), for cheap talk messages to be credible when the sender’s preferences over receiver actions are state-independent. But by the same token, now the manager can more effectively direct punishment resources toward the likely-“noble” members without running into the same ex post credibility problem as when the outsider had no information about members.
The cat-and-mouse problem between the manager and the members with unverifiable messages and visible member distinctions is sufficiently complicated that I’m hesitant to speculate about how the model would shake out. The one thing I’ll say about organization size here is that it seems like what would matter is not total size, but instead the number of members whose interests are aligned enough with the outsider that they can send messages credibly. That is distinct from the case where information is verifiable so member credibility isn’t a concern, or where members are indistinguishable to the outsider.
This work is licensed under CC BY-NC-SA 4.0
References
Footnotes
My main research project at the moment is a structural model of foreign aid competition between the US and China (coauthored with Michael Gibilisco, Anthony Luongo, and Miguel Rueda), a topic that has become all the more pressing as I write this post.↩︎
I suspect this is part of the explanation for relatively low leakage in business/media contexts, where there’s more incentive alignment than in political ones.↩︎