What History Reveals About America’s Top Secrets

BOOK REVIEW: The Declassification Engine: What History Reveals About America’s Top Secrets

By Matthew Connelly / Pantheon

Review by Terence A. Check

The Reviewer:  Terence Check is Senior Counsel, Cybersecurity and Infrastructure Security Agency; Adjunct Professor, Cleveland State University College of Law; LL.M, American University; J.D., Cleveland State University. Statements reflect the author’s personal opinion and do not reflect the position of any institution or agency.

REVIEW — Silicon Valley and Wall Street cannot get enough of “artificial intelligence” (AI) and similar magical tech buzzwords. The Declassification Engine shows that the mesmerizing effect of “AI” works equally well with publishing houses and think tanks. Despite broad ambition to reshape the historical narrative around our Nation’s classified documents and to elucidate the shadowy world of government secrets through computer analytics, The Declassification Engine is an otherwise ordinary history of the US national security apparatus in the atomic age. The book offers, in seemingly equal measure, the sober observations of a historian, entertaining interludes, and runaway speculation. Much like the datasets at the heart of The Declassification Engine, the book itself offers the careful and focused reader, a few valuable insights but one will have to sift through a lot of hay to find the needles.

The introduction begins with a compelling account of a fraught meeting between the author, Columbia University history professor Dr. Matthew Connelly, a couple of think tank representatives, and a handful of lawyers (including former intelligence lawyers) to examine whether the research conducted by Connelly and his team would violate espionage laws.

It’s not just for the President anymore. Cipher Brief Subscriber+Members have access to their own Open Source Daily Brief, keeping you up to date on global events impacting national security.  It pays to be a Subscriber+Member.

Dr. Connelly and his team sought to take a range of declassified datasets, mostly documents from Presidential libraries, and to run a series of automated analyses on the data to see what could be learned about how the US government manages its secrets. Naturally, the funders of the research and Dr. Connelly wanted to know whether the machines’ work could land them all in prison. Concerns assuaged by the limitation to work only on declassified data, the project went ahead, and The Declassification Engine reports the results in several hundred pages.

Most readers will focus first on the political arguments made in the book, which may either excite or discourage, depending on one’s worldview. (This review will not examine these political arguments in depth, as too much ink has already been spilled by better writers in never-ending debates about the “surveillance state” or “military industrial complex.”)

The beginning of the book asserts that the dark state of secrets is an aberration, that the original American republic was one that practiced “radical transparency.” It’s not clear, however, how much of that thesis relies on cherry-picked historical evidence or alternatively, describes an approach to security policy that is only weakly supported by the text of Founding documents and repudiated by actual state practice in extremis. In any case, time is better spent examining the data processing and artificial intelligence work by the History Lab that form the basis for the book’s political conclusions.

To begin, The Declassification Engine lacks transparency and makes it difficult for the reader to look “under the hood.” I cannot figure out, unfortunately, whether this opacity is some sort of meta-commentary by Dr. Connelly. These are not quibbles. For example, the body text lacks footnotes/endnotes. Instead, the scores of notes at the end of the book are organized by page number and by phrase on that page, rather than using superscript numerals in the precise location in the body text. Accordingly, the book contains no signal to the reader to look elsewhere for sourcing. This will frustrate those who may want to evaluate some of the more speculative passages in the book, such as the lengthy hypotheticals about potential Soviet infiltration into the construction of NSA headquarters decades ago (pg. 106-128).

Similarly, the body text and notes regarding the operations of the algorithms and programs themselves leave the reader desiring more detail. Oddly enough, these details appear not in the book, but on the History Lab website, the parent project that produced The Declassification Engine. Those craving statistics, data, and analysis will find more to sink their teeth into on this website. If the reader wants to see statistical measures like P values, percentages, or RoC curves, that information resides online for further parsing.

So, between the notes, the body text, and the History Lab website, a less blurry picture emerges. But, overall, many of the conclusions drawn in The Declassification Engine seem only tangentially supported by the data science. The book should have included more discussion of its experiments and its results. Are the programs written by the History Lab team auditable? Are the program’s outputs explainable? How did the team condition the data used? Do the attributes of the data used in the programs somehow slant the results in some way that requires further explanation?

My experience with facial recognition underscores the importance of all these questions, but particularly the last one. To illustrate, training a facial recognition algorithm on only criminal mugshot data may lead to slanted algorithm performance against one demographic group, say, an anti-Black bias, due to over-representation of Black individuals in such datasets. Do the History Lab’s computer programs contain similar flaws because of reliance on a handful of declassified collections?

Cipher Brief Subscriber+Members enjoy unlimited access to Cipher Brief content, including analysis with experts, private virtual briefings with experts, the M-F Open Source Report and the weekly Dead Drop – an insider look at the latest gossip in the national security space.  It pays to be a Subscriber+Member.  Upgrade your access today.

Structurally, The Declassification Engine reads like a collection of essays. Each chapter begins with an attention-grabbing anecdote, then a longer discussion of a particular aspect of the “dark state”, such as signals intelligence or trade negotiations. These passages sometimes contain lengthy expositions on why it’s necessary to care about transparency and accountability in that particular context as if the reader requires continual reassurance that pushing on is worth their time. The portions devoted to explaining each experiment and its results do not go on long enough. By my reckoning, Chapter 6 did not contain any discussion of a data science experiment at all.

The book would have been much improved with more visual aids, such as tables or side by side photographs of redacted documents and their unredacted twins. Omitting this information and discussing more sensational material like MK Ultra (pg. 238) illustrate the drawbacks of these structural choices. But The Declassification Engine, for its in-depth discussions of LSD and coups, does not extend similar treatment to the various apparatus of government managing the information and activities of the national security state. References to the Information Security Oversight Office (ISOO) appear just about as much as references to UFOs. The cadre of career officials who advise the government on these matters, records officers, privacy officers, civil liberties representatives, and government attorneys do not factor much at all to The Declassification Engine. Neither does the process of classification challenges, which I could not detect anywhere in the book or its notes. These mechanisms of self-governance and self-correction within the “dark state” are a key element that needed to be more meaningfully addressed in the book.

So, what does The Declassification Engine get right? First, the federal security bureaucracy does appear vast, complex, and expensive, even publicly available agency congressional justifications (the “receipts” provided to the taxpayer for appropriated monies) confirm as much. Billions of dollars and millions of labor hours create an information environment that will only continue to expand. In this regard, Dr. Connelly and the Declassification Engine express valid concerns over the proliferation of data and the possibility that it may become unmanageable even with an army of archivists. For academics like Dr. Connelly, the harm comes from the difficulty in preserving a historical record when the government has created and stored so much information that no one could ever make sense of it all. Security professionals may have different concerns, such as the possibility of extremely harmful material making its way into unauthorized hands by mistake, or critical data being deleted before its time because there simply are no more servers or warehouses to store the information. Dr. Connelly observes that secrecy leads to sclerosis (pg. 179) which might be made worse by a mountain of data that cannot be climbed.

Today’s constant barrage of information makes it easy for countries to wage disinformation campaigns and your emotions are the weapon of choice.  Learn how disinformation works and how we can fight it in this short video.  This is one link you can feel good about sharing.

The notion of applying algorithms and computer programs to the task of managing the government’s information may prove the most worthwhile takeaway from the book. As an attorney who studied history and enjoys the subject, the prospect of a future history lost to a chaotic and unmanageable sea of information does concern me. But beyond this, The Declassification Engine expresses a deep scepticism of the national security state and the manner in which secrets are created and managed. While the book does acknowledge that the government does need to keep some secrets, this scepticism is too firmly rooted and strays too far from academic/scientific objectivity. My fear is that many readers will be “turned off” before they reach the book’s more useful conclusions. My recommendation would be to read the introduction, then the conclusion, and select chapters to read at one’s discretion. 

The Declassification Engine shows why the government remains at a perpetual disadvantage in the public debate over classification and security policy writ large. National security frequently requires secrecy, and it is this operational imperative that prevents the government from conducting a full-throated defense in the court of public opinion. Much like proving a negative, one might struggle to describe why information must remain secret without revealing the contents of those secrets themselves. The Declassification Engine shows that most people will agree that some secrets should remain secrets, but beyond this, declassification questions are ultimately policy, even political, questions. Whether The Declassification Engine’s data science and artificial intelligence experiments will change our understanding of these matters remains to be seen, though I encourage the curious to read it and draw their own conclusions.

The Declassification Engine earns a solid 3.0 out of 4.0 trench coats.

The Cipher Brief participates in the Amazon Affiliate program and may make a small commission from purchases made via links.

Read more expert-driven national security insights, perspective and analysis in The Cipher Brief because National Security is Everyone’s Business.