I’ve recently been making notes on the fields of terrorism studies, civil war studies, and social movement studies, looking at assessments from both people within these fields and people outside of them. When looking at the internal assessments of terrorism studies, two things stood out.

• The assessments are usually very negative about the field’s methodological rigour
• They’ve become much more positive in the past five to ten years

I had already been aware of the first point. Terrorism studies has regularly been accused, by its own leading scholars, of poor research quality. As Lisa Stampnitzky has written:

It may not seem surprising that the production of knowledge about such a contentious subject would attract external critiques. What is more deeply puzzling, however, is that some of the harshest and most frequent laments have come from the practitioners of terrorism studies themselves. Terrorism researchers have characterized their field as stagnant, poorly conceptualized, lacking in rigor, and devoid of adequate theory, data, and methods.

However, I hadn’t been aware earlier of the second point, the trend towards a more positive outlook. So this post is to show how internal assessments of terrorism studies have gradually become more optimistic.

Examples of the earlier, strongly negative, assessments are easy to find. Alex P. Schmid and A. J. Jongman famously wrote in 1988 that:

There are probably few areas in the social science literature in which so much is written on the basis of so little research. Perhaps as much as 80 percent of the literature is not research-based in any rigorous sense…

Over fifteen years later, Andrew Silke reviewed the output of two core terrorism journals (Terrorism and Political Violence and Studies in Conflict and Terrorism) between 1995 and 1999, and concluded that:

Ultimately, terrorism research is not in a healthy state. It exists on a diet of fast food research: quick, cheap, ready-to-hand and nutritionally dubious…. A limited range of methodologies in data gathering, combined with a reluctance to use more rigourous analysis, has left the field with serious deficiencies in many respects. Ultimately the methods used by terrorism researchers are essentially exploratory.

In 2005 John Horgan published a book on the psychology of terrorism, extensively reviewing the literature at the time. He later reflected:

I concluded on a depressing note. The psychology of terrorism, I argued, was at best under-developed, and at worst doomed to the mercy of unrealistic expectations of those who seek quick and simple solutions to the terrorism problem. Asking counterterrorism practitioners to consider contributions from the academic literature on terrorism was, at best, a half-hearted recommendation. Yes, there was a lot of quality research out there, but the unending torrent of drivel made it ever more impossible to keep oneself afloat.

But recently there has been a shift. Within the past five to ten years, the internal assessments have become more optimistic. The same scholars who issued such damning critiques have found considerable improvement.

For example, Alex P. Schmid’s most recent review of the literature argued that:

Looking back over four decades of terrorism research, one cannot fail to see that, next to much pretentious nonsense, a fairly solid body of consolidated knowledge has emerged. In fact, Terrorism Studies has never been in better shape than now…. Terrorism Studies — despite many shortcomings — has matured.

Silke’s most recent review, covering two core journals from 1990 to 2007, also found reasons for optimism. He argued that:

There are signs that research published in recent years is less opinion-based and more rigorous in methodology and analysis…. The use of inferential statistics on terrorism data in particular has nearly quadrupled since 9/11, a trend which can only help improve the reliability and validity of the conclusions being reached by researchers.

He more recently suggested, along with co-author Jennifer Schmidt-Petersen, that terrorism studies was experiencing its golden age:

Indeed, far from being stagnant or moribund, terrorism studies is arguably enjoying a golden age. High impact articles are appearing at a rate never before seen, and the core knowledge of the area is shifting and coalescing around new research and theories.

John Horgan’s 2014 update of his 2005 study also noted the field’s improvement:

So what has changed? In the intervening 8 years, there is much to commend. For a start, the field is no longer dominated by the small handful of researchers who traditionally characterized what is now commonly known as ‘terrorism studies’ (just don’t call it a discipline). Fortunately, the increase in interest from the social and behavioral sciences has also mirrored an increase in solid, quality research output. In fact the creep of systematic, interdisciplinary research on terrorist behavior has meant that it is certainly getting easier to distinguish opinion from analysis, and snake-oil conjecture from analysis that is informed by empirical evidence.

Similarly, Peter Neumann and Scott Kleinmann recently examined the rigour of ‘radicalisation research’, which can be considered a subset of terrorism research. Their findings were also positive:

Overall, the results are not as damning as one might have expected based on Silke’s and Schmid and Jongman’s earlier surveys of the terrorism studies literature. It would not be justified, therefore, to say, as Schmid and Jongman did in 1983, that the entire field is “impressionistic, anecdotal, [and] superficial.” On the contrary, there is much to be encouraged by, not least the fact that more than half of the items in our sample scored “high” in relation to either empirical or methodological rigor, and more than a quarter (27 percent) did so in both.

They also raise issues with Silke’s critique, arguing that it uses too narrow a definition to determine rigour:

For example, Silke classifies nearly all document-based research as “secondary source,” which effectively dismisses entire academic disciplines and methods—especially historical research—as methodologically and empirically inadequate.

They also question Silke’s emphasis on the use of statistics:

Indeed, for “micro phenomena” such as terrorism and radicalization, the use of qualitative methodologies—such as detailed case studies and narratives—may, in many cases, be more appropriate and produce more valid results than the construction of large—and largely meaningless—datasets. This, of course, is no excuse for laziness and sloppy research, which—disturbingly—could be found in many historical studies in our sample. But it suggests that it would be misleading to believe that quantitative research with large datasets, which—from a strictly methodological point of view—may be cleaner and more rigorous, is necessarily also the kind of research that will produce the most relevant insights about the phenomenon that radicalization research seeks to understand. Given the “micro” nature of the subject, the lower share of studies that draw on large amounts of empirical evidence—and, consequently, the less extensive use of quantitative methods, inferential statistics, etc.—must not necessarily be a cause of concern.

So on the whole, internal assessments of terrorism studies to be more optimistic about the field today than ten or more years ago, and find huge improvement.

This view isn’t universal. Marc Sageman recently argued that the field had stagnated, prompting responses from Max Taylor, Alex P. Schmid, David H. Schanzer, Clark McCauley, Sophia Moskalenko, and Jessica Stern, followed by a rejoinder from Sageman. And all the assessments mentioned in this post still note extensive problems with the field. Peter Neumann and Scott Kleinmann’s assessment ended by noting that:

Yet, despite clusters of excellence, there remains a significant amount of research that fails to meet minimum standards of scholarly work. In most disciplines, having 34 percent of published research that is either methodologically or empirically poor would be considered unacceptable, yet in terrorism studies—and radicalization research more specifically—this state of affairs has been allowed to persist.

This all leaves the question of why, despite improvement, there is plenty of sub-par research. There’s a lot of writing on this,  I recommend three articles in particular.

The first is an external assessment, from the earlier-mentioned Lisa Stampnitzky, who is a sociologist who the studied the field itself. In this article she describes how terrorism studies, rather than functioning as a purely academic field, awkwardly straddles academia, the media and the state. The second is Magnus Ranstorp’s introduction to the book Mapping Terrorism Research, which provides a good outline of the field and some of its problems. The final one is Thomas Hegghammer’s conference paper on the future of terrorism studies, which gives good advice on how current researchers can help progress the field.