The Journal of Threat Assessment and Management has published a new systematic literature review of violent extremism threat and risk assessment tools, authored by Anna Clesle, Jonas Knäble, and Martin Rettenberger.

The main takeaway is that the Terrorist Radicalisation Assessment Protocol (TRAP-18) currently has a far stronger psychometric evidence base than any of the other eight tools examined, and that much more research is needed on these tools overall. The review also has a valuable table summarising the population of interest each tool can be applied to, the outcome the tool is intended to help practitioners prevent, and whether the tool is concerned with threat assessment or risk assessment.

However, this last point raises an issue. While the table notes the distinction between threat assessment and risk assessment, the review itself does not elaborate beyond this sentence:

Available information about the development process was collected, and instruments were categorized by the purpose of their application, that is, threat management as the prevention of imminent extremist violence and risk management to guide decision making in the context of treatment and rehabilitation…

For the purposes of the review, further elaboration may not be needed. However, for this post I want to share some resources that delve more deeply into the differences between threat assessment and risk assessment in this context.

The terms “threat” and “risk” are colloquially similar, but “threat assessment” and “risk assessment” are two different (albeit overlapping) fields of forensic practice, the former more operational and the latter more clinical. Threat assessment has a real-time (or very short term) focus, such as a police service or private security agency monitoring whether an individual appears to be preparing for an act of violence, while risk assessment has a longer-term focus, such as a psychologist inferring whether a prisoner awaiting parole is likely to reoffend at some point in the future.

Each field has its own professional associations, its own handbooks, and its own journals. In recent decades, both fields have been drawn on for counter-terrorism purposes, although counter-terrorism itself developed over many decades as a police, intelligence and military concern quite separately to either of these fields of forensic practice.

These fields have increasingly entered the counter-terrorism space, and consequently the academic field of terrorism studies has engaged with them more. However, there has been no shortage of controversy.

The “threat versus risk” question is merely one of many relevant conceptual questions, but quite a foundational one, given debates about whether long-term risk assessment is even feasible for violent extremist recidivism. The most forceful version of this argument contends that “while short-term ‘hot’ threat assessment of terrorism offenders is arguably possible with appropriate caveats, long-term ‘cold’ risk assessment of terrorists, as requested by Australian legislation, is clearly not.”

This view is not universally shared. Many would hold the same concerns about various tools and about Australia’s High Risk Terrorism Offenders legislation without concluding that violent extremism risk assessment is entirely impossible. Nonetheless, the argument highlights that it is precisely the long-term focus of risk assessment makes it so fraught. As Marc Sageman notes, it “is a bit like the weather: scientists are better at accurately predicting the weather the closer the prediction is to the event.”

So, the distinction between threat assessment and risk assessment matters. However, it is also far from straightforward. The difference between threat and risk already unclear when it comes to everyday language, but the growing literature on violent extremist threat and risk assessment can itself show inconsistencies.

For example, this article briefly discusses the difference between threat assessment and risk assessment but describes the Violent Extremism Risk Assessment (VERA) tool as a threat assessment tool, even though the name suggests risks assessment. Other studies describe VERA as a risk assessment tool but the Clesle et al. review’s table describes VERA as concerned with both threat and risk assessment. Similarly, this study of VERA briefly mentions the Terrorist Radicalisation Assessment Protocol (TRAP-18) and describes it as a risk assessment tool, but TRAP-18’s creator Reid Meloy emphatically describes it as a threat assessment tool and prominently advocates for distinguishing between threat assessment and risk assessment. Separately, Monica Lloyd describes TRAP-18 as a hybrid of both threat assessment and risk assessment, while the Scottish Risk Management authority’s inclusion of TRAP-18 in its list of sufficiently validated tools (the only terrorism-related tool they include) suggests that they are comfortable considering it under the rubric of risk assessment. This shows that at different times, VERA and Trap-18 have each been described as threat assessment tools, as risk assessment tools, or both.

Furthermore, this forthcoming Campbell systematic review covers mostly the same tools as the Clesle et al. review, but describes itself as a review of risk assessment tools while Clesle et al. describe their study as a review of threat and risk assessment tools. This all shows that even within these literatures, consensus on the demarcation between threat assessment and risk assessment is lacking.

The disagreements centre not only on how specific tools should be classified, but how these fields of forensic practice should relate to each other. For example, some accounts describe threat assessment as a type of risk assessment while others argue that “it is preferable to conceptualize threat assessment as its own genre to avoid confusing the two.” Reid Meloy takes a middle ground approach, arguing that threat assessment is distinct from risk assessment while noting that the differences are “primarily a matter of degree rather than kind” and that “the theory and research relevant to one can inform the other”.

My own view is in line with Meloy’s, but the purpose of this post is to share a range of resources so readers can investigate further.

To begin, the following three sources do an excellent job of explaining both the differences, and similarities, between threat assessment and risk assessment as fields of forensic practice in the context of countering terrorism and targeted violence.

J. Reid Meloy, Jens Hoffmann, Eugene R. D. Deisinger, and Stephen D. Hart. “Threat Assessment and Threat Management.” In International Handbook of Threat Assessment, edited by J. Reid Meloy, Jens Hoffmann. Oxford University Press, 2021. https://doi.org/10.1093/med-psych/9780190940164.003.0001

Abstract: This introductory chapter sets forth three foundations for threat assessment and management: the first foundation is the defining of basic concepts, such as threat assessment and threat management; the second foundation outlines the similarities and differences between threat assessment and violence risk assessment; and the third foundation is a detailed overview of research findings, theoretical avenues, measurement instruments, and developments in practice over the past quarter-century. The goals of the chapter are to introduce professional readers to the young scientific field of threat assessment and management and to clarify and guide seasoned professionals toward greater excellence in their work.

Monica Lloyd. “Making Sense of Terrorist Violence and Building Psychological Expertise.” In International Handbook of Threat Assessment, edited by J. Reid Meloy, Jens Hoffmann. Oxford University Press, 2021. https://doi.org/10.1093/med-psych/9780190940164.003.0034

Abstract: This chapter is broadly concerned with the assessment of targeted violence, a term referring to instrumental and predatory violence, framed and justified by a shared ideology in the case of terrorists, or a more idiosyncratic belief system in the case of lone actors. It reviews some of the frameworks developed to identify threat before the crime and to assess the risk of extremist violence after the crime, and discusses the learning that has accrued from postdictive studies and ongoing empirical research, with the aim of synthesizing this learning and deepening the understanding of what drives these crimes. The roles of criminality and mental disorder are specifically discussed in relation to a possible triple pathway model for radicalization that clarifies the role of both in each pathway. The importance of theorization is stressed at this stage of current knowledge, with some suggestions for future research.

Caroline Logan, Randy Borum, and Paul Gill, eds. Violent Extremism: A Handbook of Risk Assessment and Management. UCL Press, 2024. https://press.uchicago.edu/ucp/books/book/distributed/V/bo215807381.html

This last example is an entire edited volume, so instead of an abstract I’m pasting this paragraph from page 276 that points to some of its discussions on the threat/risk distinction.

Paragraph: Borum describes the ways in which risk assessment and management practice varies from similar activities undertaken by threat assessment professionals. This is an important distinction to make – and one that is revisited in both Chapter 5 by Hart and Vargen and Chapter 9 by Ronald Schouten. Borum also articulates – here and in Chapter 4 – different conventions in applying the structured professional judgement (SPJ) approach, often reflected in the differences in depth and attention between professionals focused on risk assessment and those focused on threat assessment. Articulating these differences is important to clarify how practitioners work separately and together to ensure collaboration rather than contrary or competitive activity.

These resources provide important context for debates over the merits of various threat assessment structured professional judgement (SPJ) tools. For instance, risk assessment SPJ tools tend to be judged by standards of validity and reliability drawn from the social science of psychometrics (the study of psychological measures). Threat assessment SPJ tools may also be judged by these standards, but the relationship is a bit more ambiguous. This is because threat assessment is closer to the domain of intelligence analysis than risk assessment is. Intelligence analysis has its own associated field of social science known as intelligence studies, which includes its own psychological focus. Intelligence practice is also necessarily more secretive than, for example, courtroom deliberations over the risk of someone reoffending.

Indeed, despite threat assessment likely overlapping more with intelligence analysis than risk assessment does, the fields of both threat assessment and risk assessment stand somewhat removed from the practice of intelligence analysis. After all, intelligence agencies engaged in counter-terrorism for decades before either “risk assessment” or “threat assessment” grew as distinct forensic fields to help authorities deal with sex offenders, violent recidivists, stalkers, school shooters, and only much more recently terrorists.

While police services often use a range of threat assessment and risk assessment tools for various types of crime (see this excellent study of a family violence risk assessment tool used by Victoria Police), I could imagine that someone working for ASIO might view these debates as arcane and not in sync with how they use the terms “threat” and “risk” themselves.

This is similar to how public servants in defence and foreign policy may view academic debates between “security studies” and “strategic studies” as irrelevant to their everyday work and out of touch with their actual decision-making processes.

Moreover, the fields of threat assessment and risk assessment also developed quite independently of the academic fields of terrorism studies and intelligence studies. This means that we are not just talking about academic-practitioner divides, but any number of practitioner-practitioner and academic-academic divides.

Taking this back to threat assessment SPJ tools, the closest traditional analogues in intelligence practice would presumably be structured analytic techniques (SATs). Given that the field of intelligence studies routinely debates the extent to which SATs actually help analysts (or whether they hinder analysts’ creativity, imagination and autonomy by imposing too much structure), the extent to which the psychometric standards used to judge the reliability and validity of SPJ threat assessment tools are appropriate for intelligence analysis remains an open question.

If violent extremism threat assessment SPJ tools are going to invoke the authority associated with traditional violence risk assessment SPJ tools, they should be held to similar psychometric standards where feasible. Trap-18 does this well (see here, here and here). But this does not mean that psychometric standards should be imposed on the whole threat assessment enterprise, particularly when much of it is within the domain of intelligence analysis and when there is ongoing debate about whether structured approaches help or hinder the work of intelligence analysts.

The three resources I shared above do not delve into these questions, so I recommend three more resources alongside them.

These next three resources all address the question of the most effective and appropriate relationship between intelligence analysis and the forensic practice of threat assessment. In my view, these three articles all somewhat overstate the differences between threat assessment and risk assessment, and sometimes characterise risk assessment in an out-of-date manner (by suggesting that it relies heavily on actuarial tools and profiling, which was more characteristic of earlier generations of violence risk assessment).

However, the premise underlying all three is valid, which is that threat assessment approaches should take great care to complement the strengths of best practices in intelligence analysis rather than seek to supersede them.

Melissa Hamilton. “A Threat Assessment Framework for Lone-Actor Terrorists.” Florida Law Review 70, no. 6 (November 20, 2020): 1319. https://scholarship.law.ufl.edu/flr/vol70/iss6/4/

Abstract: Lone-actor terrorist attacks are on the rise in the Western world in terms of numbers and severity. Public officials are eager for an evidence-based tool to assess the risk that individuals pose for terroristic involvement. Yet actuarial models of risk validated for ordinary criminal violence are unsuitable to terrorism. Lone-actor terrorists vary dramatically in their socio-psychological profiles and the base rate of terrorism is too low for actuarial modeling to achieve statistical significance. This Article proposes a new conceptual model for the terroristic threat assessment of individuals. Unlike risk assessment that is founded upon numerical probabilities, this threat assessment considers possibilistic thinking and considers the often idiosyncratic ideologies and strategies of lone-actor terrorists. The conceptual threat assessment model connects three overlapping foundations: (a) structured professional judgment concerning an individual’s goals, capabilities, and susceptibility to extremist thought, plus the imminence of a potential terroristic attack; (b) a multidisciplinary intelligence team engaging collective imaginaries of an otherwise unknown future of terrorism events; and (c) coordination between counterintelligence officials and academic communities to share data and conduct more research on lone-actor terrorists utilizing a systematic case study approach and engaging theoretical methodologies to inform about potential new ideological motivations and terroristic strategies which may be emerging due to cultural, environmental, and political drivers.

Rick Malone. “Protective Intelligence: Applying the Intelligence Cycle Model to Threat Assessment.” Journal of Threat Assessment and Management 2, no. 1 (2015): 53–62. https://doi.org/10.1037/tam0000034

Abstract: Many law enforcement agencies have established protective intelligence units that conduct threat assessments, but they often rely on standard criminal investigation approaches rather than the collection and analysis methods typically used by intelligence agencies. This article reviews a basic model of the intelligence cycle and discusses concepts that are particularly relevant to the field of threat assessment, and how it can enhance the assessment and threat management process and guide the allocation of limited resources. The estimate of analytic confidence, based on specific criteria regarding the quality and quantity of information that conclusions are derived from, is especially useful, and provides a more meaningful expression of uncertainty than the “reasonable degree of psychological certainty” often seen with forensic opinions. This approach allows for separate expressions of analytic confidence and the concern for the threat of targeted violence. A proposed matrix based on these separate estimates provides a model for guiding investigative efforts and intervention strategies, as well as the allocation of manpower and other resources. This model may further facilitate much needed research in threat management practices and associated outcomes.

Andrew Harris, and Arthur Lurigio. “Threat Assessment and Law Enforcement Practice.” Journal of Police Crisis Negotiations 12 (May 1, 2012): 51–68. https://doi.org/10.1080/15332586.2012.645375

Abstract: During the past 2 decades, the problem of targeted violence has attracted attention from policymakers, law enforcement authorities, and members of the general public. Concerns over targeted violence have grown in the aftermath of high-profile violent events and the perceived institutional failures that are thought to have contributed to those events. These events and others have become the backdrops for public policies, legislation, and law enforcement programs to prevent targeted violence. The ability to meet such challenges has required a departure from traditional law enforcement practices, as well as the adoption of new paradigms, the deployment of additional resources, and the cultivation of new skill sets among police personnel. This article examines the threat assessment model and its application to specific problems facing local law enforcement agencies. The article focuses on 3 main questions: What is the threat assessment model? How might the model be applied to specific types of targeted violence? What operational demands are associated with applying the threat assessment model in law enforcement agencies?

Returning to the Clesle et al. review, I highly recommend reading it, and keeping an eye out for the forthcoming Campbell systematic review. The six resources shared in this post provide important context as many reviews and evaluations mention the threat/risk distinction without digging into it in detail or discussing whether it has implications for how different tools should be studied.

The first three resources help to clarify the distinction between threat assessment and risk assessment without overstating the differences. The final three resources do, in my view, tend to overstate the differences, but helpfully highlight the need to not neglect the strengths of intelligence practice and the needs of intelligence analysts. This is particularly important given active debates about what approaches best help intelligence practitioners. For social scientists, these debates can be helpful for identifying the best ways to study these tools, to ensure the chosen methods match the specific purpose of each tool and that the research meets the needs of those who use them.