The Effectiveness of Profiling from a National Security Perspective

Complete.pdf document

Authors

Jimmy Bourque, Ph.D.
Stefanie LeBlanc, M.A.
Anouk Utzschneider, M.Sc.
Christopher Wright, M.A. 

Summary of the issue

  • The context of this report is one where security agencies need tools to ensure national security, yet where some sensitivity exists with respect to the rights protected by the Canadian Human Rights Act;
  • Specifically, this report examines whether profiling constitutes a valid and effective means for the State to maintain national security.

Methodology framework

  • Three main methods were used to identify the 277 documents compiled: (1) access to computerized data banks and document search engines, (2) the “snowball” method in which references for previously obtained texts are consulted, and (3) recommendations made by experts in various fields;
  • Although the texts consulted covered a range of disciplines, they focused mainly on criminology, psychology, and law;
  • They cover the time span from 1965 to 2008 and discuss profiling experiences in ten countries over four continents;
  • The literature review involved consulting various types of documents, the majority of which were scientific articles published in peer-reviewed journals;
  • Empirical treatment, which is based on data collection and analysis, was applied to the assessment of the effectiveness of profiling;
  • Analysis of the documents gathered was based on two criteria: level of evidence and strength of evidence. Studies with major deficiencies in methodology that undermined credibility were systematically eliminated.

Behavioural profiling

  • “Criminal profiling can be defined as a technique that helps identify a suspect’s main personality traits and behavioural characteristics, based on the elements of the crime he has committed” (Beauregard and Proulx, 2001, p. 20).
  •  In Canada, behavioural analysis units perform the following duties: develop profiles of unidentified offenders, analyze crime scenes, reconstruct crime scenes, conduct indirect personality assessments, provide advice on investigations and interrogations, assist in the execution of search warrants, analyze statements or testimony, analyze suspicious deaths, conduct threat assessments and promote services offered;
  • Since 1992, the training of profilers in North America has been the responsibility of the International Criminal Investigative Analysis Fellowship (ICIAF);
  • Profiling is used most often in cases where police have few clues that could help solve a case. The goal of profiling is not to directly identify the person responsible for the crime, but rather to predict the most probable characteristics of the criminal sought; 5
  • The practice of profiling is based on certain premises: (1) human behaviour is predictable, (2) offenders commit their crimes in a consistent manner and can be distinguished from other offenders, and (3) the way they commit their crimes relates to their personal characteristics;
  • Criminal profiling is currently used in three phases of the criminal justice process: investigation, arrest and trial;
  • In Canada, three agencies have a section dedicated to criminal profiling: the RCMP’s Special Services and Behavioural Sciences Branch, the Sûreté du Québec’s Behavioural Analysis Service, and the Ontario Provincial Police’s Behavioural Sciences Section;
  • We cannot conclude that behavioural profiling functions in a systematic manner. However, there is anecdotal evidence that profiling may work;
  • The literature is replete with approaches and typologies, but these models lack a theoretical basis and empirical validation. For these reasons, none of the proposed models can be considered “scientific”;
  • Few empirical studies meet the rigorous criteria of credible scientific research. We are of the view that profiling may possibly contribute to police investigations, but it is still more of an art than a science;
  • However, we are of the opinion that profiling methods should be formalized, performance criteria should be developed, and empirical research should be undertaken to measure the true effectiveness of criminal profiling in Canada.

Geographic profiling

  • Geographic profiling may be defined as “... an information strategy for […] crime investigations that analyses crime site information to determine the most probable area of offender residence” (Rossmo, 2000, p.259).
  •  Its use is based on certain premises, namely: (1) the profile must be based on multiple crime scenes (several crimes committed by the same individual or several sites linked to the same crime); (2) the crime scenes must be linked to the same offender; (3) the offender’s residence (or base of operations) and his area of criminal activity need to be a short distance apart; (4) the crime scenes must be fairly evenly distributed around the offender’s home or anchor point; and (5) the offender cannot move from one anchor point to another or operate from multiple anchor points during his or her crime series;
  • The potential effectiveness of geographic profiling in terms of reducing search areas has been demonstrated empirically.
  • Police knowledge of a limited number of simple heuristics seems to lead to results equivalent to those obtained by software.
  • The practice of geographic profiling actually consists of two stages: (1) attribution of a series of crimes to the same offender, and (2) establishment of a geographic profile defining the search area. Although it has been empirically demonstrated that the second stage may be accomplished relatively efficiently, research cannot determine the ability of investigators to complete the first stage.

Prospective profiling

  • The basic principle of prospective profiling “is to develop correlations between specific criminal activity and certain group-based traits in order to help the police identify potential suspects for investigation. [Prospective] Criminal profiling uses probabilistic analysis in order to identify suspects and target them for surveillance” (Harcourt 2003, p. 109);
  • The practice of prospective profiling relies on two basic premises: (1) the rate of criminality of the members of certain social groups is proportionately higher than their representation in the general population, and (2) if such a situation is observed, it is fair and effective to target these groups in proportion to their rate of criminality in allocating police resources;
  • In addition to these two premises, it is presumed that criminals act rationally and will react accordingly to the fluctuation in the probability of being caught. This is the logic of deterrence: the assumption is that if the probability of being arrested for a crime increases for a given group, the crime rate will decrease accordingly;
  • Of all the fields studied, there is practically no empirical support for the effectiveness of prospective profiling;
  • There do not appear to be any exceptions to the finding that the actuarial approach is more effective than heuristic profiling;
  • No statistical link has been able to be convincingly established between an ethnic group and a given type of crime ;
  • Profiling based wholly or partly on sociodemographic characteristics is particularly sensitive to various forms of substitution, which for criminal organizations involves changing the profile of their agents;
  • Our review of scientific literature has therefore not allowed us to legitimize the practice of prospective profiling on scientific, legal or moral grounds, or to advocate threat assessment for events that, statistically speaking, are extremely rare.

Judgment in uncertainty

  • Judgmental heuristics are a type of cognitive shortcut for quickly assessing a situation: “The term judgmental heuristics refers to a strategy—whether deliberate or not—that relies on a natural assessment to produce an estimation or a prediction” (Tversky and Kahneman 2002, p. 20);
  • Heuristics, as opposed to solely analytical or rational models, better reflect the way people operate in real decision-making situations;
  • However, these heuristics lead to predictable biases. Some of the most well-documented biases include representative bias, weighting bias, cognitive availability bias and mental contamination;
  • Despite these biases and the fact that they frequently lead to erroneous predictions, both men and women are frequently overconfident in their ability to predict rare events;
  • These biases have been studied in various clinical judgment contexts, where it has been demonstrated that clinical judgment had been systematically surpassed by actuarial judgment;
  • Quite early in the analytical process, clinicians often form an implicit hypothesis that subsequently guides their search for information and its interpretation. This bias has also been observed in investigations.

General conclusion

  • The systematic effectiveness of criminal profiling has not been empirically demonstrated. However, we cannot therefore conclude that the practice has no merit;
  •  Profiling may possibly be seen as an art that is useful to the police investigation process, but it cannot currently claim to be a science;
  • More substantial effort has been made to conceptualize geographic profiling and provide solid empirical support for narrowing a search area based on the geographic location of crime scenes;
  • However, research fails to determine how successfully analysts are able to attribute a series of crimes to one offender (linkage analysis);
  • Prospective profiling should be separated into two categories: profiling of frequent incidents, and profiling of low base-rate incidents;
  • IIn the first case, it has been shown that a clinical approach or simple heuristic profiling is ineffective. This observation promptly led to the adoption of actuarial risk assessment measures, which have been clearly proven to be more effective;
  • In the second case, particularly rare incidents, no empirical research could be found to support the use of profiling or actuarial risk assessment.

Recommendations

  • [R1] – Inferential methods in behavioural profiling should be formalized and recorded (which does not mean, we should point out, that they must be made public, as criminals would then come up with a method to defeat them).
  • [R2] – Performance criteria should be developed to evaluate the true effectiveness of behavioural profiling.
  • [R3] – Research should be undertaken to empirically evaluate the effectiveness of behavioural profiling in the Canadian context. This research should cover three particular aspects: 1) the performance of profilers compared to that of detectives who do not have such training (in order to establish the added value of profiling for conventional investigation methods), 2) profile accuracy (by comparing profiler predictions to offender characteristics in solved cases), and 3) the actual contribution of profiling to suspect identification and arrest.
  • [R4] – The way in which geographic profiling coordinates are selected and entered should be standardized (for example, if an altercation starts in a bar, continues outside and ends in a homicide a few blocks away, which coordinate(s) mark the crime scene?)
  • [R5] – Research should be undertaken to evaluate the performance of analysts in the first stage of geographic profiling (attribution of crimes to the same suspect).
  • [R6] – Agencies should continue to use actuarial methods rather than prospective profiling or clinical judgment for threat assessment.
  • [R7] – To optimize threat assessment, particularly with respect to terrorism, it is crucial for agencies to have credible, current and relevant information. Intelligence services should have a way to obtain this information in Canada as well as abroad, while respecting the Constitution and international law.
  • [R8] – The sharing of intelligence among agencies, particularly the RCMP, CSIS and the CBSA, should be encouraged and optimized.
  • [R9] – Performance criteria should be developed for the various actuarial tools used by the agencies. The actual effectiveness of instruments should be periodically evaluated (which ties in with a recommendation in the 2007 report by Auditor General S. Fraser with respect to the CBSA).
  • [R10] – Ethical standards should be developed to govern the practice of threat assessment from an actuarial perspective.

The opinions expressed in this report are those of the author and do not necessarily reflect the views of the Canadian Human Rights Commission or the Canadian Race Relations Foundation.

  • Last modified
    2015-07-14