Weaknesses in Computerized Linking Data Bases
By Dr. Maurice Godwin
Several projects have been undertaken to develop computerized classification systems based on the offender's actions in the crime. These include the use of computer technology to coordinate large data-bases to link offenders' crime scene antecedents a cross different jurisdictions. For example, a small number of Police in the United Kingdom have developed their own computerized data-base for linking violent criminals, for example, HOLMES. In the USA, the FBI has a linking system called the Violent Criminal Apprehension Program (VICAP). Other state linking systems are HALT in New York State and HVITS in the state of North Carolina. Broadly, the systems incorporate similar approaches to VICAP. Briefly, VICAP is a computer data base which depends on local and state police agencies to complete a check list of 186 questions about solved and unsolved crimes in their jurisdictions. The computer system is designed to flag similarities that might otherwise go unnoticed, for example, in unsolved murders. While these technological advancements have provided law-enforcement officials with more efficient ways of storing and managing data, there are inherent problems.
One problem in these computerized data base classification systems is, there are no empirical operational definitions for the questions being asked on the questionnaires. For example, there have been no attempts to explore to see if the right questions are being asked or if too many of the same questions are being asked. Research by Louis Guttman in 1979 pointed out that grand theories about relationships are rather useless from a scientific point of view if they do not include an a priori definitional system for observations. In light of these weaknesses, there has been no consideration, for example, given to explore the validity of questionnaires such as VICAP using techniques evolving out of the literature on psychometric theory. Psychometrics has a long tradition for providing an empirical way to test the reliability and validity of questionnaires (McDermott and Cobb, 1939).
The starting point for developing a robust data base linking system for inquiries into the nature of criminal action is the fact that the questions asked on forms such as VICAP do make distinctions between offenders. The application of psychological tests such as psychometrics to VICAP questions is to measure individual differences between offenders and between the actions of the same offender on different occasions. Information that originates from VICAP questions is supposed to provide investigators clues on what behaviors the instrument really measures. However, is VICAP questions measuring individual differences between offenders? Does VICAP work? Are the questions on VICAP reliable and valid? These questions are addressed in the next section.
Reliability of VICAP Questions
It appears that many VICAP questions require investigators to answer with subjective opinions, hunches, and these personal biases may lead to extravagant claims regarding what a particular question truly says about an offender's behavior. One question that requires the investigator to draw suppositions is VICAP question number 99, which refers to the offender's method of initial contact with his victim. The possible responses to this question are 1) immediately and physically overpowered the victim; 2) hit victim with hand, fist, or clubbing weapon; 3) choked; 4) stabbed; 5) shot; or 6) other direct assault.
The first response, physically overpowered his victim, is rather vague because this type of information is not likely to be known until after the offender has been arrested. Information concerning whether or not the offender physically overpowered a victim is usually gleaned from offenders' self-reports. Responses 2 through 6 it is not exactly clear what part of the offender's behavior is being measured here. For example, the FBI suggests that a variety of triggering factors can activate the offender's violence, and that many triggering factors centre around some aspect of control and dominance (see Ressler et al., 1988). Is it the initial method of contact with the victim or the degree of power the offender used to gain control over the victim that is being measured by this question?
Response one, immediately and physically overpowered the victim, seems to be the behavior that is being measured rather than functioning as one of the six responses. So, in essence, question 99 addresses two totally different emphases. One emphasis is the method of initial contact with the victim and the other is the degree of power used to gain control of the victim. Therefore, it seems more reasonable to view responses 2 through 6 as a set of actions that reflect how the offender physically overpowered his victim. The responses to question 99 are not mutual exclusive. There is no clear indication at all as to what behavior is being measured. There is no theory used to support the relationships between the possible range of responses, or the individual differences between offenders who use various methods of contact and power to subdued their victims. Determining which response to check on question 99 is solely determined on hunches made by the investigator about what the question implies.
The next VICAP question shows how additional reliability problems are encountered when different investigators interpret the same questions differently. For example, question number 145 on the VICAP form addresses whether the binding of the victim was excessive, much more than necessary to control the victim's movements. The response to this question is either a yes or no.
VICAP question 145 is clearly vague, and no indication of what behaviors the question is attempting to measure is given. For example, the degree of excessiveness can vary from one investigator to another resulting in different responses to the same question. Therefore, any future interpretation of the answers to this question as a part of an investigation can be misleading. The reliability problem in criminal data base questions, such as question 145, exist because there are no empirical operational definitions for the questions being asked on the forms. Equally as important is, similar problems exist by individuals who use deductive profiling to draw conclusions about a crime scene.
Validity of VICAP Questions
The validity of the questions asked on these type of data protocols is supposed to provide a direct check on how well this function is fulfilled. The determination of validity usually requires independent, external criteria of whatever the questions are designed to measure. However, there have been no attempts to determine how valid the questions asked on VICAP are in linking the behaviors of unsolved crimes. For example, VICAP question number 137 ask about evidence on how the offender disposed of the victim's body. The possible responses to the question are 1) openly displayed or otherwise placed to insure discovery; 2) concealed, hidden, or otherwise placed to insure discovery; or 3) the body was disposed with an apparent lack of concern as to whether or not is was discovered.
Referring to response one, body openly displayed, several hypotheses can be proposed none of which can be answered with VICAP question 137. To what extent was the body openly displayed? What behavior is being measured by this question? The question offers up a number of possibilities that are propounded in the FBI literature. One possibility seems to originate from the belief that offenders who openly discard their victims somehow this "contributes to the offender's overall fantasy" (see Ressler et al., 1988:60). Although this may be one of many possibilities, no evidence is given to support this theory as it relates to the VICAP question. It is highly unlikely that choosing response one will tell an investigator anything about the behavior of an offender.
The second response is conceal or hid the victim's body. Again, this action raises a number of hypotheses, none of which can be supported by the range of responses to the question. One hypothesis seems to suggest that concealing or hiding the victim's body "expresses the killers' relationship with or feelings towards another person by keeping the body close to them" (see Ressler et al.,1988:61). Another hypothesis is that concealing or hiding the body fuels the killer's fantasy (see Ressler et al., 1988), which is also given as possible hypothesis for openly displaying the body. VICAP question 137 tells us nothing about offenders who conceal or hide victims' bodies.
The response to question 137 that requires the investigator to draw broad assumptions about the offender's post-offence behavior is response 3, an apparent lack of concern as to whether or not the body was discovered. It is doubtful that any investigator would have knowledge about whether the offender was unconcern if the body was discovered or not. Information of this nature can only be gleaned from offenders' self-reports, and even that information is questionable. The face validity of question 137 is no doubt misleading. Briefly, face validity refers not to what the question measures, but to what it appears superficially to measure. The method of body disposal on the VICAP form, openly displayed, concealed or hidden, or an apparent lack of concern as to whether or not the body was discovered, does not advance our knowledge of criminals who perform these actions.
The lack of face validity can be found in another VICAP question. VICAP question number 166 addresses whether there is evidence that the offender disfigured the victim's body in order to delay or hinder identification. The range of responses to this question is either a yes or no. Like the VICAP questions discussed earlier, question 166 requires different investigators to draw broad and unsubstantiated conclusions about the offender's intent prior to being identified. For example, the question clearly suggests that the offender disfigures victims' bodies in order to hinder identification. Disfiguring to hide the victim's identity may be one of many hypotheses that can be discern from this type action. However, there are other possibilities. One possibility is the offender disfigures the body to retain certain parts as souvenirs, for reflection about the murder (see Ressler et al., 1988:63). If both of these hypotheses are possibilities, then, exactly which behavior is VICAP question 166 supposed to be measuring?
Utility of the VICAP System
Essentially, most of the criminal data base linking systems are built on assumptions propounded in the serial murder literature (see Ressler et al., 1988; Ressler et al., 1992). For example, one study on the relationship between organized and disorganized crime scene actions and background characteristics were measured using analysis of variance procedures (see Ressler, 1986). The problem with generating serial murder classifications using results from these types of tests is the fact that the explained variance is sensitive to the particular distribution that exist within a give sample and so the variable relationships are not usually stable across samples. The point being is, the comparison of differences between the offenders and variations that occurs within the offenders across a set of related crime scene actions are hidden when comparisons between group averages are made. Therefore, significant relationships between offence actions and offenders are less likely to be revealed through group comparisons. The problem is further compounded when the data is skewed in one direction, for example, like the organized and disorganized offender samples.
The offender and offence data entered in these linking data base systems have not been empirically tested and seems to originate from post hoc reasoning by few selected investigators without any attempt to validate the assumptions. The discussion previously of the VICAP questions demonstrates the ambiguities in interpreting what the questions are truly measuring. For instances, the relationships between crime variables in VICAP are produced by a few key individuals using work experience and intuition as a foundation. What is derived from this deductive profiling process is, the investigator's assumptions about what historical events might be connected to the behavior under study will determined what variables are, for example, studied. Consequently, no framework has been empirically established for identifying what the crucial or salient signature behaviors are for linking cases and, as a result, data base systems such as VICAP have been qualified failures.
Two important reasons were highlighted on why the computerized linking data bases are unreliable. One reason is that flat data bases are unlikely to pick-up on one to one variable comparisons or variations between offenders and offences. The problem is further exacerbated when comparisons between group averages are made. The second reason has to with how questions are designed and what behaviors are being measured. For example, questions on VICAP seem to originate from post hoc reasoning about the 'motivations' of criminals by a few investigators based on work experience and gut feelings. Consequently, there is no empirical research on the validity of VICAP questions or the data that is collected from the VICAP form.
Many of the VICAP questions are vague and provide no direct measurement on offence behaviors or offender background characteristics. The questions require investigators to draw speculative opinions about what each question implies resulting in different responses to the same question. For example, VICAP staff determines if "similar pattern characteristics exist among the individual cases in the VICAP system" (see Ressler et al., 1988: 118). This is rather an informal approach in determining if crime scene behaviors have a relationship or not. In light of VICAP weaknesses, no consideration has been given to improve the reliability and validity of the VICAP questionnaire using tests construction procedures, for example, psychometrics theory. Through psychometrics and factor-analytic techniques, linking data bases such as VICAP questions can be more systematically identified and defined. Each question could then be selected so that each represents the best available measure of the behavior under review or factors identified by factor analysis.
A longer version of this article can be found in Dr. Godwin's book Criminal Psychology and Forensic Technology.
Back to Main Site
COPYRIGHT: (2014-15) All rights reserved. No part of this may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission from Dr. Maurice Godwin.