User:TRStewart/sandbox Judgment Analysis

= Judgment Analysis =

1.     Definition
Judgment analysis is a method based on Brunswik’s lens model and used by cognitive psychologists, to develop a descriptive model of a person making judgments based on multiple cues. It is used in judgment and decision research to measure cue utilization and consistency, both of which affect accuracy. It has been applied to improving self-insight,[ref] multiple cue probability learning, [ref] interpersonal learning, [ref] and cognitive conflict management. [ref]

2.     Method (rely on Cooksey for details)
This section summarizes the elements of judgment analysis intended to be part of a scientific study. When judgment analysis is used in applications, the elements may differ depending on the type of application. For a full description of the method, see Cooksey.

2.1      Selection and definition of judgment problem and participants
The judgment problem and the participants who will make judgments are determined by the purpose of the analysis. Care must be taken to define the judgment problem clearly.

2.2      Scenario construction
A number of scenarios, called “cases,” are selected or constructed. Each case includes the information to be considered by the judge. That information consists of a set of cues whose values vary for each case. Cases may be selected from actual cases (e.g., a doctor’s patients) or constructed (“hypothetical cases”). In either case, care should be taken to assure that the cases represent the population of cases that are the focus of the study.

The way that scenarios are presented to participants may affect their judgment. Scenarios may be presented numerically, graphically, pictorially, or some combination.

2.3       Exercise of judgment
Each participant is asked to consider a set of cases (typically 30-100 cases) and make a judgment about each case. The judgment is typically made on a numeric scale that is appropriate for the judgment (e.g., a weather forecaster might judge tomorrow’s maximum temperature in Fahrenheit). This results in a data set of cue values and judgments by each participant for each case.

2.4      Data analysis
The resulting data are analyzed separately for each participant. Typically, multiple regression analysis is used with the cue values as independent variables and the judgments as the dependent variable.

This analysis yields a descriptive model of the “judgment policy” that the participant used. Based on the model, three characteristics of the judgment policy are measured.

2.4.2    Function form--the form of the relation between each cue and the judgment (typically linear, U-shaped, or inverted U)
==== 2.4.3    The fit of the model to the judgments. The multiple correlation, R, is a measure of how well the judgment is described by the multiple regression model. With some assumptions, this can be taken to be a measure of the consistency of judgment. ====