chapter 9, Ideal Observer Models of Visual Signal Detection

Excerpt

9.1 Introduction

The objective evaluation of imaging systems requires three important components: (1) identification of the intended use of the resulting images, which we shall refer to as the task; (2) specification of the observer, who will make use of the images in order to perform the task; and (3) a thorough understanding of the statistical properties of the objects and resulting images. With these components, a figure of merit can be determined for evaluating the performance of the observer on the specified task [1]. In the next sections we consider each of these elements more fully.

The ideal observer is a model that describes the performance of the optimum decision maker on a given decision task. The ideal observer therefore provides an upper bound on task performance that can be used as a gold standard in the objective evaluation of imaging systems. Knowledge of ideal-observer performance allows the physicist and perceptual scientist to determine when information needed to perform a given task is readily extracted from an image by the human observer. When ideal-observer performance is found to be far above human performance, either the system should be redesigned to better match the human's capabilities, or the human observer should be augmented or even replaced by a machine reader.

In this chapter we detail the sense in which an ideal observer is optimum, describe the calculation of the ideal-observer strategy and ensuing performance metric for a number of tasks, and show how an imaging system can be evaluated using figures of merit from signal-detection theory that summarize ideal-observer performance. Results of investigations comparing human performance to that of the ideal observer are provided for a number of visual tasks.

9.1.1 The task

Medical imaging tasks can be broadly categorized as either classification or estimation tasks. In a classification task a decision is made regarding from which class of underlying objects the data are derived. In this chapter we shall concentrate on the binary decision task, where the image is to be classified into one of two possible alternatives, truth state 1 (T₁) or truth state 2 (T₂). When the states represent signal present (abnormal) versus signal absent (normal), the task is referred to as signal detection. The determination of whether a lesion or tumor is present in an image is a signal-detection task. More generally the two states are differentiated by whatever properties of the objects in class 2 distinguish them from objects in class 1.

An estimation task involves the quantitation of one or more parameters that describe the object, based on the raw data. The parameter might be the size, location, or activity of a tumor, the amount of flow in a vessel, or the cardiac ejection fraction. In tomographic imaging the reconstruction step results in a discrete image that is meant to estimate the spatial distribution of some characteristic of the object, for example, the distribution of a radioactive tracer.

There is a natural relationship between classification and estimation tasks; one can think of estimation as classification where the number of classes is the number of possible values the parameters to be estimated can assume.