Purpose: Blinded Independent Central Review (BICR) is a well-accepted method employed in many oncology registration trials. Ongoing monitoring radiologist “reader” performance is both good clinical trial practice and a requirement by regulatory authorities. We continue to use Reader Disagreement Index (RDI) as an important measure in BICR. In this work we studied RDI as an early indicator to identify an outlier reader during the monitoring of reader performance in BICR. Early indication would enable early intervention and thus possibly improve trial outcomes.
Methods: We performed a retrospective analysis of readers’ RDIs in nineteen different clinical trials. Ninety-two reader performances were examined at five intervals in each trial. These individual trial reviews were conducted by forty-three board-certified radiologist readers using several established imaging assessment trial criteria. The objective was to see how well RDI performance above a threshold at progressive monitoring intervals would “flag” a potential overall end-point performance “issue” for that specific reader.
Results: We present results for the prediction of exceeding threshold (one standard deviation above a study mean RDI). Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV) were determined for the predicted performance outcomes. We explored interpreting multiple “flags” for each trial to improve the aforementioned metrics.
Conclusions: One would expect that a “flag” of RDI exceeding threshold at an early-stage would likely give a useful prediction of end-point reader performance. We refined our methods to use multiple flags which enable statistically improved Specificity and PPV. Improved predictive capability at early stage intervals coupled with persistent monitoring across subsequent intervals will enable trial managers to focus on specific readers. An earlier indication of possible reader performance issues can permit proactive intervention and enhance good trial practices.
Purpose: Blinded Independent Central Review (BICR) is highly recommended by regulatory authorities for oncology registration trials. “Adjudication rate” provided by “Two Reviewers and Adjudicator Paradigm” of BICR has been part of reviewer performance metrics and trial efficacy. However, adjudication rate does not consider the adjudicator agreement or disagreement rate of a reviewer. We analyzed that Reader Disagreement Index (RDI) is a better measure than adjudication rate to monitor reviewer performance in BICR. Methods: BICR adjudication data from 3 different clinical trials including 10 board-certified radiologist reviewers using Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria was analyzed. RDI for each reviewer was calculated using the below mentioned formula. Reviewer adjudication rate and adjudicator agreement rate was calculated for each reviewer along with RDI. RDI was used to identify the discordant reviewer with highest disagreement rate. Number of cases where adjudicator disagreed with given reader RDI (%) = Total number of cases read ×100 Results: RDI identified the discordant reviewer in all 3 studies. Discordant reviewers identified using RDI were not the reviewers with highest adjudication or lowest agreement rates. Adjudication rate identified the discordant reviewer in 1 of the 3 studies. Reviewer with lowest adjudicator agreement could not have been identified as discordant reviewer using only adjudication rate in monitoring reviewer performance. RDI is more robust in identifying a discordant reviewer who neither has highest adjudication nor lowest agreement rate. Conclusions: RDI is more reliable measure of reviewer performance as compared to adjudication rate and could be effectively used to monitor reviewer performance as it combines both reviewer adjudication percentage and adjudication agreement percentage.
Purpose: To develop novel monitoring methods in Blinded Independent Central Review (BICR) imaging trials in which two radiologist reviewers assess the same images. In this project we aimed to ‘flag’ any reviewer that might have an assessment bias compared to the assessments of other reviewers on a specific study. Methods: Retrospective data analysis using R programming scripts was used to evaluate discordant assessments between two reviewers. We use a binomial test to determine the probability that an estimated low adjudication agreement rate is statistically less than the expected rate for all reviewer discordant assessment pairs. Results: We determined that for five or more discordant cases we can calculate the probability that each individual reviewer might have a statistically significant probability of low adjudication agreement for each discordant pair of assessments. We then analyzed the assessment data for sixteen oncological BICR clinical trials. Conclusions: The basic methods described can ‘flag’ or ‘signal’ a potential assessment ‘bias’. Although we initially focused on studies following one published clinical trial criteria to evaluate solid tumor we have applied the methods to other oncological studies with different published criteria which also may require double radiological reviews.
Purpose: Perform a retrospective review of a number studies (n=20) for the purpose of proposing basic likelihood metrics for evaluation of discordance between two reviewers performing RECIST (Response Evaluation Criteria in Solid Tumors) assessments in a Blinded Independent Central Review (BICR)
Methods: Retrospective data analysis using R programming scripts to determine discordance subsets of interest and analyze these datasets for both time point discordance and case discordance.
Results: We present a basic time point discordant ratio and a cases discordance ratio based on a range of aggregated time points per case for RECIST datasets.
Conclusions: We propose basic ratios that that might be useful to improve reviewer performance monitoring models
In this work we propose a machine-learning MO based on Naive-Bayes classification (NB-MO) for the diagnostic tasks of detection, localization and assessment of perfusion defects in clinical SPECT Myocardial Perfusion Imaging (MPI), with the goal of evaluating several image reconstruction methods used in clinical practice. NB-MO uses image features extracted from polar-maps in order to predict lesion detection, localization and severity scores given by human readers in a series of 3D SPECT-MPI. The population used to tune (i.e. train) the NB-MO consisted of simulated SPECT-MPI cases – divided into normals or with lesions in variable sizes and locations – reconstructed using filtered backprojection (FBP) method. An ensemble of five human specialists (physicians) read a subset of simulated reconstructed images, and assigned a perfusion score for each region of the left-ventricle (LV). Polar-maps generated from the simulated volumes along with their corresponding human scores were used to train five NB-MOs (one per human reader), which are subsequently applied (i.e. tested) on three sets of clinical SPECT-MPI polar maps, in order to predict human detection and localization scores. The clinical “testing” population comprises healthy individuals and patients suffering from coronary artery disease (CAD) in three possible regions, namely: LAD, LcX and RCA. Each clinical case was reconstructed using three reconstruction strategies, namely: FBP with no SC (i.e. scatter compensation), OSEM with Triple Energy Window (TEW) SC method, and OSEM with Effective Source Scatter Estimation (ESSE) SC. Alternative Free-Response (AFROC) analysis of perfusion scores shows that NB-MO predicts a higher human performance for scatter-compensated reconstructions, in agreement with what has been reported in published literature. These results suggest that NB-MO has good potential to generalize well to reconstruction methods not used during training, even for reasonably dissimilar datasets (i.e. simulated vs. clinical).
KEYWORDS: 3D modeling, Breast, Computer aided diagnosis and therapy, Radon transform, 3D image processing, Image classification, Mammography, Computed tomography, Cancer, Breast cancer
Current computer aided diagnosis (CADx) software for digital mammography relies mainly on 2D techniques. With the emergence of three-dimensional (3D) breast imaging modalities such as breast Computed Tomography (BCT), there is an opportunity to analyze 3D features in the classification of calcifications. We previously reported our initial work on automated 3D feature detection and classification based on morphological descriptions for single microcalcifications within clusters [1]. In this work, we propose the expansion of the 3D classification methods to include novel microcalcification morphological feature detection such as including more morphological classes and replacing the 2D Radon transform by a 3D Radon transform. Results show that the classification rate improved compared to the previously reported results from a total of 546 to 559 consistently classified calcifications out of 635 total calcifications. This slight improvement is due to the use of the 3D Radon transform and incorporating methods to detect two classes not previously implemented. Future work will focus on adding feature detection and classification of cluster patterns.
Model observers (MO) are widely used in medical imaging to act as surrogates of human observers in task-based image quality evaluation, frequently towards optimization of reconstruction algorithms. In SPECT myocardial perfusion imaging (MPI), a realistic task-based approach involves detection and localization of perfusion defects, as well as a subsequent assessment of defect severity. In this paper we explore a machine-learning MO based on Naive- Bayes classification (NB-MO). NB-MO uses a set of polar-map image features to predict lesion detection, localization and severity scores given by five human readers for a set of simulated 3D SPECT-MPI patients. The simulated dataset included lesions with different sizes, perfusion-reduction ratios, and locations. Simulated projections were reconstructed using two readily used methods namely: FBP and OSEM. For validation, a multireader multi-case (MRMC) analysis of alternative free-response ROC (AFROC) curve was performed for NB-MO and human observers. For comparison, we also report performances of a statistical Hotelling Observer applied on polar-map images. Results show excellent agreement between NB-MO and humans, as well as model’s good generalization between different reconstruction treatments.
Our previous Single Photon Emission Computed Tomography (SPECT) myocardial perfusion imaging (MPI) research
explored the utility of numerical observers. We recently created two hundred and eighty simulated SPECT cardiac cases
using Dynamic MCAT (DMCAT) and SIMIND Monte Carlo tools. All simulated cases were then processed with two
reconstruction methods: iterative ordered subset expectation maximization (OSEM) and filtered back-projection (FBP).
Observer study sets were assembled for both OSEM and FBP methods. Five physicians performed an observer study on
one hundred and seventy-nine images from the simulated cases. The observer task was to indicate detection of any
myocardial perfusion defect using the American Society of Nuclear Cardiology (ASNC) 17-segment cardiac model and
the ASNC five-scale rating guidelines. Human observer Receiver Operating Characteristic (ROC) studies established the
guidelines for the subsequent evaluation of numerical model observer (NO) performance. Several NOs were formulated
and their performance was compared with the human observer performance. One type of NO was based on evaluation of
a cardiac polar map that had been pre-processed using a gradient-magnitude watershed segmentation algorithm. The
second type of NO was also based on analysis of a cardiac polar map but with use of a priori calculated average image
derived from an ensemble of normal cases.
Polar maps have been used to assist clinicians diagnose coronary artery diseases (CAD) in single photon emission
computed tomography (SPECT) myocardial perfusion imaging. Herein, we investigate the optimization of collimator
design for perfusion defect detection in SPECT imaging when reconstruction includes modeling of the collimator. The
optimization employs an LROC clinical model observer (CMO), which emulates the clinical task of polar map detection
of CAD. By utilizing a CMO, which better mimics the clinical perfusion-defect detection task than previous SKE based
observers, our objective is to optimize collimator design for SPECT myocardial perfusion imaging when reconstruction
includes compensation for collimator spatial resolution. Comparison of lesion detection accuracy will then be employed
to determine if a lower spatial resolution hence higher sensitivity collimator design than currently recommended could be
utilized to reduce the radiation dose to the patient, imaging time, or a combination of both. As the first step in this
investigation, we report herein on the optimization of the three-dimensional (3D) post-reconstruction Gaussian filtering
of and the number of iterations used to reconstruct the SPECT slices of projections acquired by a low-energy generalpurpose
(LEGP) collimator. The optimization was in terms of detection accuracy as determined by our CMO and four
human observers. Both the human and all four CMO variants agreed that the optimal post-filtering was with sigma of
the Gaussian in the range of 0.75 to 1.0 pixels. In terms of number of iterations, the human observers showed a
preference for 5 iterations; however, only one of the variants of the CMO agreed with this selection. The others showed a
preference for 15 iterations. We shall thus proceed to optimize the reconstruction parameters for even higher sensitivity
collimators using this CMO, and then do the final comparison between collimators using their individually optimized
parameters with human observers and three times the test images to reduce the statistical variation seen in our present
results.
Previous Single Photon Emission Computed Tomography (SPECT) myocardial perfusion imaging (MPI) research has
explored the utility of numerical observers. One previous study proposed that the model of holistic visual search of a
myocardial perfusion image by an expert human observer might improve the development of a SPECT MPI numerical
observer. Further examination of numerical processing techniques that seem to be analogous to initial stage of human
holistic image search has helped to further refine the numerical observer. The current numerical observer considers
some fundamental issues in the refinement of the numerical observer: the need for background estimation, the
determination of blobs and the 'search-like' selection of a few blobs for subsequent decision analysis.
KEYWORDS: Signal to noise ratio, Breast, Reconstruction algorithms, Digital breast tomosynthesis, Signal attenuation, Tumor growth modeling, Optical spheres, Visibility, Visualization, Image filtering
Digital breast tomosynthesis (DBT) is a 3D imaging modality with limited angle projection data.
The ability of tomosynthesis systems to accurately detect smaller microcalcifications is debatable. This is
because of the higher noise in the projection data (lower average dose per projection), which is then
propagated through the reconstructed image . Reconstruction methods that minimize the propagation of
quantum noise have potential to improve microcalcification detectability using DBT. In this paper we show
that penalized maximum likelihood (PML) reconstruction in DBT yields images with an improved
resolution/noise tradeoff as compared to conventional filtered backprojection (FBP). Signal to noise ratio
(SNR) using PML was observed to be higher than that obtained using the standard FBP algorithm. Our
results indicate that for microcalcifications, using the PML algorithm, reconstructions obtained with a
mean glandular dose (MGD) of 1.5 mGy yielded better SNR than that those obtained with FBP using a
4mGy total dose. Thus perhaps total dose could be reduced to one-third or lower with same
microcalcification detectability, if PML reconstruction is used instead of FBP. Visibility of low contrast
masses with various contrast levels were studied using a contrast-detail phantom in a breast shape
structure with an average breast density. Images generated using various dose levels indicate that
visibility of low contrast masses generated using PML reconstructions are significantly better than those
generated using FBP. SNR measurements in the low-contrast study did not appear to correlate with the
visual subjective analysis of the reconstruction indicating that SNR is not a good figure of merit to be
used.
A number of groups are currently investigating tomographic imaging of the breast, but the optimal design and
acquisition parameters for such systems remains uncertain. One useful tool for investigating optimal parameters is
computer simulation software. A computer program that simulates xray transport through a breast object model
followed by signal and noise propagation through a CsI flatpanel detector has been modified, restructured and enhanced
in order to provide a fast yet sufficiently accurate research tool. The main focus of this work was to validate the
simulated response of a CsI flatpanel detector with a real detector namely, the Paxscan 2520 (Varian Medical Systems,
Salt Lake City, UT). Preliminary results indicate that the program provides comparable quantitative accuracy, that can
be used to obtain accurate and meaningful results to assist in research in tomosynthesis and CT breast imaging system
design.
The purpose of this study is to evaluate the recently proposed variable dose (VD) acquisition scheme that has been
hypothesized to overcome the limitations of microcalcification detection in breast tomosynthesis. In this acquisition
methodology, approximately half of the total dose is used for one central projection. This central projection view is
similar to a conventional mammogram and used to detect microcalcifications. The other half of the total dose is split
among the rest of the projection views. These variable dose projection data are then reconstructed and the 3D slices
are used for detection of masses. This novel acquisition methodology can potentially overcome the current
limitations with microcalcification detection in breast tomosynthesis (BT) and may result in faster and more accurate
detection of both microcalcifications and masses. Having access to both a conventional mammogram (i.e., the
central projection) and tomosynthesis slices would also act as a bridge for radiologists who are used to viewing
single projection images. In the current study, a comparison of microcalcification detection accuracy obtained using
VD and conventional BT was conducted. A realistic computer simulation was used to model the realistic noise and
blur encountered in BT systems. The simulation used a compressed breast phantom, modeled using CT images of
compressed mastectomy specimens. Localization receiver operating characteristic (LROC) analysis was performed
for detecting microcalcifications of size ranging from 147 microns to 178 microns. The results suggested higher
microcalcification detection and localization accuracy using the VD technique. The complete study will also consist
of evaluating detection of masses for the two strategies.
Preliminary evidence has suggested that computerized tomographic (CT) imaging of the breast using a cone-beam,
flat-panel detector system dedicated solely to breast imaging has potential for improving detection and
diagnosis of early-stage breast cancer. Hypothetically, a powerful mechanism for assisting in early stage breast
cancer detection from annual screening breast CT studies would be to examine temporal changes in the breast from
year-to-year. We hypothesize that 3D image registration could be used to automatically register breast CT volumes
scanned at different times (e.g., yearly screening exams). This would allow radiologists to quickly visualize small
changes in the breast that have developed during the period since the last screening CT scan, and use this
information to improve the diagnostic accuracy of early-stage breast cancer detection. To test our hypothesis, fresh
mastectomy specimens were imaged with a flat-panel CT system at different time points, after moving the specimen
to emulate the re-positioning motion of the breast between yearly screening exams. Synthetic tumors were then
digitally inserted into the second CT scan at a clinically realistic location (to emulate tumor growth from year-to-year).
An affine and a spline-based 3D image registration algorithm was implemented and applied to the CT
reconstructions of the specimens acquired at different times. Subtraction of registered image volumes was then
performed to better analyze temporal change. Results from this study suggests that temporal change analysis in 3D
breast CT can potentially be a powerful tool in improving the visualization of small lesion growth.
KEYWORDS: Breast, 3D modeling, Systems modeling, X-ray computed tomography, Sensors, Tumor growth modeling, Tissues, Breast imaging, Data modeling, X-rays
Dedicated x-ray computed tomography (CT) of the breast using a
cone-beam flat-panel detector system is a modality under investigation by a number of research teams. As previously reported, we have fabricated a prototype, bench-top flat-panel CT breast imaging (CTBI) system and developed computer simulation software to
model such a system. We are developing a methodology to use high resolution, low noise CT reconstructions of fresh mastectomy specimens for generating an ensemble of 3D digital breast phantoms that realistically model 3D compressed and uncompressed breast anatomy. These breast models can be used to simulate realistic projection data for both breast tomosynthesis (BT) and CT systems thereby providing a powerful evaluation and optimization
mechanism.
KEYWORDS: Computed tomography, Sensors, Breast imaging, Tumors, Reconstruction algorithms, Breast cancer, X-ray computed tomography, Monte Carlo methods, Diagnostics, X-rays
Dedicated CT breast imaging using a flat-panel detector system holds great promise for improving the detection
and diagnosis of early stage breast cancer. It is currently unclear whether dedicated CTBI systems will be useful
for screening of the general population. Possibly a more realistic goal will be contrast-enhanced, flat-panel CTBI
to assist in the diagnostic workup of suspected breast cancer patients. It has been suggested that the specificity of
CE-CTBI can be improved by acquiring a dynamic sequence of CT images, characterizing the lesion enhancement
pattern. To make dynamic CE-CTBI feasible, it is important to perform very fast CT acquisitions, with minimal
radiation dose. One technique for reducing the time required for CT acquisitions, is to use a half-scan cone-beam
acquisition, requiring a scan of 180° plus the detector width. In addition to achieving a shorter CT scan, half-scan
acquisition can provide a number of benefits in CTBI system design. This study compares different half-scan
reconstruction methods with a focus on evaluating the quantitative performance in estimating the CT number of
iodinated contrast enhanced lesions. Results indicate that half-scan cone-beam acquisition can be used with little
loss in quantitative accuracy.
Planar X-ray mammography is the standard medical imaging modality for the early detection of breast cancer. Based on
advancements in digital flat-panel detector technology, dedicated x-ray computed tomography (CT) mammography is a
modality under investigation that offers the potential for improved breast tumor imaging. We have implemented a
prototype half cone-beam CT breast imaging system that utilizes an indirect flat-panel detector. This prototype can be
used to explore and evaluate the effect of varying acquisition and reconstruction parameters on image quality. This
report describes our system and characterizes the performance of the system through the analysis of Modulation Transfer
Function (MTF) and Noise Power Spectrum (NPS). All CT reconstructions were made using Feldkamp's filtered
backprojection algorithm. The 3D MTF was determined by the analysis of the plane spread function (PlSF) derived
from the surface spread function (SSF) of reconstructed 6.3mm spheres. 3D NPS characterization was performed
through the analysis of a 3D volume extracted from zero-mean CT noise of air reconstructions. The effect of varying
locations on MTF and the effect of different Butterworth filter cutoff frequencies on NPS are reported. Finally, we
present CT images of mastectomy excised breast tissue. Breast specimen images were acquired on our CTMS using an
x-ray technique similar to the one used during performance characterization. Specimen images demonstrate the inherent
CT capability to reduce the masking effect of anatomical noise. Both the quantitative system characterization and the
breast specimen images continue to reinforce the hope that dedicated flat-panel detector, x-ray cone-beam CT will
eventually provide enhanced breast cancer detection capability.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.