LETTER

Article in PDF format - JOP Home page

Assessment of the Severity of Acute Pancreatitis. The Usefulness of ROC Analysis in Comparative Studies of Clinical and Imaging Prognostic Indices

Paraskevas S Brestas¹, Urania G Dafni²

¹Department of Radiology, General Hospital of Athens. ²Division of Public Health, Laboratory of Biostatistics, Department of Nursing, University of Athens. Athens, Greece

Dear Sir:

We read the original article regarding assessment of the predictive value of the CT severity index (CTSI) for the severity of acute pancreatitis by Gürleyic et al. with great interest [1]. The purpose of this study was to make a comparative assessment of the accuracy of the CTSI proposed by Baltazar et al. [2], the APACHE II score and serum CRP concentrations in predicting the severity of acute pancreatitis. The cut-off values of 3 for the CTSI and 7 for the APACHE II were used in the present study, based on the results of studies which did not have exactly the same discrimination endpoint [2, 3]. The discrimination endpoint in this study is defined as the ability to separate those patients who had mild pancreatitis from those who had severe pancreatitis, according to the classification criteria of the 1992 Atlanta International Symposium [4].

Receiver operating characteristic (ROC) curves could have been used to determine the most appropriate cut-off point for the selected discrimination endpoint which corresponds to the best possible trade-off between sensitivity and specificity which were estimated in the present sample. Moreover, the area under the ROC curve is a reliable measure of overall predictive discrimination and a previously described method for comparison of the areas under the ROC curves, derived from the same cases, could also have been used [5, 6].

The additional information provided by ROC curves in studies of prognostic indices of acute pancreatitis severity derives from the complete illustration of the relationship between sensitivity and specificity for a certain discrimination endpoint (severe vs. mild pancreatitis in this case). This might have been useful because: a) the clinical impact of the two types of misclassification (failure to correctly identify a case of severe pancreatitis or failure to correctly identify a case of mild pancreatitis) is not the same, and b) it is necessary to realize new prospective comparative studies for assessing the clinical impact of promising imaging techniques, such as MRI or contrast-enhanced US in the near future [7, 8].

Moreover, it should be noted that even though the Atlanta classification system provides a reliable basis for experimental studies for the clinical management of acute pancreatitis, it is not considered to be a perfect system since intermediate forms of the disease do occur [9]. If an imperfect gold standard is used, the estimated accuracy of the tests may suffer ("imperfect gold standard bias"). Another type of bias affecting ROC analysis is "verification bias" and takes place if some of the patients with test results do not have verified disease status or if the decision to verify a patient is influenced by the test results. Calculation of the accuracy of a diagnostic test using standard definitions unavoidably includes the risk of some kind of bias under certain circumstances. ROC analysis offers the possibility of bias-correction methods [10] and methods of non-parametric estimation of ROC curves have been also suggested recently in the case in which the gold standard is not binary or in the absence of a gold standard, [11, 12]. Thus, ROC analysis should be the preferred method for the assessment of the predictive value of imaging techniques.

Article in PDF format
Look up who cited this article

Received December 14^th, 2005

Keywords Inflammation; Pancreas; Pancreatitis, Acute Necrotizing; ROC Curve

Abbreviations CTSI: CT severity index

Correspondence
Paraskevas S Brestas
Ag. Marinis 21, Melissia
15127 Athens
Greece
Phone: +30-210.804.6453/4120, +30-697.272.9154
Fax: +30-210.804.4120
E-mail: pbrestas@yahoo.gr