ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Quasi-continuous and discrete confidence rating scales for observer performance studies: Effects on ROC analysis

Hadjiiski, Lubomir*; Chan, Heang-Ping; Sahiner, Berkman; Helvie, Mark A.; Roubidoux, Marilyn A.
ACADEMIC RADIOLOGY2007-1DOI: 10.1016/j.acra.2006.09.048
19被引10阅读
引用收藏
分享

摘要

Rationale and Objectives. To examine the effects of the number of categories in the rating scale used in an observer experiment on the results of ROC analysis by a simulation study. Materials and Methods. We have previously evaluated the effects of computer-aided diagnosis on radiologists' characterization of malignant and benign breast masses in serial mammograms. The evaluation of the likelihood of malignancy was performed on a quasi-continuous (0-100 points) confidence rating scale. In this study, we simulated the use of discrete confidence rating scales with fewer number of categories and analyzed the results with receiver operating characteristic (ROC) methodology. The observers' estimates of the likelihood of malignancy were also mapped to BI-RADS assessments with five and seven categories and ROC analysis was per-formed. The area under the ROC curve and the partial area index obtained from ROC analysis of the different confidence rating scales were compared. Results. The fitted ROC curves and the performance indices do not change significantly when the confidence rating scales were varied from 6 to 101 points if the estimated operating points obtained directly from the data are distributed relatively evenly over the entire range of true-positive fraction (TPF) and false-positive fraction (FPF). The mapping of the likelihood of malignancy observer data to the seven-category BI-RADS assessment scale allowed reliable ROC analysis, whereas mapping to the five-category BI-RADS scale could cause erratic ROC curve fitting because of the lack of operating points in the mid-range or failure in ROC curve fitting because of data degeneration for some observers. Conclusion. ROC analysis of discrete confidence rating scales with few but relatively evenly distributed data points over the entire FPF and TPF range is comparable to that of a quasi-continuous rating scale. However, ROC analysis of discrete confidence rating scales with few and unevenly distributed data points may cause unreliable estimations.

学科与关键词

关键词
computer-aided diagnosiscontinuous and discrete confidence rating scalesROC observer studyclassificationmammography
相关
加载中…
  • Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine
    Obuchowski, Nancy A.*; Bullen, Jennifer A.Physics in Medicine and Biology, 2018, 63(7): 1-28.被引 298
  • Reader variability in mammography and its implications for expected utility over the population of readers and cases
    Wagner, RE*; Beam, CA; Beiden, SVMedical Decision Making, 2004, 24(6): 561-572.被引 36
  • A scaling transformation for classifier output based on likelihood ratio: Applications to a CAD workstation for diagnosis of breast cancer
    Horsch, Karla*; Pesce, Lorenzo L.; Giger, Maryellen L.; Metz, Charles E.; Jiang, YuleiMedical Physics, 2012, 39(5): 2787-2804.被引 6
查看全部 →

发表信息

论文状态
公开发表
期刊
ACADEMIC RADIOLOGY
发表日期
2007-01
卷期
第 14 卷, 第 1 期
页码
38-48
DOI
10.1016/j.acra.2006.09.048
语言
ENGLISH
成果收录
SCIE
资助标注
NCI NIH HHS [R01 CA095153, R01 CA095153-05, CA95153] Funding Source: Medline

作者

  • H

    Hadjiiski, Lubomir (通讯) (第一作者)

  • C

    Chan, Heang-Ping

查看全部 5 位作者收起作者
  • S

    Sahiner, Berkman

  • H

    Helvie, Mark A.

  • R

    Roubidoux, Marilyn A.

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417