PMC:2948156 / 3599-15136 JSONTXT

{"project":"0_colil","denotations":[{"id":"20632009-19727744-50858","span":{"begin":496,"end":497},"obj":"19727744"},{"id":"20632009-19727744-50859","span":{"begin":1314,"end":1315},"obj":"19727744"},{"id":"20632009-16169887-50860","span":{"begin":5485,"end":5487},"obj":"16169887"},{"id":"20632009-16169887-50861","span":{"begin":5727,"end":5729},"obj":"16169887"},{"id":"20632009-17502262-50862","span":{"begin":6611,"end":6613},"obj":"17502262"},{"id":"20632009-9419629-50863","span":{"begin":8455,"end":8457},"obj":"9419629"},{"id":"20632009-1399456-50864","span":{"begin":9567,"end":9569},"obj":"1399456"},{"id":"20632009-1399456-50865","span":{"begin":10222,"end":10224},"obj":"1399456"},{"id":"20632009-15377098-50866","span":{"begin":10735,"end":10737},"obj":"15377098"},{"id":"20632009-1399456-50867","span":{"begin":11384,"end":11386},"obj":"1399456"}],"text":"Materials and methods\n\nImaging studies\n\nMammograms\nStandard cranio-caudal (CC) and medio-lateral oblique (MLO) views of each breast were available for all cases. If implants were present, displacement views were included. Original analog films (66 cases) or prints of digital films (36 cases) were used for review. All cases used in the study provided informed consent, and the protocol was approved by the Institutional Review Boards at each hospital, or The Western Institutional Review Board [9].\n\nAWBUs\nAutomated whole-breast ultrasound (AWBU) is a computer-based system for performing, recording, and reading whole-breast ultrasound examinations similar in appearance to 2D freehand imaging (SonoCine, Reno, NV). Images were collected with 7- to 12-MHz multi-frequency transducers. The transducer is attached to a computer-guided mechanical arm that acquires images in CC rows overlapping 7 to 10 mm insuring complete coverage of both breasts. Images are collected approximately 0.8 mm apart. The AWBU software creates a ciné loop for review of approximately 3,000 images, simulating real-time imaging. The Windows®-based reading station uses a high-definition 1,600 × 1,200 monitor and special software to increase cancers’ conspicuity. The AWBU procedure was described more fully in a previous publication [9].\n\nReaders\nTwelve board-certified breast radiologists who use breast ultrasound in their practices were recruited as readers for the trial. Remuneration for 3.5 days was at the prevailing US rate. Eleven readers were from the USA and one from Great Britain. Eleven had no experience with AWBU. One had reviewed limited AWBUs 8 years earlier during the developmental phase of the technology. No reader had foreknowledge of the positivity rate of the test set.\nEach reader had a 4-h tutorial with one author (KK) explaining the AWBU reading station operation. The readers reviewed and discussed approximately 12 AWBUs with known cancers, not part of the test set. They were not in the test set because either palpable findings were present or there were no concurrent mammograms. Nothing concerning the study was discussed, other than the use of the data form (Appendix A) and the number of cases to be reviewed.\n\nProcedure\nA set of 51 malignant cases (3 cases with bilateral cancers), including invasive and in situ cancer were collected for the trial (Table 1). Screening mammography and AWBU were performed within 2 months of each other. No cancers were associated with prospective palpable findings or symptoms suggestive of cancer. The mammograms were heterogeneously dense or extremely dense breast tissue (BIRADS 3 or 4) on the original reports. All imaging was performed from 2003 to 2008. The data set included all cases meeting the above criteria in the AWBU archives. Twelve cancers were included that were not prospectively reported on either imaging technique, but are visible in retrospect. Four of these became palpable within 1 year, three in more than 1 year; five were discovered in a subsequent screening round, three by AWBU only, and two by both AWBU and mammography.\nTable 1 Pathological diagnosis of 51 positive cases (54 cancers)\n≤1 cm \u003e1 to ≤2 cm \u003e2 cm Total\nDCIS 2 0 4 6\nIDC 17 19 5 41\nILC 3 2 1 6\nMixed IDC and ILC 0 1 0 1\nTotal 22 22 10 54\nDCIS ductal carcinoma in situ, IDC invasive ductal carcinoma, ILC invasive lobular carcinoma\nFifty-one normal cases performed from 2003 to 2008 were matched with each of the positive cases for the following factors: Facility\nDigital or analog mammogram\nUltrasound machine model\nAmerican breast cup size (A–DD)\nACR BIRADS breast density\nImplant (saline or silicone) and location (pre- or retropectoral)\nBreast cancer history\nAge\nThe normal case matching factors 1 to 7 closest to the age of the positive case was matched as the normal partner case. The mean difference in age between the positive case and its matched normal was 31 days.\nTesting occurred on a subsequent date at each reader’s own site with only the reader and a research assistant (monitor) present. The same monitor was present for all readers. She had no knowledge of the test set makeup, had no mammography or ultrasound training, reviewed the test data forms in real-time for completeness, and transferred the data to the study database.\nAt each test site 102 mammograms were placed on a film alternator in random order, generated once, and used for all readers. Excluding breaks, the test subject’s time for review was recorded. The upper half of a data form (Appendix A) was completed for each case, checked by the monitor, and entered into the database.\nFour questions were asked: Would you request further evaluation based on this mammogram, or recommend routine screening?\nWhere is/are the most suspicious (up to 3) lesions, identifying their location by breast and clock face position?\nWhat would be your prediction of the final ACR BIRADS after any needed diagnostic workup was completed?\nWhat is the reader’s confidence level that the woman has or does not have cancer (DMIST likelihood scale)?\nThe American College of Radiology Breast Imaging Reporting and Data System (BIRADS) is a seven-point scale (0 = incomplete, needs additional assessment; 1 = normal; 2 = benign; 3 = probably benign; 4a = possible malignancy; 4b = probable malignancy, or 5 = highly suggestive of malignancy) designed to categorize the results of mammography and other imaging studies [3, 11]. Scores from 1 to 5 were allowed. Similar to the DMIST [12], readers were asked to predict a BIRADS score before any diagnostic workup.\nThe DMIST likelihood rating is a seven-point scale to express the confidence of the diagnosis, and ranges from definitely not cancer to definitely cancer [3, 11, 12].\nA correct location response was recorded for an hour position marked within the half of the breast centered at the middle of the cancer.\nA true positive (TP) was recorded for mammography for any malignant case if ‘callback’ was marked for mammography and any correct tumor location was identified. A TP was recorded for mammography plus AWBU if ‘callback’ was marked on either or both halves of the form in the malignant cases, with at least one correct location identified. Thus, a correctly identified TP found with mammography would remain TP even were it not identified again on AWBU. AWBU findings could change the outcome to TP if a cancer was correctly identified with AWBU , but missed with mammography. We evaluated readings on a per-case (i.e., per-patient) basis rather than a per-score basis because screening serves as a “go no-go” gatekeeper for subsequent workup [13].\nA true negative (TN) was recorded for mammography for any normal case if ‘callback’ was not marked for mammography. A TN was recorded for mammography plus AWBU for any normal case if ‘callback’ was not marked on the second half of the form. This allowed the reader to reverse a callback for an asymmetric density seen mammographically but cleared by the AWBU as no suspicion. To validate TN cases, all cases were followed for at least 1 year or more.\nA false positive (FP) was recorded for mammography in two situations: Callback was marked for mammography in a normal case.\nCallback was marked for mammography in a cancer case, but none of the marked locations corresponded to the cancer.\nAn FP was recorded for mammography plus AWBU in the same two situations as above when callback was marked for AWBU. A false negative (FN) was recorded for mammography when callback was not marked in a cancer case in the mammography portion of the form. Similarly, an FN was recorded for mammography plus AWBU when callback was not marked in a cancer case in either portion of the form.\nThe 102 ABWUs were reviewed by readers on a review station brought by the research assistant acting as a monitor. They worked approximately 8 h daily for 3 days, with breaks at the readers’ choosing. The readers were given the corresponding mammograms with each AWBU and completed the second half of the data sheet with the knowledge from the mammogram-only evaluation available. The same questions were answered for AWBU and the reading time of each AWBU recorded.\n\nStatistical analysis\nAnalyses were conducted in a multi-reader multi-case (MRMC) framework where each reader screened all cases and each case contained both screening techniques. The MRMC design efficiently reduces the number of readers and cases needed to detect improvements across techniques [14]. Analyses appropriate for an MRMC design were chosen both to correctly model correlations between readings on the same case across readers and to estimate correctly standard errors. Unless specified otherwise, analyses were conducted in SAS software version 9.2 (SAS Institute Inc., Cary, NC, USA). We present F statistics, shown as F(numerator degrees of freedom, denominator degrees of freedom), and p values for comparisons between mammography plus AWBU and mammography alone.\nCases identified for further imaging were assessed by four binary measures: sensitivity = number of TP/number of cancer cases; specificity = number of TN/number of non-cancer cases; positive predictive value (PPV) = number of cancer cases/(number of TP + FP cases); and negative predictive value (NPV) = number of non-cancer cases/(number of FN + TN). Random-effect logistic regression models were used to test whether each binary measure differed significantly between mammography plus AWBU versus mammography alone. To account for the MRMC framework, we included random effects for readers and cases similar to the DBM model [15].\nAccuracy was assessed through BIRADS ratings and DMIST likelihood scores, comparing two commonly used indicators of accuracy between mammography plus AWBU versus mammography alone: areas under the curve (AUC) and figures of merit (FOM). The FOM incorporates information from each reader on the region of suspected malignancy, as well as their confidence level in the finding, incorporated in an AUC. Because it includes both confidence level and location accuracy, the FOM is more powerful than AUC in detecting differences between techniques. We include both analyses, as described below:\nAreas under the curve (AUC) were estimated in DBM MRMC 2.1 [15] (available from http://perception.radiology.uiowa.edu) using the trapezoidal/Wilcoxon method. Readers and patients were treated as random factors. We also present reader-averaged receiver operating characteristic (ROC) curves; average values were calculated from separate ROC analyses conducted on each reader in the PROC LOGISTIC procedure.\nFigures of merit (FOM) were estimated by using jackknife alternative free-response receiver operating characteristic methodology as implemented in JAFROC Version 1.0 [16] (available from http://www.devchakraborty.com). The FOM is defined as the probability that a cancer on an abnormal image is scored higher than a falsely marked location on a normal image and is analogous to the ROC curve; a higher FOM indicates improvement in reader performance.\n\nConfidence in identification of cases for further imaging\nWe used linear regression, comparing BIRADS ratings and DMIST likelihood scores across the two screening techniques among TP cases; mean ratings and scores are estimated by the regression for each screening technique. To account for the MRMC framework, we included random effects similar to the DBM model [15]; the model included a fixed effect for technique, classified as mammography plus AWBU or mammography alone, and random effects for readers and cases."}

{"project":"2_test","denotations":[{"id":"20632009-19727744-29359433","span":{"begin":496,"end":497},"obj":"19727744"},{"id":"20632009-19727744-29359434","span":{"begin":1314,"end":1315},"obj":"19727744"},{"id":"20632009-16169887-29359435","span":{"begin":5485,"end":5487},"obj":"16169887"},{"id":"20632009-16169887-29359436","span":{"begin":5727,"end":5729},"obj":"16169887"},{"id":"20632009-17502262-29359437","span":{"begin":6611,"end":6613},"obj":"17502262"},{"id":"20632009-9419629-29359438","span":{"begin":8455,"end":8457},"obj":"9419629"},{"id":"20632009-1399456-29359439","span":{"begin":9567,"end":9569},"obj":"1399456"},{"id":"20632009-1399456-29359440","span":{"begin":10222,"end":10224},"obj":"1399456"},{"id":"20632009-15377098-29359441","span":{"begin":10735,"end":10737},"obj":"15377098"},{"id":"20632009-1399456-29359442","span":{"begin":11384,"end":11386},"obj":"1399456"}],"text":"Materials and methods\n\nImaging studies\n\nMammograms\nStandard cranio-caudal (CC) and medio-lateral oblique (MLO) views of each breast were available for all cases. If implants were present, displacement views were included. Original analog films (66 cases) or prints of digital films (36 cases) were used for review. All cases used in the study provided informed consent, and the protocol was approved by the Institutional Review Boards at each hospital, or The Western Institutional Review Board [9].\n\nAWBUs\nAutomated whole-breast ultrasound (AWBU) is a computer-based system for performing, recording, and reading whole-breast ultrasound examinations similar in appearance to 2D freehand imaging (SonoCine, Reno, NV). Images were collected with 7- to 12-MHz multi-frequency transducers. The transducer is attached to a computer-guided mechanical arm that acquires images in CC rows overlapping 7 to 10 mm insuring complete coverage of both breasts. Images are collected approximately 0.8 mm apart. The AWBU software creates a ciné loop for review of approximately 3,000 images, simulating real-time imaging. The Windows®-based reading station uses a high-definition 1,600 × 1,200 monitor and special software to increase cancers’ conspicuity. The AWBU procedure was described more fully in a previous publication [9].\n\nReaders\nTwelve board-certified breast radiologists who use breast ultrasound in their practices were recruited as readers for the trial. Remuneration for 3.5 days was at the prevailing US rate. Eleven readers were from the USA and one from Great Britain. Eleven had no experience with AWBU. One had reviewed limited AWBUs 8 years earlier during the developmental phase of the technology. No reader had foreknowledge of the positivity rate of the test set.\nEach reader had a 4-h tutorial with one author (KK) explaining the AWBU reading station operation. The readers reviewed and discussed approximately 12 AWBUs with known cancers, not part of the test set. They were not in the test set because either palpable findings were present or there were no concurrent mammograms. Nothing concerning the study was discussed, other than the use of the data form (Appendix A) and the number of cases to be reviewed.\n\nProcedure\nA set of 51 malignant cases (3 cases with bilateral cancers), including invasive and in situ cancer were collected for the trial (Table 1). Screening mammography and AWBU were performed within 2 months of each other. No cancers were associated with prospective palpable findings or symptoms suggestive of cancer. The mammograms were heterogeneously dense or extremely dense breast tissue (BIRADS 3 or 4) on the original reports. All imaging was performed from 2003 to 2008. The data set included all cases meeting the above criteria in the AWBU archives. Twelve cancers were included that were not prospectively reported on either imaging technique, but are visible in retrospect. Four of these became palpable within 1 year, three in more than 1 year; five were discovered in a subsequent screening round, three by AWBU only, and two by both AWBU and mammography.\nTable 1 Pathological diagnosis of 51 positive cases (54 cancers)\n≤1 cm \u003e1 to ≤2 cm \u003e2 cm Total\nDCIS 2 0 4 6\nIDC 17 19 5 41\nILC 3 2 1 6\nMixed IDC and ILC 0 1 0 1\nTotal 22 22 10 54\nDCIS ductal carcinoma in situ, IDC invasive ductal carcinoma, ILC invasive lobular carcinoma\nFifty-one normal cases performed from 2003 to 2008 were matched with each of the positive cases for the following factors: Facility\nDigital or analog mammogram\nUltrasound machine model\nAmerican breast cup size (A–DD)\nACR BIRADS breast density\nImplant (saline or silicone) and location (pre- or retropectoral)\nBreast cancer history\nAge\nThe normal case matching factors 1 to 7 closest to the age of the positive case was matched as the normal partner case. The mean difference in age between the positive case and its matched normal was 31 days.\nTesting occurred on a subsequent date at each reader’s own site with only the reader and a research assistant (monitor) present. The same monitor was present for all readers. She had no knowledge of the test set makeup, had no mammography or ultrasound training, reviewed the test data forms in real-time for completeness, and transferred the data to the study database.\nAt each test site 102 mammograms were placed on a film alternator in random order, generated once, and used for all readers. Excluding breaks, the test subject’s time for review was recorded. The upper half of a data form (Appendix A) was completed for each case, checked by the monitor, and entered into the database.\nFour questions were asked: Would you request further evaluation based on this mammogram, or recommend routine screening?\nWhere is/are the most suspicious (up to 3) lesions, identifying their location by breast and clock face position?\nWhat would be your prediction of the final ACR BIRADS after any needed diagnostic workup was completed?\nWhat is the reader’s confidence level that the woman has or does not have cancer (DMIST likelihood scale)?\nThe American College of Radiology Breast Imaging Reporting and Data System (BIRADS) is a seven-point scale (0 = incomplete, needs additional assessment; 1 = normal; 2 = benign; 3 = probably benign; 4a = possible malignancy; 4b = probable malignancy, or 5 = highly suggestive of malignancy) designed to categorize the results of mammography and other imaging studies [3, 11]. Scores from 1 to 5 were allowed. Similar to the DMIST [12], readers were asked to predict a BIRADS score before any diagnostic workup.\nThe DMIST likelihood rating is a seven-point scale to express the confidence of the diagnosis, and ranges from definitely not cancer to definitely cancer [3, 11, 12].\nA correct location response was recorded for an hour position marked within the half of the breast centered at the middle of the cancer.\nA true positive (TP) was recorded for mammography for any malignant case if ‘callback’ was marked for mammography and any correct tumor location was identified. A TP was recorded for mammography plus AWBU if ‘callback’ was marked on either or both halves of the form in the malignant cases, with at least one correct location identified. Thus, a correctly identified TP found with mammography would remain TP even were it not identified again on AWBU. AWBU findings could change the outcome to TP if a cancer was correctly identified with AWBU , but missed with mammography. We evaluated readings on a per-case (i.e., per-patient) basis rather than a per-score basis because screening serves as a “go no-go” gatekeeper for subsequent workup [13].\nA true negative (TN) was recorded for mammography for any normal case if ‘callback’ was not marked for mammography. A TN was recorded for mammography plus AWBU for any normal case if ‘callback’ was not marked on the second half of the form. This allowed the reader to reverse a callback for an asymmetric density seen mammographically but cleared by the AWBU as no suspicion. To validate TN cases, all cases were followed for at least 1 year or more.\nA false positive (FP) was recorded for mammography in two situations: Callback was marked for mammography in a normal case.\nCallback was marked for mammography in a cancer case, but none of the marked locations corresponded to the cancer.\nAn FP was recorded for mammography plus AWBU in the same two situations as above when callback was marked for AWBU. A false negative (FN) was recorded for mammography when callback was not marked in a cancer case in the mammography portion of the form. Similarly, an FN was recorded for mammography plus AWBU when callback was not marked in a cancer case in either portion of the form.\nThe 102 ABWUs were reviewed by readers on a review station brought by the research assistant acting as a monitor. They worked approximately 8 h daily for 3 days, with breaks at the readers’ choosing. The readers were given the corresponding mammograms with each AWBU and completed the second half of the data sheet with the knowledge from the mammogram-only evaluation available. The same questions were answered for AWBU and the reading time of each AWBU recorded.\n\nStatistical analysis\nAnalyses were conducted in a multi-reader multi-case (MRMC) framework where each reader screened all cases and each case contained both screening techniques. The MRMC design efficiently reduces the number of readers and cases needed to detect improvements across techniques [14]. Analyses appropriate for an MRMC design were chosen both to correctly model correlations between readings on the same case across readers and to estimate correctly standard errors. Unless specified otherwise, analyses were conducted in SAS software version 9.2 (SAS Institute Inc., Cary, NC, USA). We present F statistics, shown as F(numerator degrees of freedom, denominator degrees of freedom), and p values for comparisons between mammography plus AWBU and mammography alone.\nCases identified for further imaging were assessed by four binary measures: sensitivity = number of TP/number of cancer cases; specificity = number of TN/number of non-cancer cases; positive predictive value (PPV) = number of cancer cases/(number of TP + FP cases); and negative predictive value (NPV) = number of non-cancer cases/(number of FN + TN). Random-effect logistic regression models were used to test whether each binary measure differed significantly between mammography plus AWBU versus mammography alone. To account for the MRMC framework, we included random effects for readers and cases similar to the DBM model [15].\nAccuracy was assessed through BIRADS ratings and DMIST likelihood scores, comparing two commonly used indicators of accuracy between mammography plus AWBU versus mammography alone: areas under the curve (AUC) and figures of merit (FOM). The FOM incorporates information from each reader on the region of suspected malignancy, as well as their confidence level in the finding, incorporated in an AUC. Because it includes both confidence level and location accuracy, the FOM is more powerful than AUC in detecting differences between techniques. We include both analyses, as described below:\nAreas under the curve (AUC) were estimated in DBM MRMC 2.1 [15] (available from http://perception.radiology.uiowa.edu) using the trapezoidal/Wilcoxon method. Readers and patients were treated as random factors. We also present reader-averaged receiver operating characteristic (ROC) curves; average values were calculated from separate ROC analyses conducted on each reader in the PROC LOGISTIC procedure.\nFigures of merit (FOM) were estimated by using jackknife alternative free-response receiver operating characteristic methodology as implemented in JAFROC Version 1.0 [16] (available from http://www.devchakraborty.com). The FOM is defined as the probability that a cancer on an abnormal image is scored higher than a falsely marked location on a normal image and is analogous to the ROC curve; a higher FOM indicates improvement in reader performance.\n\nConfidence in identification of cases for further imaging\nWe used linear regression, comparing BIRADS ratings and DMIST likelihood scores across the two screening techniques among TP cases; mean ratings and scores are estimated by the regression for each screening technique. To account for the MRMC framework, we included random effects similar to the DBM model [15]; the model included a fixed effect for technique, classified as mammography plus AWBU or mammography alone, and random effects for readers and cases."}

{"project":"TEST0","denotations":[{"id":"20632009-181-186-50858","span":{"begin":496,"end":497},"obj":"[\"19727744\"]"},{"id":"20632009-206-211-50859","span":{"begin":1314,"end":1315},"obj":"[\"19727744\"]"},{"id":"20632009-22-28-50860","span":{"begin":5485,"end":5487},"obj":"[\"16169887\"]"},{"id":"20632009-162-168-50861","span":{"begin":5727,"end":5729},"obj":"[\"16169887\"]"},{"id":"20632009-167-173-50862","span":{"begin":6611,"end":6613},"obj":"[\"17502262\"]"},{"id":"20632009-117-123-50863","span":{"begin":8455,"end":8457},"obj":"[\"9419629\"]"},{"id":"20632009-110-116-50864","span":{"begin":9567,"end":9569},"obj":"[\"1399456\"]"},{"id":"20632009-60-66-50865","span":{"begin":10222,"end":10224},"obj":"[\"1399456\"]"},{"id":"20632009-167-173-50866","span":{"begin":10735,"end":10737},"obj":"[\"15377098\"]"},{"id":"20632009-88-94-50867","span":{"begin":11384,"end":11386},"obj":"[\"1399456\"]"}],"text":"Materials and methods\n\nImaging studies\n\nMammograms\nStandard cranio-caudal (CC) and medio-lateral oblique (MLO) views of each breast were available for all cases. If implants were present, displacement views were included. Original analog films (66 cases) or prints of digital films (36 cases) were used for review. All cases used in the study provided informed consent, and the protocol was approved by the Institutional Review Boards at each hospital, or The Western Institutional Review Board [9].\n\nAWBUs\nAutomated whole-breast ultrasound (AWBU) is a computer-based system for performing, recording, and reading whole-breast ultrasound examinations similar in appearance to 2D freehand imaging (SonoCine, Reno, NV). Images were collected with 7- to 12-MHz multi-frequency transducers. The transducer is attached to a computer-guided mechanical arm that acquires images in CC rows overlapping 7 to 10 mm insuring complete coverage of both breasts. Images are collected approximately 0.8 mm apart. The AWBU software creates a ciné loop for review of approximately 3,000 images, simulating real-time imaging. The Windows®-based reading station uses a high-definition 1,600 × 1,200 monitor and special software to increase cancers’ conspicuity. The AWBU procedure was described more fully in a previous publication [9].\n\nReaders\nTwelve board-certified breast radiologists who use breast ultrasound in their practices were recruited as readers for the trial. Remuneration for 3.5 days was at the prevailing US rate. Eleven readers were from the USA and one from Great Britain. Eleven had no experience with AWBU. One had reviewed limited AWBUs 8 years earlier during the developmental phase of the technology. No reader had foreknowledge of the positivity rate of the test set.\nEach reader had a 4-h tutorial with one author (KK) explaining the AWBU reading station operation. The readers reviewed and discussed approximately 12 AWBUs with known cancers, not part of the test set. They were not in the test set because either palpable findings were present or there were no concurrent mammograms. Nothing concerning the study was discussed, other than the use of the data form (Appendix A) and the number of cases to be reviewed.\n\nProcedure\nA set of 51 malignant cases (3 cases with bilateral cancers), including invasive and in situ cancer were collected for the trial (Table 1). Screening mammography and AWBU were performed within 2 months of each other. No cancers were associated with prospective palpable findings or symptoms suggestive of cancer. The mammograms were heterogeneously dense or extremely dense breast tissue (BIRADS 3 or 4) on the original reports. All imaging was performed from 2003 to 2008. The data set included all cases meeting the above criteria in the AWBU archives. Twelve cancers were included that were not prospectively reported on either imaging technique, but are visible in retrospect. Four of these became palpable within 1 year, three in more than 1 year; five were discovered in a subsequent screening round, three by AWBU only, and two by both AWBU and mammography.\nTable 1 Pathological diagnosis of 51 positive cases (54 cancers)\n≤1 cm \u003e1 to ≤2 cm \u003e2 cm Total\nDCIS 2 0 4 6\nIDC 17 19 5 41\nILC 3 2 1 6\nMixed IDC and ILC 0 1 0 1\nTotal 22 22 10 54\nDCIS ductal carcinoma in situ, IDC invasive ductal carcinoma, ILC invasive lobular carcinoma\nFifty-one normal cases performed from 2003 to 2008 were matched with each of the positive cases for the following factors: Facility\nDigital or analog mammogram\nUltrasound machine model\nAmerican breast cup size (A–DD)\nACR BIRADS breast density\nImplant (saline or silicone) and location (pre- or retropectoral)\nBreast cancer history\nAge\nThe normal case matching factors 1 to 7 closest to the age of the positive case was matched as the normal partner case. The mean difference in age between the positive case and its matched normal was 31 days.\nTesting occurred on a subsequent date at each reader’s own site with only the reader and a research assistant (monitor) present. The same monitor was present for all readers. She had no knowledge of the test set makeup, had no mammography or ultrasound training, reviewed the test data forms in real-time for completeness, and transferred the data to the study database.\nAt each test site 102 mammograms were placed on a film alternator in random order, generated once, and used for all readers. Excluding breaks, the test subject’s time for review was recorded. The upper half of a data form (Appendix A) was completed for each case, checked by the monitor, and entered into the database.\nFour questions were asked: Would you request further evaluation based on this mammogram, or recommend routine screening?\nWhere is/are the most suspicious (up to 3) lesions, identifying their location by breast and clock face position?\nWhat would be your prediction of the final ACR BIRADS after any needed diagnostic workup was completed?\nWhat is the reader’s confidence level that the woman has or does not have cancer (DMIST likelihood scale)?\nThe American College of Radiology Breast Imaging Reporting and Data System (BIRADS) is a seven-point scale (0 = incomplete, needs additional assessment; 1 = normal; 2 = benign; 3 = probably benign; 4a = possible malignancy; 4b = probable malignancy, or 5 = highly suggestive of malignancy) designed to categorize the results of mammography and other imaging studies [3, 11]. Scores from 1 to 5 were allowed. Similar to the DMIST [12], readers were asked to predict a BIRADS score before any diagnostic workup.\nThe DMIST likelihood rating is a seven-point scale to express the confidence of the diagnosis, and ranges from definitely not cancer to definitely cancer [3, 11, 12].\nA correct location response was recorded for an hour position marked within the half of the breast centered at the middle of the cancer.\nA true positive (TP) was recorded for mammography for any malignant case if ‘callback’ was marked for mammography and any correct tumor location was identified. A TP was recorded for mammography plus AWBU if ‘callback’ was marked on either or both halves of the form in the malignant cases, with at least one correct location identified. Thus, a correctly identified TP found with mammography would remain TP even were it not identified again on AWBU. AWBU findings could change the outcome to TP if a cancer was correctly identified with AWBU , but missed with mammography. We evaluated readings on a per-case (i.e., per-patient) basis rather than a per-score basis because screening serves as a “go no-go” gatekeeper for subsequent workup [13].\nA true negative (TN) was recorded for mammography for any normal case if ‘callback’ was not marked for mammography. A TN was recorded for mammography plus AWBU for any normal case if ‘callback’ was not marked on the second half of the form. This allowed the reader to reverse a callback for an asymmetric density seen mammographically but cleared by the AWBU as no suspicion. To validate TN cases, all cases were followed for at least 1 year or more.\nA false positive (FP) was recorded for mammography in two situations: Callback was marked for mammography in a normal case.\nCallback was marked for mammography in a cancer case, but none of the marked locations corresponded to the cancer.\nAn FP was recorded for mammography plus AWBU in the same two situations as above when callback was marked for AWBU. A false negative (FN) was recorded for mammography when callback was not marked in a cancer case in the mammography portion of the form. Similarly, an FN was recorded for mammography plus AWBU when callback was not marked in a cancer case in either portion of the form.\nThe 102 ABWUs were reviewed by readers on a review station brought by the research assistant acting as a monitor. They worked approximately 8 h daily for 3 days, with breaks at the readers’ choosing. The readers were given the corresponding mammograms with each AWBU and completed the second half of the data sheet with the knowledge from the mammogram-only evaluation available. The same questions were answered for AWBU and the reading time of each AWBU recorded.\n\nStatistical analysis\nAnalyses were conducted in a multi-reader multi-case (MRMC) framework where each reader screened all cases and each case contained both screening techniques. The MRMC design efficiently reduces the number of readers and cases needed to detect improvements across techniques [14]. Analyses appropriate for an MRMC design were chosen both to correctly model correlations between readings on the same case across readers and to estimate correctly standard errors. Unless specified otherwise, analyses were conducted in SAS software version 9.2 (SAS Institute Inc., Cary, NC, USA). We present F statistics, shown as F(numerator degrees of freedom, denominator degrees of freedom), and p values for comparisons between mammography plus AWBU and mammography alone.\nCases identified for further imaging were assessed by four binary measures: sensitivity = number of TP/number of cancer cases; specificity = number of TN/number of non-cancer cases; positive predictive value (PPV) = number of cancer cases/(number of TP + FP cases); and negative predictive value (NPV) = number of non-cancer cases/(number of FN + TN). Random-effect logistic regression models were used to test whether each binary measure differed significantly between mammography plus AWBU versus mammography alone. To account for the MRMC framework, we included random effects for readers and cases similar to the DBM model [15].\nAccuracy was assessed through BIRADS ratings and DMIST likelihood scores, comparing two commonly used indicators of accuracy between mammography plus AWBU versus mammography alone: areas under the curve (AUC) and figures of merit (FOM). The FOM incorporates information from each reader on the region of suspected malignancy, as well as their confidence level in the finding, incorporated in an AUC. Because it includes both confidence level and location accuracy, the FOM is more powerful than AUC in detecting differences between techniques. We include both analyses, as described below:\nAreas under the curve (AUC) were estimated in DBM MRMC 2.1 [15] (available from http://perception.radiology.uiowa.edu) using the trapezoidal/Wilcoxon method. Readers and patients were treated as random factors. We also present reader-averaged receiver operating characteristic (ROC) curves; average values were calculated from separate ROC analyses conducted on each reader in the PROC LOGISTIC procedure.\nFigures of merit (FOM) were estimated by using jackknife alternative free-response receiver operating characteristic methodology as implemented in JAFROC Version 1.0 [16] (available from http://www.devchakraborty.com). The FOM is defined as the probability that a cancer on an abnormal image is scored higher than a falsely marked location on a normal image and is analogous to the ROC curve; a higher FOM indicates improvement in reader performance.\n\nConfidence in identification of cases for further imaging\nWe used linear regression, comparing BIRADS ratings and DMIST likelihood scores across the two screening techniques among TP cases; mean ratings and scores are estimated by the regression for each screening technique. To account for the MRMC framework, we included random effects similar to the DBM model [15]; the model included a fixed effect for technique, classified as mammography plus AWBU or mammography alone, and random effects for readers and cases."}

{"project":"MyTest","denotations":[{"id":"20632009-19727744-29359433","span":{"begin":496,"end":497},"obj":"19727744"},{"id":"20632009-19727744-29359434","span":{"begin":1314,"end":1315},"obj":"19727744"},{"id":"20632009-16169887-29359435","span":{"begin":5485,"end":5487},"obj":"16169887"},{"id":"20632009-16169887-29359436","span":{"begin":5727,"end":5729},"obj":"16169887"},{"id":"20632009-17502262-29359437","span":{"begin":6611,"end":6613},"obj":"17502262"},{"id":"20632009-9419629-29359438","span":{"begin":8455,"end":8457},"obj":"9419629"},{"id":"20632009-1399456-29359439","span":{"begin":9567,"end":9569},"obj":"1399456"},{"id":"20632009-1399456-29359440","span":{"begin":10222,"end":10224},"obj":"1399456"},{"id":"20632009-15377098-29359441","span":{"begin":10735,"end":10737},"obj":"15377098"},{"id":"20632009-1399456-29359442","span":{"begin":11384,"end":11386},"obj":"1399456"}],"namespaces":[{"prefix":"_base","uri":"https://www.uniprot.org/uniprot/testbase"},{"prefix":"UniProtKB","uri":"https://www.uniprot.org/uniprot/"},{"prefix":"uniprot","uri":"https://www.uniprot.org/uniprotkb/"}],"text":"Materials and methods\n\nImaging studies\n\nMammograms\nStandard cranio-caudal (CC) and medio-lateral oblique (MLO) views of each breast were available for all cases. If implants were present, displacement views were included. Original analog films (66 cases) or prints of digital films (36 cases) were used for review. All cases used in the study provided informed consent, and the protocol was approved by the Institutional Review Boards at each hospital, or The Western Institutional Review Board [9].\n\nAWBUs\nAutomated whole-breast ultrasound (AWBU) is a computer-based system for performing, recording, and reading whole-breast ultrasound examinations similar in appearance to 2D freehand imaging (SonoCine, Reno, NV). Images were collected with 7- to 12-MHz multi-frequency transducers. The transducer is attached to a computer-guided mechanical arm that acquires images in CC rows overlapping 7 to 10 mm insuring complete coverage of both breasts. Images are collected approximately 0.8 mm apart. The AWBU software creates a ciné loop for review of approximately 3,000 images, simulating real-time imaging. The Windows®-based reading station uses a high-definition 1,600 × 1,200 monitor and special software to increase cancers’ conspicuity. The AWBU procedure was described more fully in a previous publication [9].\n\nReaders\nTwelve board-certified breast radiologists who use breast ultrasound in their practices were recruited as readers for the trial. Remuneration for 3.5 days was at the prevailing US rate. Eleven readers were from the USA and one from Great Britain. Eleven had no experience with AWBU. One had reviewed limited AWBUs 8 years earlier during the developmental phase of the technology. No reader had foreknowledge of the positivity rate of the test set.\nEach reader had a 4-h tutorial with one author (KK) explaining the AWBU reading station operation. The readers reviewed and discussed approximately 12 AWBUs with known cancers, not part of the test set. They were not in the test set because either palpable findings were present or there were no concurrent mammograms. Nothing concerning the study was discussed, other than the use of the data form (Appendix A) and the number of cases to be reviewed.\n\nProcedure\nA set of 51 malignant cases (3 cases with bilateral cancers), including invasive and in situ cancer were collected for the trial (Table 1). Screening mammography and AWBU were performed within 2 months of each other. No cancers were associated with prospective palpable findings or symptoms suggestive of cancer. The mammograms were heterogeneously dense or extremely dense breast tissue (BIRADS 3 or 4) on the original reports. All imaging was performed from 2003 to 2008. The data set included all cases meeting the above criteria in the AWBU archives. Twelve cancers were included that were not prospectively reported on either imaging technique, but are visible in retrospect. Four of these became palpable within 1 year, three in more than 1 year; five were discovered in a subsequent screening round, three by AWBU only, and two by both AWBU and mammography.\nTable 1 Pathological diagnosis of 51 positive cases (54 cancers)\n≤1 cm \u003e1 to ≤2 cm \u003e2 cm Total\nDCIS 2 0 4 6\nIDC 17 19 5 41\nILC 3 2 1 6\nMixed IDC and ILC 0 1 0 1\nTotal 22 22 10 54\nDCIS ductal carcinoma in situ, IDC invasive ductal carcinoma, ILC invasive lobular carcinoma\nFifty-one normal cases performed from 2003 to 2008 were matched with each of the positive cases for the following factors: Facility\nDigital or analog mammogram\nUltrasound machine model\nAmerican breast cup size (A–DD)\nACR BIRADS breast density\nImplant (saline or silicone) and location (pre- or retropectoral)\nBreast cancer history\nAge\nThe normal case matching factors 1 to 7 closest to the age of the positive case was matched as the normal partner case. The mean difference in age between the positive case and its matched normal was 31 days.\nTesting occurred on a subsequent date at each reader’s own site with only the reader and a research assistant (monitor) present. The same monitor was present for all readers. She had no knowledge of the test set makeup, had no mammography or ultrasound training, reviewed the test data forms in real-time for completeness, and transferred the data to the study database.\nAt each test site 102 mammograms were placed on a film alternator in random order, generated once, and used for all readers. Excluding breaks, the test subject’s time for review was recorded. The upper half of a data form (Appendix A) was completed for each case, checked by the monitor, and entered into the database.\nFour questions were asked: Would you request further evaluation based on this mammogram, or recommend routine screening?\nWhere is/are the most suspicious (up to 3) lesions, identifying their location by breast and clock face position?\nWhat would be your prediction of the final ACR BIRADS after any needed diagnostic workup was completed?\nWhat is the reader’s confidence level that the woman has or does not have cancer (DMIST likelihood scale)?\nThe American College of Radiology Breast Imaging Reporting and Data System (BIRADS) is a seven-point scale (0 = incomplete, needs additional assessment; 1 = normal; 2 = benign; 3 = probably benign; 4a = possible malignancy; 4b = probable malignancy, or 5 = highly suggestive of malignancy) designed to categorize the results of mammography and other imaging studies [3, 11]. Scores from 1 to 5 were allowed. Similar to the DMIST [12], readers were asked to predict a BIRADS score before any diagnostic workup.\nThe DMIST likelihood rating is a seven-point scale to express the confidence of the diagnosis, and ranges from definitely not cancer to definitely cancer [3, 11, 12].\nA correct location response was recorded for an hour position marked within the half of the breast centered at the middle of the cancer.\nA true positive (TP) was recorded for mammography for any malignant case if ‘callback’ was marked for mammography and any correct tumor location was identified. A TP was recorded for mammography plus AWBU if ‘callback’ was marked on either or both halves of the form in the malignant cases, with at least one correct location identified. Thus, a correctly identified TP found with mammography would remain TP even were it not identified again on AWBU. AWBU findings could change the outcome to TP if a cancer was correctly identified with AWBU , but missed with mammography. We evaluated readings on a per-case (i.e., per-patient) basis rather than a per-score basis because screening serves as a “go no-go” gatekeeper for subsequent workup [13].\nA true negative (TN) was recorded for mammography for any normal case if ‘callback’ was not marked for mammography. A TN was recorded for mammography plus AWBU for any normal case if ‘callback’ was not marked on the second half of the form. This allowed the reader to reverse a callback for an asymmetric density seen mammographically but cleared by the AWBU as no suspicion. To validate TN cases, all cases were followed for at least 1 year or more.\nA false positive (FP) was recorded for mammography in two situations: Callback was marked for mammography in a normal case.\nCallback was marked for mammography in a cancer case, but none of the marked locations corresponded to the cancer.\nAn FP was recorded for mammography plus AWBU in the same two situations as above when callback was marked for AWBU. A false negative (FN) was recorded for mammography when callback was not marked in a cancer case in the mammography portion of the form. Similarly, an FN was recorded for mammography plus AWBU when callback was not marked in a cancer case in either portion of the form.\nThe 102 ABWUs were reviewed by readers on a review station brought by the research assistant acting as a monitor. They worked approximately 8 h daily for 3 days, with breaks at the readers’ choosing. The readers were given the corresponding mammograms with each AWBU and completed the second half of the data sheet with the knowledge from the mammogram-only evaluation available. The same questions were answered for AWBU and the reading time of each AWBU recorded.\n\nStatistical analysis\nAnalyses were conducted in a multi-reader multi-case (MRMC) framework where each reader screened all cases and each case contained both screening techniques. The MRMC design efficiently reduces the number of readers and cases needed to detect improvements across techniques [14]. Analyses appropriate for an MRMC design were chosen both to correctly model correlations between readings on the same case across readers and to estimate correctly standard errors. Unless specified otherwise, analyses were conducted in SAS software version 9.2 (SAS Institute Inc., Cary, NC, USA). We present F statistics, shown as F(numerator degrees of freedom, denominator degrees of freedom), and p values for comparisons between mammography plus AWBU and mammography alone.\nCases identified for further imaging were assessed by four binary measures: sensitivity = number of TP/number of cancer cases; specificity = number of TN/number of non-cancer cases; positive predictive value (PPV) = number of cancer cases/(number of TP + FP cases); and negative predictive value (NPV) = number of non-cancer cases/(number of FN + TN). Random-effect logistic regression models were used to test whether each binary measure differed significantly between mammography plus AWBU versus mammography alone. To account for the MRMC framework, we included random effects for readers and cases similar to the DBM model [15].\nAccuracy was assessed through BIRADS ratings and DMIST likelihood scores, comparing two commonly used indicators of accuracy between mammography plus AWBU versus mammography alone: areas under the curve (AUC) and figures of merit (FOM). The FOM incorporates information from each reader on the region of suspected malignancy, as well as their confidence level in the finding, incorporated in an AUC. Because it includes both confidence level and location accuracy, the FOM is more powerful than AUC in detecting differences between techniques. We include both analyses, as described below:\nAreas under the curve (AUC) were estimated in DBM MRMC 2.1 [15] (available from http://perception.radiology.uiowa.edu) using the trapezoidal/Wilcoxon method. Readers and patients were treated as random factors. We also present reader-averaged receiver operating characteristic (ROC) curves; average values were calculated from separate ROC analyses conducted on each reader in the PROC LOGISTIC procedure.\nFigures of merit (FOM) were estimated by using jackknife alternative free-response receiver operating characteristic methodology as implemented in JAFROC Version 1.0 [16] (available from http://www.devchakraborty.com). The FOM is defined as the probability that a cancer on an abnormal image is scored higher than a falsely marked location on a normal image and is analogous to the ROC curve; a higher FOM indicates improvement in reader performance.\n\nConfidence in identification of cases for further imaging\nWe used linear regression, comparing BIRADS ratings and DMIST likelihood scores across the two screening techniques among TP cases; mean ratings and scores are estimated by the regression for each screening technique. To account for the MRMC framework, we included random effects similar to the DBM model [15]; the model included a fixed effect for technique, classified as mammography plus AWBU or mammography alone, and random effects for readers and cases."}