Do Neural Information Extraction Algorithms Generalize Across Institutions?
JCO CLINICAL CANCER INFORMATICS
Authors: Santus, Enrico; Li, Clara; Yala, Adam; Peck, Donald; Soomro, Rufina; Faridi, Naveen; Mamshad, Isra; Tang, Rong; Lanahan, Conor R.; Barzilay, Regina; Hughes, Kevin
Abstract
PURPOSE Natural language processing (NLP) techniques have been adopted to reduce the curation costs of electronic health records. However, studies have questioned whether such techniques can be applied to data from previously unseen institutions. We investigated the performance of a common neural NLP algorithm on data from both known and heldout (ie, institutions whose data were withheld from the training set and only used for testing) hospitals. We also explored how diversity in the training data affects the system's generalization ability. METHODS We collected 24,881 breast pathology reports from seven hospitals and manually annotated them with nine key attributes that describe types of atypia and cancer. We trained a convolutional neural network (CNN) on annotations from either only one (CNN1), only two (CNN2), or only four (CNN4) hospitals. The trained systems were tested on data from five organizations, including both known and heldout ones. For every setting, we provide the accuracy scores as well as the learning curves that show how much data are necessary to achieve good performance and generalizability. RESULTS The system achieved a cross-institutional accuracy of 93.87% when trained on reports from only one hospital (CNN1). Performance improved to 95.7% and 96%, respectively, when the system was trained on reports from two (CNN2) and four (CNN4) hospitals. The introduction of diversity during training did not lead to improvements on the known institutions, but it boosted performance on the heldout institutions. When tested on reports from heldout hospitals, CNN4 outperformed CNN1 and CNN2 by 2.13% and 0.3%, respectively. CONCLUSION Real-world scenarios require that neural NLP approaches scale to data from previously unseen institutions. We show that a common neural NLP algorithm for information extraction can achieve this goal, especially when diverse data are used during training. JCO Clin Cancer Inform. (C) 2019 by American Society of Clinical Oncology
False-Positive Reduction on Lung Nodules Detection in Chest Radiographs by Ensemble of Convolutional Neural Networks
IEEE ACCESS
Authors: Li, Chaofeng; Zhu, Guoce; Wu, Xiaojun; Wang, Yuanquan
Abstract
Aiming at the problem that traditional lung nodules detection method can only get low sensitivities with a lot of false positives, we propose a new framework of ensemble of convolutional neural networks (E-CNNs) and use it to significantly reduce the number of false positive on lung nodules detection in chest radiographs (CXRs). First, unsharp mask technique is used to enhance the nodules in the CXRs. Then, we cut patches in the 229 x 229 image containing or not containing nodule from the enhanced CXRs, which correspond to the positive and negative samples. Third, three optimized CNNs of different input sizes and different depths, namely, CNN1, CNN2 and CNN3, are constructed to detect lung nodule separately, and their input sizes are 12 x 12, 32 x 32, and 60 x 60, and the number of layers are 5, 7, and 9, separately. Finally, a logical AND operator is used to fuse the results of CNN1, CNN2, and CNN3, and E-CNNs are constructed for detecting lung nodules. Our experimental results on the Japanese Society of Radiological Technology database show our proposed E-CNNs attain a sensitivity of 94% and 84% with an average of 5.0 false-positives (FPs) per image and 2.0 FPs per image, respectively, in a five-fold cross-validation test, which far outperforms the state of the art.