Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary information Supplementary information is available for this paper at 10.1038/s42003-020-01535-7. Acknowledgements We would like to thank the Ministry of Science and Technology of the People’s Republic of China (Grant No. 2017YFB1400100) and the National Natural Science Foundation of China (Grant No. 61876059) for their support. Author contributions S.L. and Y.G. contributed significantly to the conception of the study. S.L. designed the network and conduct the experiments. S.L. and Y.G. provided, marked, and analyzed the experimental results. H.L. contributed with valuable discussions and analyzed the experimental results. Y.G. supported and supervised the work and contributed with valuable scientific advice as the corresponding author. X.G. collected the medical image data from Youan Hospital and contributed with valuable discussions. H.L. and L.L. provided analysis and interpretation of the medical data. Z.W., M.L., and L.T. contributed with valuable discussions and revisions. All authors contributed to writing this manuscript. Data availability The data sets used in this study (named Hybrid Datasets) are composed of public data sets from four public data repositories and a hospital data set provided by the cooperative hospital (Beijing Youan hospital). The four public data repositories are Covid-ChestXray-Dataset (CCD), Rsna-pneumonia-detection-challenge (RSNA), Lung Nodule Analysis 2016 (LUNA16), and Images of COVID-19 positive and negative pneumonia patients (ICNP), respectively. Full data of the Hybrid Data sets are available at Figshare (10.6084/m9.figshare.13235009). Code availability We used standard software packages as described in the “Methods” section. The implementation details of the proposed framework can be downloaded from https://github.com/SHERLOCKLS/Detection-of-COVID-19-from-medical-images. Competing interests The authors declare no competing interests.