In this study, we identified the RBD fragment in SARS-CoV-2 S protein and found that the recombinant RBD protein bound strongly to human ACE2 (hACE2) and bat ACE2 (bACE2) receptors. In addition, it blocked the entry of SARS-CoV-2 and SARS-CoV into their respective hACE2-expressing cells, suggesting that it may serve as a viral attachment inhibitor against SARS-CoV-2 and SARS-CoV infection. Moreover, we demonstrated that SARS-CoV RBD-specific polyclonal antibodies cross-reacted with SARS-CoV-2 RBD protein and inhibited SARS-CoV-2 entry into hACE2-expressing cells. We have also shown that SARS-CoV RBD-specific polyclonal antibodies could cross-neutralize SARS-CoV-2 pseudovirus infection, suggesting the potential to develop SARS-CoV RBD-based vaccine for prevention of infection by SARS-CoV-2 and SARS-CoV.