Cavitation erosion degrades the performance and reliability of hydraulic machinery. Selective laser melting (SLM) is a type of metal additive manufacturing technology that can fabricate metal parts directly and provide lightweight design in various industrial applications. However, the cavitation erosion behaviors of SLM-fabricated parts have rarely been studied. In this study, SLM 316L stainless steel samples were fabricated via SLM technology considering the scanning strategy, scanning speed, laser power, and build orientation. The effect of the process parameters on the cavitation erosion resistance of the SLM-fabricated 316L stainless steel samples was illustrated using an ultrasonic vibratory cavitation system. The mass loss and surface topography were employed to evaluate the surface cavitation damage of the SLM-fabricated 316L stainless steel samples after the cavitation test. The cavitation damage mechanism of the SLM-fabricated samples was discussed. The results show that the degree of cavitation damage of the sample fabricated via SLM with a few defects, anisotropic build direction, and columnar microstructure is significantly decreased. Defects such as pores, which are attributed to low laser power and high scanning speed, may severely aggravate the cavitation damage of the SLM-fabricated samples. The sample fabricated via SLM with a low laser power and exposure time exhibited the highest porosity and poor cavitation erosion resistance. The cellular structures are more prone to cavitation damage compared with the columnar structures. A sample with a high density of grain boundaries will severely suffer cavitation damage.