Radiology doctors perform text-based image retrieval when they want to retrieve medical images. However, the accu-racy and efficiency of such retrieval cannot keep up with the requirements. An innovative algorithm is being proposed to re-trieve similar medical images. First, we extract the professional terms from the ontology structure and use them to annotate the CT images. Second, the semantic similarity matrix of ontology terms is calculated according to the structure of the ontol-ogy. Lastly, the corresponding semantic distance is calculated according to the marked vector, which contains different an-notations. We use 120 real liver CT images ( divided into six categories) of a top three-hospital to run the algorithm of the program. Result shows that the retrieval index"Precision"is 80. 81%, and the classification index"AUC( Area Under Curve)"under the"ROC curve"(Receiver Operating Characteristic)is 0. 945.