As a fine-grained classification problem, food image classification faces many difficulties in the specific implementation. Different countries and regions have different eating habits. In particular, Asian food images have a complicated structure, and the related classification methods are still very scarce. There is an urgent need to develop a feature extraction and fusion scheme based on the characteristics of Asian food images. To solve the above problems, we proposed an image classification model SLGC (SURF-Local and Global Color) that combines image segmentation and feature fusion. By studying the unique structure of Asian foods, the color features of the images are merged into the representation vectors in the local and global dimensions, respectively, thereby further enhancing the effect of feature extraction. The experimental results show that the SLGC model can express the intrinsic characteristics of Asian food images more comprehensively and improve classification accuracy.