Flower pollination algorithm (FPA) is one of the well-known evolutionary techniques used extensively to solve optimization problems. Despite its efficiency and wide use, the identical search behaviors may lead the algorithm to converge to local optima. In this paper, an adaptive FPA based on chaotic map (CAFPA) is proposed. The proposed algorithm first used the ergodicity of the logistic chaos mechanism, and chaotic mapping of the initial population to make the initial iterative population more evenly distributed in the solution space. Then at the self-pollination stage, the over-random condition of the gamete renewal was improved, the traction force of contemporary optimal position was given, and adaptive logarithmic inertia weight was introduced to adjust the proportion between the contemporary pollen position and disturbance to improve the performance of the algorithm. By comparing the new algorithm with three famous optimization algorithms, the accuracy and performance of the proposed approach are evaluated by 14 well-known benchmark functions. Statistical comparisons of experimental results show that CAFPA is superior to FPA, PSO, and BOA in terms of convergence speed and robustness.