Based on ServiceBSP model, a hierarchical resource load balancing algorithm with Multi-Agent is put forward in this paper which achieves the goal of dynamic load balancing and favorable Fault-tolerant. The algorithm calculates the load value according to the attributes of resource and scheduling tasks relies on the load value, while updating the load information dynamically depending on Multi-Agent. The method avoids frequent communications on load information. Furthermore, the paper introduces the function of agents, relations and communications among agents in details. Finally, by comparing response time and distribution of load using proposed method with other available methods such as without no load balancing and load balancing only giving regards to CPU, the experimental simulation shows that the load balancing based on Multi-Agent possesses superior performance on response time and load balancing.