The main purpose of this paper was to present a distributed speech interactive system built on LAN (Local area network), which integrated Automatic speech recognition (ASR), Text to speech (TTS) and Natural language perception (NLP) technologies to allow multiusers to access and converse with it concurrently. During a typical telephone based interaction, users could retrieve up-to-date stock information and call for somebody by directly speaking their name. Though there were far too many papers published to describe such system, most of them all focused on the algorithm on speech recognition rather than the architecture. Our paper was written to make for it and proposed an efficient and effective dynamic load balancing algorithm with sender active and non-preemptive concepts based on the distributed architecture and multithread concurrency in order to solve the loads allocating when multi-users accessed the system and gave rise to much burden to the servers. This paper was composed of five part: (1) an introduction about why such architecture was built; (2) an overview about the system; (3) proposed and implemented the algorithm; (4) evaluated and tested the system performance based on the architecture; and (5) a brief summary.