With respect m dialectal-accented speech recognition, dialect-specific lexicon adaptation is an indispensable component to improve the performance. In this paper, a phone-based confusion matrix is adopted to obtain dialect-specific pronunciation variants. A weighting method, improved context-dependent weighting (ICDW) is proposed to characterize the pronunciation probability precisely by taking both the surface form left-context-dependency and the base form left-context-dependency into account. To make a much robust lexicon, a pruning criterion, syllable-dependent pruning (SDP), is also proposed which achieves the most effective result. In summary, the dialect-specific dialect adaptation reduces a 2.9% absolute and a 3.6% absolute in syllable error rate (SER) respectively on read speech and spontaneous speech from Shanghai-accented speakers.