In order to efficiently eliminate the security shortcomings of the dynamic ID based remote user authentication scheme using smart cards (short for XJWM scheme) proposed by Xu et al.,a new multi server and multi factor anonymous remote authenticated key agreement protocol was presented,based on Diffie Hellman key agreement algorithm and biometrical authentication technology.The new protocol can not only overcome the security flaws of XJWM scheme,but also add smart card’s password and bio information authentication for the cardholder to avoid the smart card stolen attack.The security of the new protocol was proved by the improved BAN logic and the result showed that the new scheme can ensure the correctness of key agreement,key confidentiality,key freshness and mutual authentication.The security and performance analysis demonstrated that the proposed protocol provides better security without increasing too much computation overhead.