In order to improve the efficiency and convergence speed of reinforcement learning algorithm, a method of heuristic reinforcement learning based on acquired path guiding knowledge (PHQL) was proposed.During the learning process using PHQL, embeded background knowledge was not needed to agent.While the agent updated the Q table in each episode，the path knowledge was also built，revised and optimized autonomously.After that,the lerning process was guided and accelerated by means of acquired path knowledge, which decreased the blindness of agent.In addition,three sorts of action selection methods of exploration,exploit and heuristic were analyzed, and also a practical method that action selection probilities changed over time was put forward.In a path planning environment, the PHQL was compared to the standard Q-learning and other relevant reinforcement learning algorithms.The experimental results showed that present methods accelerate the learning process obviously,and improve the convergence speed distinctly.
