Spatial Attention for Autonomous Decision-making in Highway Scene

Automated decision making is still a significant challenge to realize fully autonomous driving. A common method that encoding surrounding vehicles in a grid map is used to describe observation space for decision making algorithm. It preserves vehicles spatial characteristics. But commonly in human driving, distinct position and speed surrounding vehicles contribute differently to make decision. We introduce a spatial attention module to calculate weights for each vehicle and integrate the attention mechanism into Deep Q network to make decision actions. The agent, ego vehicle, is trained in a simulated highway environment. Simulation results show the proposed method can get significant performance gains compared with other deep reinforcement learning methods by using two kinds of metrics.