Q-Learning-Based Energy-Aware Route Planning for Electric Vehicles on Real Road Networks with Charging Constraints

Abstract

The electric vehicle (EV) routing problem requires not only minimizing the travel distance but also considering energy constraints and the spatial distribution of charging stations. In this work, we propose Q-learning and hierarchical Q-learning (HRL) based agents trained to generate energy-aware routes on real road networks. The environment is modeled as graph structures obtained from Berlin, Istanbul, and Ankara cities using OpenStreetMap data. Each state is represented by a 16-dimensional vector that includes distance to the destination, battery level, number of neighbors, nearby charging stations, and final reward dynamics. The reward function is a multi-component structure that incentivizes approaching the destination, penalizes unnecessary loops and backtracking, and rewards smart charging decisions. Experimental results show that HRL demonstrates superiority up to 25% in compact cities (Berlin), while Q-learning performs better in large areas and resource-constrained environments. Both approaches deviate from the shortest path only when necessary due to energy constraints and their performance is evaluated by metrics such as total reward, route length and number of charges. © 2025 IEEE.