Multiagent reinforcement learning has shown its potential for tackling real world problems, like traffic. We consider the toll-based route choice problem, where self-interested agents repeatedly commute attempting to minimise their travel costs. In this paper, we introduce Generalised Toll-based Q-learning (GTQ-learning), a multiagent reinforcement learning algorithm capable of realigning agents' heterogeneous preferences over travel time and monetary expenses to obtain a system-efficient equilibrium. GTQ-learning also includes a mechanism to enforce agents to truthfully report their preferences. Our theoretical analysis and empirical results show that GTQ-learning minimises congestion on realistic road networks.