According to the social intelligence hypothesis, cooperation is considered a keycomponent of intelligence and is required to solve a wide range of problems, fromeveryday challenges like scheduling meetings to global challenges like mitigatingclimate change and providing humanitarian aid. Extending the ability for artificialintelligence (AI) to cooperate well is critical as AI becomes more prevalent in ourlives. In recent years, multi-agent reinforcement learning (MARL) has emergedas a powerful approach to model and analyse the problem of cooperation amongartificial agents. In this paper, we investigate the impact of communication oncooperation among reinforcement learning agents in social dilemmas. These aresettings in which the short-term individual interests are in conflict with the longterm collective ones, thus each individual profits from defecting, but the overallgroup would benefit if everyone cooperates. We particularly focus on a temporally and spatially extended Stag-Hunt-like social dilemma that models animalforaging behaviour using principles from Optimal Foraging Theory. We proposea method for communication that combines a graph-based attention mechanismwith deep reinforcement learning methods. Additionally, we examine severalfacets of communication, including the effects of the communication topologyand the communication range. We find that greater cooperative behaviour canbe achieved through graph-based communication using reinforcement learning insocial dilemmas. Additionally, we find that during foraging, local communicationpromotes better cooperation than long-distance communication. Finally, we visualise and investigate the learned attention weights and explain how agents processcommunications from other agents.