Cannot interact with Evertree Map Device

When using a controller, I cannot properly interact with the Evertree Map Device. The prompt to toggle tracking is all I can interact with. Even with Keyboard/Mouse it is very difficult to find a spot to click on that actually opens the world map.