We had faced the same issue once.
What we found that the size of that particular xaml ( toolbar xaml in your case) was bit big.So, when we clicked near the toolbar (we were assuming that we click on the map), click was actually made on the xaml and not on the map.
So I would suggest to check the size of your Toolbar XAML.I think it is not allowing map click due to its size.
HTH.