I use an RX 5700 XT and had the same issues. I found that using the old AMD 20.4.2 drivers fixed the performance issues. However, if you go back to this old driver you may run into an issue where the game can hang, black screen and crash when you alt-tab to another application. The fix I found for this was to run the game in windowed full screen. Now my game doesn't have performance issues and the game doesn't hang or crash when I wanna use DIM or Discord while playing Destiny.