Exotics don't get "worse" when more people have them. It's not like the damage gets reduced when someone purchases an exotic from Xur. Your exotic is exactly like their exotic. And guess what? Every single person who has been playing this game and wanting those weapons and armor has technically been "grinding" for them. Just because they eventually drop for you doesn't mean you deserved them any more or they deserved them any less, everyone is working just as hard as everyone else.