Lore wise is because Ahamkaras could bend space and time....Warlocks were unaffected because of their high intelligence which allowed us to hunt them down. Hunters went after smaller ones who didn't have the ability.