Well hunters weren't official guardians in the early days. They actually lived outside the cities and used the Traveler's Light to better withstand and dispense justice in the harsh and lawless frontier. So who knows which came first. After the tower and vanguard was created, they saw the potential in the hunters and officially gave them the position of scouters, mappers and explorers. They happily accepted since they were most comfortable out in the frontier where there were no restrictions compared to the battle-driven titans and knowledge-seeking warlocks who preferred defending the city and learning more about the Light in their arcane studies. We all have our part to play