I was going about it the wrong way ig. I couldn't see him unless I went down to the middle ground. Although I can confirm that if you have not found the ghost memories and you join the foreteam to kill him you still get the cult ghost.