Dead Space was so scary for me that I couldn't play it.
I think the elements that haunted me most:
- Lighting - Key parts where lights would flicker on and off; Same goes for Bioshock
- Sound - I definitely had to turn the sound down sometimes
- Utterly insanely creepy monsters - Ok, this will vary from person to person but I lost it with the Dead Space Aliens. And, actually, the Bioshock Splicers were pretty damn creepy too.
Feeling of helplessness and being trapped are key. Like the game mechanics itself is one big haunted house.