The subtle brand of horror that makes "Paranormal Activity" work so well was born (or at least perfected) in Robert Wise's "The Haunting." Instead of a haunted house with a lot of ghosts and murder, most of the scares in "The Haunting" are no more complex than a weird sound in the middle of the night, or a strange jolting of the walls. And most people know that those are the scariest things of all.
"The Blair Witch Project"
Although this horror movie doesn't have the best reputation in the world since its phenom-like release back in 1999. But it's the progenitor of the faux-documentary-style horror movie that "Paranormal Activity" represents, and if you can get past all the "SNL"-parodied moments it's still a pretty scary movie. They're both fake documentaries -what else do you want?
"The Last Exorcism"
Speaking of horror faux-documentaries, this is another good one that also features a demonic villain like "Paranormal Activity." Also like "Paranormal Activity," it starts out in almost a humorous vein, following a charlatan preacher as he seeks to perform an "exorcism" in order to prove the whole enterprise is a sham. Unfortunately, the possessed girl he comes across to make this point might actually be possessed. Good for business, bad for safety.
One thing that gives "Paranormal Activity" a unique flavor is that the horror doesn't ramp up to its maximum point until almost the end, so the audience isn't even sure anything supernatural is really happening until it's almost over. This is a quality it shares with "The Omen," which could almost be interpreted as not having anything supernatural in it at all, if you can buy a lot of terrifying coincidences.