I think when I attributed part of the blame for not being able to inflict horror on mechanics I misspoke; that would imply the system was ever meant to convey or generate it. But that's not the case, not even for those which advertise themselves as "games of personal horror" - they are not. The moment names and abilities are assigned to things the veil of mystery is gone because there's a framework for what it is or could be.
Is something from beyond the grave stirring in your kitchen? Well, man, there's a best case and a worst case scenario here; you either have what it takes to stop it ("pft, I got Death 3") or you don't and you'll suffer whatever mechanical fate is in store (death, insanity). It's cut and dried, since almost every system that I've ever seen including Cthulhu variants is designed with adventuring in mind; you got your numbers and your opposition has its numbers. There's an encounter and a resolution.
My idea of horror (which of course each player has a different one) is based on the unknown, the unthinkable. There's no resolution, there can't be one because what's happening is so unfathomable. There's no opposition because the things that are happening are phenomena, manifestations, not monsters with motives and goals; even if you manage to find that rite which silences the indian graveyard under your house you're not safe because you don't know what just happened, or why it started or ended. You don't know if the wards you're hanging outside your window do anything. All you know is you're this tiny fleshbag who witnessed something massively bigger than themselves and you're scared shitless because of it.
But how to channel this into a plot... I don't know how. It doesn't help that PCs tend to be terribly competent on their own ("I roll int+occult, can I tell what those signs on the ceiling mean?") or that they can call for backup and get someone who believes them. PCs never need stand alone. They never have to wonder if they aren't losing it because when they all know a whole lot of supernatural stuff already exists why doubt this latest insanity also does? Sure man, I've seen ghosts, I've fought zombies but what the hell are you saying, spirits? Those don't exist, are you nuts?
But I'd love to. I'd love to run an actually scary story.