Second person is like a second world country—overlooked and under-appreciated, and no one considers themselves to be one.
But what makes second person such a despised literary form? Well, in a story, this didn’t actually happen to you, and so you’re repeatedly reminded that it is a lie, a non-truth, a fiction.
Secondly, I believe there are difficulties with the rhythm of second person. “You” repeats too often, and there really isn’t a synonym for it. The “I” has me and mine, He/She has their names, but you will always have to be you. “You look about yourself, gauging the incline of the foredeck. Something is amiss, you think, but try not to dwell on your worries, turning about to find a lifeboat, then thinking about the others. You consider searching the hold for any stragglers, when a loud crack! shatters your concentration. Just then: the great white whale rises before you.”
The third drawback is when someone, in real life, begins talking to you in second person, they are basically upstaging your own interpretations and actions, and that’s patronizing and infuriating. And they deserve a kick in the shins until they return to speaking of themselves. Even the royal we is stronger than the you.
But—despite all this, I don’t think you should be dismissed out of hand. There’s something to be mined through the second person, especially if the you has a distinct personality, which becomes revealed through the trials and instances detailed in the stream of prose.