Unpredictable story twists comes in three forms.
2. Deus ex machina
The second never fails to piss me off, even when used ironically (I'm looking at you, Adaptation). I don't care if it was good enough for Greek tragedies. If I wanted a random act to solve all the story's problems, I would have asked my five year old niece to weave a tale.
This leaves us with Delightful and Dud. Please pardon this spoilerific post as I break down the most famous unpredictable moments in movie history and how they cross the line for me.
1. The Usual Suspects
A gimp describing to the police how a mysterious and nefarious mob boss toyed with and took down his gang turns out to be the mastermind he's been describing.
2. Planet of the Apes (original)
Human trying to escape back to earth from a planet overrun by apes realizes he is on earth...in the future.
3. The Empire Strikes Back
Whiny teenager learns the big bad guy he's trying to take down is his father.
4. The Sixth Sense
Psychologist trying to help a kid overcome his ghostly encounters realizes he himself is one of the ghostly encounters.
A man with no long term memory avenging his life will never remember he's already had his revenge but will never remember it and thus never know peace.
1. Life of David Gale
Man allows himself to be wrongly executed for murder to make a point about the death penalty.
2. The Village
Simple villagers learn their life is a grand experiment to leave behind the evils of the modern world.
Guy describing his attempt to stay alive at the hands of a demon is oops, really the demon himself.
4. The Forgotten
Woman tries to find her missing son who everyone insists didn't exist and realizes she's part of an alien sociological experiment to test parental bonds.
Ten strangers die off one by one in a motel and turn out all to be the imaginings of someone with schizophrenia.
Here's my call on the difference between the two: the twist needs to be on a character, not on the audience.
While The Usual Suspects gives us an aha moment, Kaiser Soze was pulling a fast one on an arrogant police detective, not us, and we feel along for the ride as the lame stuttering character proves to be the clever one.
In Planet of the Apes, Charlton Heston realizes his arrogance in assuming humans were the top of the evolutionary ladder.
In Empire Strikes Back, it's Luke's world that is ripped apart.
For Six Sense, Bruce Willis's revelation of his dead state comes with character catharsis.
For Memento, Guy Pierce doesn't even get to be in on his unpredictable twist, only we realize that his quest for revenge is fruitless in every possible way, as most quests for revenge are.
Point is, across all five, we live the lesson only secondary to the characters in the story. For the five duds, the characters almost seem not to be the point.
The Life of David Gale crams a message down our throat about the death penalty and didn't bother to make me care about the issue nor the characters involved. It doesn't matter to the guy who dies nor the reporter telling his story because they were already on board with their respective moral codes.
The Village does the same on the perils of modern life. We don't see this realization change anyone - it's supposed to change us. Well, no thanks.
Same with Fallen for making me root for a demon. We end the story exactly where we started (literally, same scene) and I'm supposed to be the one who feels taken for a ride instead of any particular character.
The Forgotten was just so bizarre - aliens, really - and a message about the bond of parenthood being unbreakable. Geez, thanks.
And Identity - once I realize the ten people are essentially not real and neither are the murders I checked out.
The latter five try to be clever at the expense of the audience, try to be didactic. Don't make me part of your story. Don't have your message, your twist, depend on me. I'm a mixed bag. You never know what you're going to get. I'm as unpredictable as you're trying to be in your story.
Just tell the story.