If a gang of inmates from an insane asylum were to stage a low-rent episode of "Dynasty," it might look something like "Tough Guys Don't Dance." The movie, which was directed by Norman Mailer from his own novel, is hard to classify; at times you laugh raucously at what's up on the screen; at others you stare dumbly, in stunned amazement.

"Tough Guys Don't Dance" is, by turns, hysterical, incoherent and blasphemously original. The idea of a bloody murder mystery set in the perpetual twilight of Cape Cod -- instead of the usual urban backdrop -- is a choice one, and from the outset it contributes to the movie's odd, gothic feel.

But if the narrative is Pilgrim James M. Cain, the tone is a kind of Charles Addamsy Grand Guignol. After Tim (Ryan O'Neal), a failed writer living in Provincetown, Mass., gets dumped by his wife (Debra Sandlund), his existence turns rancid. One morning he wakes up with blood all over the front seat of his car, a tattoo on his arm and no memory of how either got there. And that's just the start of his troubles.

The film's plot is much too intricate -- and too inanely exaggerated -- to make sense of here. Most of the events are recounted in flashback by Tim to his father Dougy (Lawrence Tierney), who's dying of cancer and drops by because he doesn't want to expire without Tim's knowing the "high regard" he has for him. High regard in this case is based on the fact that Tim turned out to be a tough guy and not a punk; because he took his three years in a Florida prison "standing up" and didn't "dance."

This is Norman Mailer in full homosexual panic. And it's this lunatic energy -- combined with the author's usual pent-up obsessiveness about sex, drugs, the underworld, women, violence -- that fuels the film. Familiar Mailer themes crop up, but you feel foolish discussing the film in these terms. It won't bear up under that kind of scrutiny. But when the corpses -- at least two of them without heads -- start turning up, you know what sort of dementia he's working off.

It's this same dementia that pushes the movie over the edge. The film isn't as wildly experimental as Mailer's earlier movie ventures. It's unconventional, but it's not an attempt at something revolutionary; he's not trying to set the art of film on its ear. But Mailer doesn't attempt to curb himself either; he's still busting up furniture and, working from his own novel, he must have felt completely uninhibited.

Why Mailer would choose to adapt this, his most unreadable book, to the screen is perhaps more of a puzzle than anything the movie provides. The movie is somewhat of an improvement on the original material; at least it's never dull.

"Tough Guys" never finds its footing, though, or a workable, consistent style. The movie has its pleasures; I laughed at a lot it, but I've rarely found it harder to determine whether the film's effects were the result of the artist's designs or his ineptitude. Watching the movie fluctuate wildly between florid, butcher-shop comedy and slapstick horror, you always have questions in your head about the filmmaker's intentions. And because of the film's drunken, all-over-the-road tone, you never know if you're laughing with, or at, the movie.

But the film's macabre comic tone feels like an afterthought; it's as if Mailer wanted make a film noir and, failing that, settled for a noir comedy. This is a familiar Mailer ploy: When he can't be a genius, he becomes a clown. He doesn't just lie down and play it safe. He's attempting more here than a lot of filmmakers do, but it's still a bail-out move; he's covering himself. The sendup is preemptive; he's laughing at himself before anybody else gets the chance. Mailer's excessiveness in "Tough Guys" isn't the problem. The film's violence isn't anything to get upset about because the circumstances are so loopy and outrageous. And Mailer has been as extravagantly confrontational on the page and made it work. But he's in control when he's writing and he's not here. Watching him direct in "Tough Guys" is like watching a guy climb behind the wheel of powerful sports car and slam it, full speed, into the wall -- then back up and slam into it again. And there's a fascination in seeing this kind of spectacle because it's not just anybody behind the wheel -- it's Norman Mailer. But it's not the same kind of satisfaction that you get from him when he's writing at his best and he's banking through corners at top speed.

Mailer's greatest failure is with his actors. The dialogue is flamboyantly, exuberantly profane, and it sits uncomfortably in their mouths. The characters speak in a Maileresque patois that's part dime novel, part Screw magazine, but his performers don't seem to know how to play it; they look lost, baffled. Ryan O'Neal looks a little battered up, which makes him more interesting, but only marginally. He does manage to get his lines out and hang on to what little character he's been given. Isabella Rossellini, who has a small part as Tim's former love, mostly stands around with nothing to do. But as Tim's wife Patty Lareine -- who at one point is described as "a study on low greed" -- Debra Sandlund is like a dinner-theater diva on amphetamines; her scenery-gobbling, as well as John Bedford Lloyd's, who, as Patty Lareine's second husband, has his moments, and Wing Hauser's -- he's the "acting" chief of police -- might have been a real kick, but it's not so much here because it takes real talent to perform in this over-the-top style and these guys don't have it.

"Tough Guys" has some classic lines, and it's full of Mailer's exhibitionistic, swaggering brattiness. This is both Mailer's substitute for content and an attempt to shock. But the movie doesn't so much shock as boggle. You'll see worse films than "Tough Guys Don't Dance," and God knows you'll see better, but you'll have to go a long way before you see anything stranger.

Tough Guys Don't Dance, at area theaters, is rated R and contains violence, profanity, nudity and other suggestive material.