THE WHISTLE BLOWER. By John Hale. Atheneum. 239 pp. $15.95

THE WHISTLE BLOWER, by John Hale is the best sort of thriller: intelligent, understated, challenging, and chilling.

"Frank Jones was an ordinary man," the story begins. His whole life, in fact, is perfectly ordinary . . . until a policeman comes to the door with the news that his son Bob, who worked as a translator for British Intelligence at Cheltenham, has died in an accidental fall. Stunned, Frank goes to Cheltenham. He asks a few questions, perfectly ordinary questions, but the answers seem wrong. Like any puzzled parent, he asks more questions . . . and the answers are even stranger. Gradually, Frank must admit his unthinkable conclusion: Bob was killed by his own employers. And the more he learns about the secret world of intelligence, the more he must believe that until now his life "had been lived in a state of innocence."

That sense of innocence lost is a central element of the novel, made all the more effective by Hale's subtle allusions to Alice's Adventures in Wonderland. Frank is not merely bewildered and frightened by the secret world he discovers; he realizes that he too may be its target, that "he might be counted among the bad guys simply by being Bob's father. The idea was part of the new reality." This ordinary man's tumble into that "new reality" becomes most frightening when he finds himself thinking the way they think, believing nothing and trusting no one.

The Whistle Blower, filled with human drama, lights up the dark side of an important modern question. THE MAN WHO LOVED MATA HARI. By Dan Sherman. Donald I. Fine. 352 pp. $17.95.

DAN Sherman's The Man Who Loved Mata Hari is a very special book. It's a thriller, and a sort of inside-out love story, but it is really the tale of an obsession, one that, we come to believe, its author fully shared with his characters.

Mata Hari was the renowned beauty, exotic dancer, and infamous spy who was put to death by the French in 1917. After much research, Sherman concludes that "in the end they did not execute her for a crime -- they murdered her for their own reasons."

Margeretha Zelle appeared in Paris early in the century, posing for nude photographs. By all accounts she had the power -- with her eyes, her body, her manner -- to mesmerize any man she wanted . . . and she wanted many men. She took a notion to be a dancer and, under the name Mata Hari ("Eye of the Dawn" in Indonesian), danced in most of Europe's proverbially glittering capitals.

Her ever-growing circle of wealthy and powerful admirers helped, of course, besides providing a steady stream of furs, jewels, and money. Among them, hopelessly hypnotized, was a young English painter named Nicholas Gray, the only man who truly loved her to and beyond the end. And another was Rudolph Spangler, who turned out to be a German spy. Her contact with him -- through a tangle of mischances, misunderstandings, and malice, would bring her before a French firing squad.

Sherman's semi-documentary style is so convincing that, at the end, when he reminds us that this is a novel and that some of his characters never existed, you'll hate to admit it as much as he does. KILGORAN. By Elaine Crowley. Doubleday. 351 pp. $16.95.

SOME novels rely for their strength on the power of the familiar, and it's a mistake to deny their appeal.

"What's it about?" a level-headed, well-read friend asked me about Elaine Crowley's Kilgoran.

I sighed. "Well, it's set in Ireland, 1837. You know. There's beautiful, golden-haired, high-spirited Katy, of course, who's promised to the steadfast Peader but who yearns in her heart for the handsome Jamsie. And there's a new priest in the village, and a kind-hearted master in the big house above on the hill. The master is a widower with a ne'er-do-well son and three daughters, the youngest of whom is most sympathetic. I expect there's something dramatic ahead for her. And oh, you know, there are earth-shaking questions to be answered. What will become of gentle, lonely Charlotte? Will Charles come to a bad end? Will Catherine survive the terrors of the wedding night? Will Peader stay in Dublin where he's fled in shame and sorrow after being thrown over by Katy? Will Katy make a good marriage with Jamsie? Will Jamsie give up his drinking? Will the evil landlord on the neighboring hillside raise the rents again? You know, that sort of thing. Oh, and there's the Famine yet to come."

"How much have you read?" my friend asked.

"About 50 or 60 pages so far."

"My Gosh," said my friend, "what's not to like!?"

What, indeed. Crowley writes well enough, and her characters -- who rather resemble peasants in a ballet, seldom kept from their dancing even by dire calamity -- are human enough to care about. PEOPLE WHO KNOCK ON THE DOOR. By Patricia Highsmith. Penzler. 327 pp. $15.95.

EVEN good novelists occasionally have a lapse, and Patricia Highsmith had a very bad lapse of several hundred pages when she wrote People Who Knock on the Door. It's the story of Arthur, 17, and the effects on him and his family when his father becomes a born-again Christian and tries to revise all their lives and impose his moral views on others.

Things come to a head when Arthur's girlfriend reveals that she is pregnant and opts for an abortion. Because Arthur approves the decision, he is put out of the house and denied funds for college. Meanwhile his younger brother Robbie becomes the father's faithful ally. Mom tries to keep peace by keeping everyone well-fed. In the end, inevitably, principles are challenged and hypocrisy revealed, and also inevitably, terrible violence ensues.

Alas, it's all thoroughly unconvincing. For one thing, Robbie is outlandish; when did you last see a normal 15-year- old playing horsey? And how many teen-agers casually drop by to have a cocktail with the old lady next door, or "smooch," or describe something as "kooky," or hang out with their buddies listening to the Beach Boys and Cole Porter? They certainly don't do that on my block. And, besides, has Highsmith never heard of Planned Parenthood? These characters act like nothing more than characters in a novel. SAN ANDREAS. By Alistair MacLean. Doubleday. 326 pp. $15.95

ALISTAIR MacLean's San Andreas has fewer howlers than his last novel, Floodgate, but since the howlers are the best part in a MacLean novel -- he does not, for example, know the difference between "transmit" and "transport," thereby failing both English and Latin simultaneously -- this one disappoints on all counts.

San Andreas is about a World War II hospital ship whose sole purpose seems to be to steam aimlessly through the most dangerous waters of the North Atlantic, inviting constant attack by enemy aircraft. The hellishly jumbled plot -- about a saboteur on board and a gold store secretly used as ballast -- has enough holes to earn it a watery grave.

Here's the captain in a tense moment: "He's an extraordinary seaman and he's never pestered anyone in his life. Let's have Janet along here to see if she bears out your preposterous allegations." Sounds like Gilbert & Sullivan to me. For drama, we get "My God, how foolish can I be! . . . Of course they've got radar on the island." Both dialogue and narrative are by turns pompous ("Bitter experience makes for a splendid conductor to belated wisdom"), meaningless ("They were useful planes and had their successes but were not particularly effective"), or simply hilarious ("Nothing like locking the door when the horse has ruined the stable").

MacLean's language cries out for quotation. "The foregoing," he writes, "may strain the bounds of incredulity or, at least, seem far-fetched." Indeed.

Alan Ryan is a novelist and journalist. His forthcoming books include "The Bones Wizard" and "Quadriphobia," story collections, and "The Coffin Chronicles," a novel.