THE BRANDON PAPERS. By Quentin Bell. Harcourt Brace Jovanovich. 214 pp. $15.95.

QUENTIN BELL, at 75, has made his first foray into fiction, and that, in itself, is an interesting departure. He is already a man of many hats: an art historian and critic, a professor, painter, sculptor, potter. To American readers he is probably best known for his biography of his aunt, Virginia Woolf, and for his Bloomsbury connection. As the son of Vanessa and Clive Bell, he is second- generation Bloomsbury, and foremost among the practitioners of what is now a cottage industry: the chronicles of the accomplishments and affairs of the first generation. All these facets (except potting perhaps) have a bearing on this bewitching new enterprise, an "investigation" into the life of Lady Mary Brandon, the beautiful and distiguished Victorian feminist and philanthropist,

The Brandon Papers opens as family chronicle (with genealogical chart) of the Brandons, who devoted the 19th century to dissipating fortunes and cultivating vices, saving themselves from ruin by marrying money. The book then becomes a Gothic thriller in the account of Mary's childhood exile in Newfoundland, guarded by a wicked stepmother and demented servants until her "rescue" by her cousin Henry. It reads at times like a feminist tract, though the diatribes against the role of the Victorian woman are the least successful parts of the novel. It is above all a tale of amorous, ambiguous intrigue and disguise -- just what a child of Bloomsbury would produce. It is also full of violent acts -- at least one explicit orgy of sexual sadism and a plethora of murders and violent death.

The Times of London reviewed the book in its crime and mystery section, yet The Brandon Papers is not a conventional mystery. If we hang upon the story's twists and turns, it is not in anticipation of the final unmasking; for The Brandon Papers' denouement, the great hidden secret, is revealed a third of the way through the book.

Rather than a mystery, The Brandon Paper is a novel of secrets. It is about the nature of concealment, the things lurking in dark corners, locked in upstairs attic bedrooms. Bell's technique is to peel away level after level of appearance, approaching facts crabwise, obliquely, through multiple narrators and differing points of view. He sustains our interest through the tension created by our complicity; knowing the secret from the start, we sympathetically involve ourselves in the fate and inner life of the central character.

The guiding metaphor of the novel is the Victorian gown -- those gouging stays, that carapace of respectability which conceals all manner of bizarre realities. The metaphor points us to The Brandon Papers' most serious theme: the near pathological habit of concealment in late Victorian England. Outward propriety, inner corruption is the standard rule. The Brandon tribe is "beautiful as a company of angels;" but "in character they were at best feeble, at worst despicable."

PLOT, CHARACTER, and a network of literary allusions expand this theme. On the one hand, the book is full of explicit references to the pillars of the 19th-century novel from Scott to Henry James. In one scene, Mary Brandon's tutor tells her that she can cure herself of her unladylike language "if you read Jane Austen, Thackeray, Trollope, and George Eliot and imitate their ladies." And the whole project of making a respectable lady out of Mary is a strange version of Shaw's Pygmalion. On the other hand, there are also allusions to Wilde and Swinburne (author of Lesbia Brandon -- surely no coincidence); and plainly Bell intends us to read the novel in full consciousness of e other, shadowy side of 19th-century literature -- of works like Lady Audley's Secret, Conan Doyle's "The Yellow Face," Twain's The Mysterious Stranger and "The Medieval Mask," even Jane Eyre.

Beyond literature the novel invites us to think of 19th-century life and culture. Mary Brandon's library is replete with cases of famous transvestites and homosexuals. (There are certain notable omissions: The Cleveland Street Scandal of 1889, in which distinguished members of the aristocracy were arrested in drag in a male brothel, the major general in the medical corps whose true sexual identity was discovered only after death -- she was a woman. Bell would certainly have known of these but perhaps found them too obvious for his indirect approach.) All this dark concealment, and the perversion and violence it fosters, are played out against our own time, the age of apparent openness and the analyst, where the secret must be a delicious frisson reserved for fiction.

A final twist is Bell's refusal to acknowledge authorship of The Brandon Papers. In his foreword he tells us that the manuscript arrived, unannounced, by mail (an old 19th century novelistic tradition). It is, he says, the work of one Maurice Evans. who upon overhearing an enigmatic conversation felt there was more to the story of Lady Brandon than met the eye of her uncritical official biographer. Evans was led to some startling conclusions but, alas, was killed before his work would be made known. And so the task of publication has passed to Bell. It is a gentle joke, perfectly seamless.

But nevertheless it is quite clear that all the work is Bell's alone -- not only all the narrations but the diagrams and family trees, even the cover portrait, which Bell, the art historian, attributes to Whistler. Allusions to the joke run throughout the book. "If this were a novel I should lead up to this scene with great care," one character remarks. But The Brandon Papers is more than a literary spoof. It is a thriller, a puzzler, a study of identity at once serious and satirical -- in short an extraordinarily readable hybrid of biography, mystery, sex and romance.