IT REALLY is time, after 179 years, to say flatly whether or not Thomas Jefferson had a mulatto slave as a mistress.

This has been a "controversy" since 1802, in the sense that if some imbecile makes an unfounded charge against a great man there will be plenty of other imbeciles who believe it or who reckon that where there's smoke there's fire.

You could say, I guess, that Abigail Adams had a weakness for Austrian ski instructors and had an affair with four of them simultaneously for three years; and there would always be a few people hwo liked to think of that and who clucked late at night over their milk toast about Awful Abigail, and in this way the subject would be called "controversial" after a few years.

But Jefferson himself, to get back to alleged sex on the old plantation, used to say endlessly that it's all right for error to abound, as long as truth is free to combat it.

One trouble of our own day, I suspect, is that error is delicious in print, and truth may be a bit dull.

Recently I visited Virginius Dabney in Richmond to mull over with him the sex life of Mr. Jefferson, since he has written a book, "The Jefferson Scandals," which rebuts the charges that Jefferson lived with Sally Hemings, a mulatto slave of his, for 38 years. Or, indeed, for 37 years or any other number of years.

I have seen reviews of Dabney's book that claim "he protests too much." The implication is that if Dabney goes into a lot of detail and argues vigorously, well, you know how that is -- the truth must be so weak on his side that he has to argue a lot.

On the other hand, if someone (Jefferson adopted this course, himself) assumes that the best answer to asinine and scurrilous charges is aloof silence, then you hear the argument that "he never even denied it."

The truth is that if you wish to believe the worst, of Jefferson or anybody else, you can usually find grounds for believing it, and if you can't find reasonable grounds, you can find ridiculous ones. Which brings us to the peculiarly shaky, not to say vicious, base of the charge that Jefferson lived with his mistress and sired five children by her, one of which (a gorgeous girl) was sold at auction in New Orleans.

The first thing to notice is that only I -- not the eminent Jefferson scholars -- flatly deny he had such a relationship. The scholars say they have no way of knowing what Jefferson did at 2 a.m. on Aug. 3, 1979. They cannot prove he never slept with the woman. They cannot prove he did not father her five children.

All they profess to do is examine the evidence and conclude as honestly as possible that Jefferson did not have Sally as a mistress.

But apart from those scholars there have been others, not recognized Jefferson scholars of the sort who have studied his life and his times exhaustively, but who have come somewhat casually to their subject. Some of these have not merely concluded that their own weighing of the evidence leads them to suspect Jefferson had the slave mistress. On the contrary, they flatly assert this relationship was a fact.

You may wonder, therefore, on what evidence they flatly assert this. Perhaps they have new evidence that the old-line Jefferson scholars knew nothing about?

Virginius Dabney, who is not now and never was a Jefferson scholar, but who has always been fascinated with that giant of a president, reviewed the whole controversy for his own satisfaction, and his "Scandals" book summarizes all he has been able to find out.

First, we might ask where and how the charges first arose, and fortunately there is no argument here. They began with James Calender in 1802. There is also not much argument about Callender:

He was a Scot who came to America in the 1790s to avoid trial for sedition. He had libeled the king and thought it well to put an ocean between himself and the law.

Here he promptly fell into a posture of adoration before Jefferson. Jefferson was the savior, the wonder, the jewel of the blue-eyed world as far as Callender was concerned. But Callender had not only the gift of adoring Jefferson, he also had a knack for vitriol and he used it here and there. He said George Washington was a "scandalous hypocrite" who "authorized the robbery and ruin of his own army" in order to make money thereby. He said John Adams was "a British spy" and much else. He said Alexander Hamilton dipped into the public till to enrich himself. And so on.

In America at that time you could be tried and put in jail for libeling the president, so Callender was imprisoned for his libels against President Adams.

But Jefferson always opposed sedition laws. Jefferson felt free speech, however scandalous and however untrue, was safer for the nation than a set of laws that allowed citizens to be imprisoned for saying scandalous things against presidents. Jefferson foresaw well enough the abuses of sedition laws, by which the government could simply imprison anybody who said things embarrassing to the government.

Jefferson, when he became president, pardoned those who had been imprisoned for speaking their minds, and he pardoned Callender who was still in jail for libeling Adams.

Jefferson also thought Callender, and the others, should be paid cash for having been imprisoned for speaking. Jefferson paid from his own pocket some of the money Callender received in restitution for time in prison.

Callender then applied for the job of postmaster at Richmond. He felt, apparently, that Jefferson could secure this job for him and should, in payment perhaps not only for Callender's unjust imprisonment under Adams, but also for Callender's flattering writings about Jefferson.

Jefferson, however, did not intervene to get the job for Callender. Callender was angry. Callender said if Jefferson did not give him the postmastership, he would tell a few juicy things he knew about Jefferson.

Jefferson (writing to a friend) said Callender was in effect a blackmailer, but as far as Jefferson was concerned Callender could say anything he liked. There was nothing Callender knew about Jefferson (Jefferson wrote) that Jefferson would hesitate to have the whole world know.

Very soon after this, Callender published in Richmond the charge that Jefferson was, after all, president of the United States at the time, and like all president had many enemies.

He was reelected by a landslide, well after the charges had been printed and buzzed about throughout the nation. Which suggests that the contemporary electorate did not attach too much importance to the Callender charges.

The thing to note is that Callender was unarguably the original source of the national dissemination of these charges.

As the historian James Truslow Adams observed, there is hardly a derogatory story about Jefferson still whispered that cannot be traced directly to Callender.

The mere fact that a man is a disappointed office seeker does not prove he is also a liar, of course. The fact that a man once adored Jefferson and later vilified him does not mean the man was wrong. But Callender offered no proof whatever that Jefferson had the mistress and the mulatto children. He produced no witnesses, quoted no reliable sources, and did not even go himself to Monticello to ask the slaves about this matter, and needless to say he did not interview Jefferson or any of Jefferson's family whether there was any truth in it.

It is safe to say Callender's titillating gossip has circulated from 1802 until the present.I first heard it myself in Charlottesville in 1939.

Since Callender was the source, it is germane to ask how reliable Callender was. Jefferson said of Callender that even those who were on Callender's side, politically, revolted at Callender's filth (the filth of Callender's charges).

Well after Callender's charges had been circulated nationally, Jefferson wrote a friend saying he had once "offered love" inappropriately to a married woman when he was young -- it was wrong and Jefferson was sorry about it, but that was the only item among the endless rumors against Jefferson that had any truth in it, Jefferson wrote.

Jefferson died in 1826. In 1873, a slave freed by Jefferson, Madison Hemings, was interviewed in Ohio to the effect that Hemings was Jefferson's son, and that Hemings' mother, Sally, became Jefferson's concubine in Paris in 1787 when Jefferson was minister to France, and when the girl, Sally (then aged 14) came to Paris with Jefferson's youngest daughter.

hemings was speaking of events that occurred long before his own birth and of which he had no personal knowledge, therefore. Possibly he was told these things, possibly by Sally, his mother. An ex-slave's insistence on the Jefferson's death is interesting and conceivably true. It is hardly conclusive evidence, however.

Apart from Callender and Madison Hemings, there appears to be nothing on which to base the case against Jefferson. Callender was a scurilous son of a bitch and Hemings was a somewhat interested witness (a mulatto with Jefferson as father may well have some advantages in 1873 over a mulatto who did not), and while both men may, for all anyone can flawlessly prove, be telling the truth, still their stories have various holes in them, and both stories have the awkward defect of advancing the two men's purposes who told them.

In 1974 a histroian at the University of California, Fawn Brodie, wrote an "intimate biography" of Jefferson, relying heavily on both Callender and Madison Hemings, and never mind what other scholars said, who had examined the same evidence and found it too shaky to believe.

Brodie said Callender was a "generaly accurate reporter," an opinion that has left a good many scholars breathless with wonder at Brodie's judgment.

I have asked a few people interested in Jefferson what they thought Brodie had in mind, reviving the old rumors about the slave girl which had been pretty well discredited by a centruy or so of scholarly research.

Unfortunately Mrs. Brodie died earlier this year. Nobody I have talked with was willing to say Mrs. Brodie wrote the book to make money or to titillate people, or to please that market which always exists if you purport to show a great man had feet of clay.

As it happened I once spent an afternoon with Mrs. Brodie. I asked her what she would think if a letter of Jefferson's were dug up in which he flatly denied having Sally as his mistress.

"If such a letter were found," she said, "I would consider if proof that Jefferson never had Sally as a misress. Jefferson would not lie about such a thing. But no such letter exists."

A well-known letter does exist, of course, written two years after the Callender charges, in which Jefferson flatly says the incident of "offered love" to a married woman is the only thing that's true, among the many scandalous charges against him. But Mrs. Brodie did not accept that as a denial, since the point of that letter was a discussion of the "offered love" episode, and since it did not specify in particular that there was no truth to the Sally story.

Now this is an ugly thing right here: Some people suspect Brodie wrote the book the make money. It sold better than any other history the year of its publication, and it certainly made Brodie's name known to millions who had not heard of her.

But having spent some hours with Brodie, I am utterly certain in my own mind she had no such low motive, and I am also certain she sincerely believed the truth of her book's central assumption, that Jefferson was Sally's lover for 38 years.

Since nobody, not even an amateur, let alone a history professor like Brodie, could believe for one minute tht the Callender-Hemings evidence was sound enough to justify the assertion of Jefferson's miscegenation which is central to the Brodie book, you must ask what other evidence Brodie found.

She found none worthy of the name of evidence. But she did find clues that, in the light of psycho-analysis and psychic study generally, might reinforce the shaky evidence that scholoars had long refused to believe.

It may sound obsurd. On the other hand, sometimes Brodie's discoveries were rather wonderful. To give one example among several:

Jefferson once wrote that the orangutan, that notable ape, preferred black women to the females of its own species. Then, in quite another context, he referred to himself as an orangutan.

Why? Did Jefferson mean that, like the orangutan, he too preferred black women to women of his own race?

You must give Brodie credit for noticing the remarkable fact that Jefferson (one of the least ape-like of the American presidents) called himself an orangutan, and also had expressed his belief that orangutans preferred black women.

Brodie uncovered several other coincidences of this sort. Putting them together with the Callender and Hemings stories -- themselves far too shaky to believe -- she decided the old rumors were true.

Debney felt some or most of these insights by Brodie were either preposterous or demonstrably wrong headed. I myself felt some of them were farfetched, and that many of them were susceptible to much simpler explanations that Brodie had dreamed up for them.

If you yourself were to say, "I went to Baltimore and bought 10 crabs," then I have no real reason to read deep and secret. But if we had been talking about Aristotle when you started mentioning Baltimore, and if two days later you were charged with murdering Crabbe Tenne of Baltimore, then I would think it rather a coincidence.

It would not be proof. Psychic insights, coincidences of words, may indeed suggest clues. I might well wonder why you mentioned 10 crabs just at the time you were accused of murdering Crabbe Tenne, and Brodie die had every right to be astonished at the orangutan provided the Jefferson.

But Jefferson wrote thousands of letters, vast numbers of which have been preserved, and as other scholars have been quick to point out, you can play games with them. If the orangutan letter shows jefferson preferred black women, then what about another letter that (using a similar sort of psychological insight based on unusual associations of words) suggests Jefferson was a homosexual pining away for somebody like (God save us all) John Adams?

But here is my own double conclusion, based on what I have found to read on the subject, which is not everything, and based on my conversations with Brodie as well as a more lustrous and famous scholar, Dumas Malone, and with Virginius Dabney and others interested in Jefferson without being authorities on him:

Brodie was so enchanted with her psychological insights that she could never give them up. I myself would hardly give up the orangutan coincidence if I were in her position as a Jefferson biographer. Other scholars had not noticed these coincidences and she did. They were new, so far as Brodie could tell, and they were hers alone. Understandably (understandably in a writer) she overvalued them.

But not only did she overvalue her own insights, she thrust them forward even when all the mountain of contrary evidence seemed to oppose them.

If (back to Crabbe Tenne) it turns out that Crabbe's mistress confesses to the murder to collect the insurance for which she was beneficiary, and if she gave a detailed and corroborated account of how she killed him, and if the police apologized to you for having charged you with the crime and explained how this began with a computer error, when I think I would have to give up my faith in my insight about your sentence in which you went to Baltimore and bought 10 crabs.

I would still think it a coincidence. Coincidence or not, I would have to weigh it with due regard to other firm documented evidence, and if the vast preponderance of that evidence showed that despite your sentence about Baltimore crabs you could not have been the murderer, then I would have to give up my cherished insight and hunch.

The crime of Fawn Brodie, as I have gradually, not hastily, concluded in thinking about this matter since 1974, is not that she wanted to write a book that would titilate America. Nor did she care a fig about making money on a book.

As I see it, her flaw was one common to writers, common to most men, of having found something nobody else found, something novel and gee-whiz and potentially valuable. Something that seemed to shed light on a subject that nobody knows about certainly. Something that seemed to corroborate the shaky stories of two underdogs, Calender and Hemmings. It is very hard to resist such discoveries, such insights, such hunches, such felicitous fusions of the reflective and brooding mind.

But it is a fatal flaw, as I see it, in a historian or a writer of fact, to hold so tenaciously to his own discoveries, his own insights, that he can no longer fairly judge objective evidence when it conflicts with or flatly contradicts his own best and brightest guesses.

Ultimately, for a historian or for a writer, it is less shocking to succumb to the lure of money or public celebrity than to fall so in love with one's own notions as to be unable to judge fairly the total evidence.

Charming though I found Brodie, the more carefully I read her, the more convinced I became that she had fallen in love with her own notions, not with truth, and her cleverness (so commendable and rare and useful in itself) deceived her to the point that she quite misled herself.

It has never been a question -- not with those who admire Jefferson most -- that Jefferson was beyond mortal failings. If in fact he had a slave mistress, I think every scholar of Jefferson would want to be the first to know, and those like me, who consider him the chief ornament of American society thus far, would wish it to be known as widely as possible.

But if it is not a fact, then it is simply unfair to pass it off as a fact. It has been said that the Charlottesville Establishment, those scholars and Jefferson's town, cannot bear to learn or to publish anything derogatory to Jefferson. But that is not so.

Malone, the most famous of Jefferson scholars, is only one of several Jefferson's scholars who is rather severe on Jefferson's misjudgments and errors, even Jefferson's failure on occasion to be quite honest.

Jefferson once ordered (this is a shocker) all dogs owned by his slaves to be killed, and they were. Slaves had little to call their own to begin with. The picture of a slave boy of 8 with his pup -- it is fairly damned hideous picture to us in the 20th century, removed as we are from the 18th century coarsemess toward dogs. We really do not need to know this about Jefferson, and it hardly raises him in our estimation. But Malone has it in his biography. There are other things in Malone's tremendous study of Jeffersn that we do not have to know and which injure Jefferson's reputation.

Just one more example: When he was trying to ram his pet project, the establishment of the University of Virginia, throught he legislature, he sometimes stooped to unworthy tactics. He said, for example, the state should have his new university because otherwise Virginia youths would continue to flock to places like Harvard where they would "be imbued" with non-Virginian notions, to the great danger of Virginia. This was a redneck argument bad enough when voiced by a redneck, and unforgivable when voiced by a gentleman who in fact had a very high opinion of New England education. It was an appeal to the worst chauvinism of the legislature, and it encouraged that very provincialism Jefferson despised.But he suspected this low appeal might work and it did. He got his university. But he paid for this particular ploy to the extent that men at his school sneer a little at Jefferson for stopping to it.

I first learned of it in Malone's biography. So it is not true that he or other scholars of the Charlottesville School draw the veil at Jefferson's shortcomings.

Their objection to the currently fashionable error that Jefferson had a slave mistress is not that they would be shocked if this were true, but that they are reasonably certain from their most careful research that it is not true at all. They object not because it is embarassing, they object because it seems to them a lie. If any assertation, no matter how wild, can be passed off as historical fact, them they wonder why anyone should undergo the drudgery of writing history or bear the long years of learning and investigating.

If the world values merely what amuses people who like to see the great with their pants down, then what is to prevent a series of scandalous and delicious and false biographies about Mamie Eisenhower, Bess Truman, Nancy Reagan? Surely it is as easy to ignore all the evidence about them and simply assert some lurid past for the three of them as it is to assert a lurid past for Jefferson.

And if truth makes no diffeance, then why not?

There is one thing, especially touchy. Some among blacks have leapt to the Sally bandwagon as if racial pride were involved. The first disseminator of the Sally rumors, however, was James Callender, the one who vilified Washington, Adams, Hamilton and others besides Jefferson, and who said of Sally (she is an almost totally unknown figure, by the way, except for Jefferson's listing of her and her slave children in his Farm Book, and for a somewhat disapproving reference to her by Abigail Adams, and for her appearance in Callender and Madison Hemings accounts):

" . . . The black wench and her mulatto litter. . . "

Callender's observations of Sally as the "black Venus" who was part of Jefferson's "Congo harem" were reprinted throughout America after 1802. Blacks at the time, it is widely suspected, did not make Callender a hero. Callender died in July 1803, in Richmond. The official verdict was that he drowned accidentally (in three feet of water, in fact) while drunk. A Richmond newspaper at the time said "it may be inferred that he got excessively drunk for the express purpose of putting an easy end to his life."

Of course newspapers do not always recognize a "generally accurate reporter," or, for that matter, a totally insane one, but the Richmond newspaper assertation that Callender killed himself a year after circulating the Jefferson rumors is worth thinking about, though it is not proof.

Actually, there are many more factors in the rebuttal of the Sally legend than can de dealt with in anything short of a book like Dabney's, which really should be sold as a companion volume to Fawn Brodie's "Thomas Jefferson."

One thing to keep in mind is Jefferson's life long horror at breeding between races. He may have been wrong, but there is no argument whatever that he opposed it all his life. It is a bit hard, unless you like to think Jefferson was timid, to believe he sired five mulatto children while thundering against mixing the races. It is also a bit hard to believe that (as Madison Hemings said) Jefferson never said an affectionate word to any of his mulatto children, even though (as Brodie argues) his liaison with Sally lasted 38 years and gave him the greatest joy.

Jefferson's patience and prudence were marked, and you rather wonder that if he became president in 1801, saw all the changes about his slave mistress in 1802, and then was reelected president for a second term, he would so defy public opinion as to continue to be Sally's lover while president and in his 60s. It hardly seems the Jefferson style, either, to return to America from France in a small ship with his two daughters and Sally, the maid of one of them, Sally was conspicuously pregnant.

But none of the evidence makes any differance at all if one wishes to believe Jefferson and Sally live it up for 38 years.

My only qualification for even speaking of this is that for some years I have admired Jefferson, spent many many days at Monticello and at the school he founded, and have read a bit here and there about him, and thus like millions of Americans felt I know something of the man. And one thing I feel fairly sure about: if Jefferson had a slave mistress for 38 years, he would have inserted into the Declaration of Independence or the Constitution or on his tombstone or in the regulations of his school an exhortation to live joyfully with 14-year-old slave girls and never mind whether you're taking advantage of them or not. He thought there should be intermarriage with Indians. Why, if he had a slave mistress, should he not equally have spoken in favor of marriage with blacks?

The strongest impression I have of Jefferson is that whatever he did or thought of doing, he expected you to admire the hell out of it. But few would admire a man who seduces a 14-year-old girl who had no defenses, legal or psychological, against him, and who gave her five children that he never bothered to say an affectionate word to till the day he died. It doesn't sound quite like Jefferson, and I thought Virginia Dabndy did the country a service in his "Jefferson Scandals" by pointing out some of the reasons the leading historians think the Sally story is an impossible fabrication.