HARDLY ANY male in Washington has the indifference, the gall or the nerve to say of the Bullets, "Who gives a good goddam," but instead everyone pretends to be highly anxious about scores.

Now the Orfilas had one of their scrumtious dinners Wednesday night at their California Street house for the Mexican painter Cuevas - Rep. John Brademas (D-Ind.) got up to say he would have to be brief since he is not a senator (a little joke lost on New Yorkers) and offered a toast. Livingston Biddle, chairman of the arts endowment, had made a toast on behalf of the government, and Brademas said people needed to be reminded (especially some presidents, he said) that the "government" is not just the executive branch, but the legislative as well. So he toasted the Mexican painter on behalf of Congress.

Polly Logan adjusted her lipstick and said she was happy as a bird, and Jerry Goldsmith (of Coahoma, Miss.; Helena, Ark., and New York, N.Y.) said it was hard to manage a great collection of paintings any more, and hoped the government was getting the message from the California taxpayers, and Nikki Haskell, a retired stockbroker (she is now perhaps 29 or so) who was down from New York said her grief is that she is a hard-line sweeter and goes to pieces if she sees ice cream or cookies, to say nothing of raspberry bombes. She is no bigger round than a wren, but that is because she suffers all day every day and has for years.

Now a male I shall call Ben, said here we all are in all this glitter and in the communications capital of the world and nobody, nobody, knows the Bullets score.

He had been to a posh dinner at the National Gallery a few nights ago and had been able to arrange for a girl to run back and forth bringing him the score that night every five minutes.

The secretary general of the Organization of American States, Alejandro Orfila, and his wife Helga, however, have a Latin staff, not all that clued in to American sports. Brute Ben had no girl to run scores. All through supper he never knew how the Bullets were doing.

Listen, life is rough amid the marniered cherries and sacrifice, as I have so often noticed, is called for everywhere.

Muhammad Ali, luncheon guest at the Soviet Embassy yesterday, stood up against the dark oak with gilded garlands and said he feels like a kid leaving home for the first time as he departs (this Sunday) for a 10-day visit to the land of the vodka and the bear.

"Want to meet the common everyday workingman and shake his hand," said the former heavyweight champ, and remind him "Americans want peace" and have a lot in common with him. We Americans, for example (he went on) "like pretty girls." A bond everywhere.

He was decked out in a dark suit of blue with touches of other shades of blue but seemed a shade too large to pass as a baroque angel amid the gilt.

The Soviet ambassador, Anatoliy Dobrynin, said, "He's assured me of one thing - he is not going there to fight."

Ali expressed an interest in visiting Russia and it was arranged for him to go as guest of the Soviet state. "I certainly hope I'm not going to pay the bill," he said. (Laughter).

He said Dobrynin was a "lovely man" and the ambassador said Ali was "warm" and generally wonderful and Ali said, "I'm getting kind of old now (he is 36) for boxing, and I don't really have anything more to prove," though he intends to regain the heavyweight title from Leon Spinks.

Ali will visit Moscow and some other cities; "I can't pronounce their names," he said, and Dobrynin grinned.

"If you have to give a punch or two in my country," said the ambassador, "make them strike for peace."

A elot of reporters made way and the little party entered the cream and gilt dining room where avocados stuffed with caviar were served to guests seated on folding steel chairs - someone must have made off with the fruitwood quinzes and quatorzes.

"Many people never get such an honor," Ali said before striding in with his wife, Veronica. "I feel like a small kid."