Jimmy Carter brought an old-fashioned gospel sing to the White House yesterday, and while he turned his ear to the music most of the afternoon his wife, Rosalynn, turned hers to the crowd.

What both heard was a cross section of contemporary gospel music mixed with current political cross fire.

The predominant question of the day was whether Sen. Edward M. Kennedy (D-Mass.) has asked Carter to step down as a candidate for reelection when the two of them had lunch together Friday at the White House.

Rosalynn Carter became her husband's busiest spokesperson on the subject, circulating among groups of reporters as well as guests. By the end of the day, Carter himself, when asked about an account of the lunch published in an Atlanta newspaper, said it was not true.

"No, that's silly," he said, looking a little taken aback at the question. "I'm not a candidate -- yet."

At first, Mrs. Carter assumed a "no comment" stance on the story, which reported that Kennedy had called Carter a political "cripple" who should not seek reelection. She said she had been at the Friday lunch for a half hour discussing upcoming mental health legislation with Kennedy.

"I was not in on their conversation," she said, "but I cannot believe that happened."

Democratic Party chairman John White called it "pure horse manure" that Carter was a political cripple. And, later, he said that it would have been a "terrible blunder" for Kennedy to say it -- and that Kennedy doesn't make blunders like that.

Later, displaying growing agitation over the matter, Mrs. Carter pounded her fist in her hand as she spoke.

"I never heard him [Kennedy] make any kind of a judgement -- I never heard him say anything bad about Jimmy," she said. "He's always said, 'I support the president.' I can't imagine it, I don't believe it. I don't pay any attention to things like that."

While Mrs. Carter toured the grounds giving mini-interviews, on-stage artists representing pop, rock, folk, traditional and contemporary gospel were taking turns at the microphone in the three-hour-long concert.

More than 800 guests wore their Sunday best and ate picnic fare from paper plates: fried chicken, potato salad, cole slaw and lemonade.

Amy Carter perched on a tree limb and read a book, but others listened to the music in more conventional ways, sitting on blankets they brought along. President and Mrs. Carter took off their shoes and settled down on white wool blankets carried out from the mansion with pomp and circumstance by White House social aides.

But Weda Baughns of Lexington, Mass., who edits textbooks, said she marched up to the President as he sat on his blanket and told him that he had disappointed her.

"He had the saddest expression," she said later. "He was hurt by it. I was so shocked by what I said that I tried to smooth it over. I told him that I liked him as a person. And he said thank you."

Judy Spencer of Los Angeles never sat on her blanket.

"When the invitation said to bring a blanket, I said it can't be just any blanket," she said.

So she and some friends quilted one in a week and yesterday Spencer took it around getting such autographs as Secretary of State Cyrus Vance, Treasury Secretary G. William Miller, presidential assistants Alonzo McDonald and Anne Wexler and her husband Joseph Duffey, head of the National Endowment for the Humanities.

Planned for several months but put together in less than two weeks, the event was organized by the 3,500-member Gospel Music Association. It was the latest in a series of White House concerts spotlighting different types of music -- country and western, classical, jazz, soul and rock.

Carter said that he and Mrs. Carter had spent many a "fifth Sunday" at 24-hour sings. "The preacher would preach four Sundays," he explained, "and on the fifth Sunday people would have an all-day sing." The president defined gospel as "rural music -- it has both black and white derivations. It's not a racial music . . . it's a music of pain, of longing, of searching, of hope and of faith."

In the audience -- in addition to some members of Congress, the Cabinet and White House staff -- were trade people, publishers, disc jockeys, recording company executives and promoters. It was a predominantly white crowd, handpicked by the predominantly white Gospel Music Association.

Said Edward Smith, secretary of the predominately black Gospel Music Workshop of America headed by singer James Cleveland: "We hope this will lead to more integrated programs. Maybe James will do a concert with the [white] Goodman Family."

Hailed as the "Crown Prince of Gospel," Cleveland, in fact, wound up the show's 16 acts by telling the crowd: "We'd like to express our appreciation to one of the greatest presidents this country has ever had. Gospel musicians have stood on the sidelines for many years and watched others enter the gates."

In a pre-concert ceremony, singer Brock Speer of the Speer Family told Carter that bringing gospel to the White House would help cure the "ills of the country." Carter replied that it had not been by accident that gospel music was chosen for yesterday's concert.

Nor was the political significance of the day lost on some of those present.

"I would guess the people who tend to like gospel music like Jimmy Carter, too -- though I couldn't prove that," said Jody Powell, Carter's press secretary.

"I'd hate to think the criteria for having this was politics," said Larry Orrell of Nashville, Tenn. "He's doing us a favor by having it. He's probably the cleanest president we've had, and I like that image."

Singer James Blackwood, a 36-year-veteran of gospel music, said: "I think there's a lot of support for the president that's not showing. I believe in his honesty and integrity. He's had a rough time."

But Grammy Award winner Shirley Caesar said: "I feel he's fallen way down the totem pole. Do I feel that he'll be reelected? Right now, no. Will I support Ted Kennedy? Right now, yes."

Although connoisseurs felt the performers didn't have time to warm up, the president and the crowd heard a broad sampling of contemporary gospel. Cleveland and Blackwood, each in his own down-home way, shouting feverishly or moaning a subtle cry, represented the best of black and white traditional gospel. Reba Rambo's airy melodies and the Mighty Clouds of Joy's freight train rhythms captured the feelings of the new gospel.

Cleveland capped the whole concert with a surging rendition of 'Can't Nobody Do Me Like Jesus." He was joined on stage briefly by Shirley Caesar and the Mighty Clouds. The musical proceedings ended with Rep. William G. Hefner (D-N.C.) leading the combined singers in a spirited "I'll Fly Away."

Said Hefner: "It wasn't intended to be a political rally. It was a good day."

When it was over and Carter moved toward the White House, people stopped to thank him for inviting them.

"I sure enjoyed your front yard," called singer Barry McGuire of Van, Tex., turning to tell a bystander that "I think he's a lamb caught up in a wolf's game."

Meanwhile, near the stage where TV crews were folding their tripods, Rosalynn Carter continued her interviews, speaking out for her husband.