"I know he is working for something more important than my music," said Mstislav Rostropovich, the Russian expatriate cellist whom Jimmy Carter invited to perform at the White House on nationwide television yesterday.

With the president still tied up yesterday afternoon with the summit meeting at Camp David, Rosalynn Carter hosted the Rostropovich recital.

The ebullient cellist, who is also music director of the National Symphony, played a one-hour concert on a small stage in the East Room, accompanied by his daughter Elena on piano. His other daughter, Olga, served as page-turner for her sister; both are students at the Juilliard School of Music.

The concert, attended by about 200 guests, was telecast live over public broadcasting and repeated in full later last night.

Mrs. Carter told a guest that she did not know until late Saturday that the president was not going to attend the recital. "And I did not tell Mr. Rostropovich until this morning," she said.

The outspoken Rostropovich, who was stripped of his citizenship earlier this year by the Soviet Union, said after the concert that he was "praying for peace with my music."

He also said that what President Carter was doing at Camp David and what he was trying to do at the White House had a common intention: "It is a matter of human rights, and all music is human rights."

The dominant composition of the hour-long program was one of political significance. It was the relatively early Sonata for Cello and Piano, Opus 40, by the late Dimitri Shostakovich, a mentor of Rostropovich who was himself the victim of political harassment.

In the receiving line, Rostropovich said that the choice of that Sonata, an acerbic work which Soviet officials of the 1930s held in disfavor, was "no accident."

His wife, soprano Galina Vishnevskaya, said she fully expected the Soviet government to interpret the selection of that work as "an anti-Soviet gesture."

The Rostropoviches were at the White House last February as guests when the Carters gave their first Sunday afternoon musicale for broadcast on public television.

After the concert yesterday, a small crowd began to gather around Rostropovich in the Blue Room to hear his impassioned remarks on the topic of his ostracism, which he explained at one point by exclaiming. "The devil is so strong" in the Soviet Union today.

"We are true Russians, we love the whole world with all our hearts. We do not know of any faults we might have had in order to deprive a person of his land."

Two guests whom the Rostropoviches asked he invited were pianist Rudolf Serkin and his wife. Serkin has the dual honor of having been the first performer at a Carter administration White House state dinner and the first soloist to appear at the National Symphony under Rostropovich's conducting.

Vishnevskaya broke into the conversation between the two instrumentalists. "The two of you have got to start palying sonatas together," she said.

Serkin had praise for another pianist, Elena Rostropovich, whom he described as not only promising but "yet another remarkable musician in a remarkable musical family."

The afternoon's guests were drawn from several constituencies, the arts world, the media, the administration and Capitol Hill. Sen. Abraham Ribicoff, (D-Conn.) saw it as an occasion honoring a "great musician - I don't think this was any game being played at a time when both the Soviet Union and the United States are going back from the chasm."

There were others, however, who saw in Mrs. Carter's opening remarks reminders of her husband's stand on human rights within the Soviet Union.

Rostropovich, she said, "has undergone very much hardship and if anyone understands the meaning of liberty" it is he.

"He honors us this afternoon not only with his presence at the White House," she said, "but with his presence in our country."

Mrs. Simcha Dinitz, wife of the Israeli ambassador, said that music - that by Rostropovich yesterday and by the Israeli Philharmonic on Saturday - had seen her through the weekend. "My husband said, 'It's not good if I come home Friday.'" He didn't and I was glad."

Mrs. Ashraf A. Ghorbal, wife of the Egyptian ambassador, called the recital "a glorious afternoon at the White House. I hope they will have the same glorious one at Camp David."