One of the largest audiences ever to attend a social event at the White House gathered yesterday afternoon to hear an all-star festival of jazz, America's most indigenous form of music.

In many ways it was a first. And the man who opened the 1 1/2-hour program, venerable pianist Eubie Blake, was inside the White House for the first time. "Never. I've never been here before. Only outside," said the 95-year-old Blake, spry and debonair in a white suit. Blake referring to his effort to pick up a little money be selling records to the crowd assembled for President Truman's 1949 inaugural.

"When Mr. Truman was here, we brought 250 copies of 'I'm Just Wild About Harry,' and do you know how many we sold? One. It was so cold that day that people couldn't put their hands in their pockets to get the money out."

The afternoon was organized to salute the 25th anniversay of the New port Jazz Festival, the country's annual jazz bash. George Wein, the festival's head, unabashedly suggested to Rep. Fernand St. Germaine (D-R.L.) that the White House should pay tribute to jazz and to Newport. The White House agreed. And by 5 o'clock yesterday afternoon, 40 top jazz artists had rehearsed and met President Carter. And pots of jambalaya were perfuming the White House Lawn with garlic, sausage and onion smells.

"I didn't get excited until we got here this afternoon. Before that is was just a gig," said Wein, just before the actual program started. "It wasn't a rehearsal - when you have pros like this you don't have to rehearse - but the president came out and shook everyone's hands."

Representing nine decades of jazz were Katharine Handy Lewis, (daughter of W. C. Handy) Handy Lewis, Jo Jones, Clark Terry, Dexter Gordon, Sonny Rollins, Max Roach, Ron Carter, Herbie Hancock, George Benson, Dizzy Gillespie, Ornette Coleman, Stan Getz and Zoot Sims. In the audience for the buffet dinner, which started at 5 and the concert which started at 6:30 were several well-known musicians. Among the first to arrive were pianist Billy Taylor and baritone saxophonist Gerry Mulligan.

"This is very gratifying for the jazz scene," said Mulligan. Asked about his political feelings, Mulligan said, "I am simply a musician and composer. Any man who has us over to play on his back lawn is okay."

For some of the musicians, like Mulligan, this White House event brought back memories of the evening that many of them attended several years ago in honor of Duke Ellington's 70th birthday. "That was magnificent," said Mulligan, who turned to Willis Conover asking if a tape had ever been made of that evening. Mulligan explained that he had written a composition especially for that evening that he had never heard otherwise.

Before the music started, everyone sat down at plant-decked picnic tables and ate a Louisiana feast. A team of experts from Gonzales, which is up-stream from New Orleans, had cooked 150 pounds of chicken, 170 pounds of pork and sausage, 170 pounds of rice, and 100 pounds of onions seasoned with 6 boxes of salt, 2 boxes of Tabasco sauce, and "a little black pepper and garlic."

Gonzales explained Paul Leblanc, Association, is the undisputed capital of this creole dish. And the cooks even president of the Jambalaya Festival brought the huge black cast-iron pot to insure authenticity.

There was a strong wind - welcome in the 90-degree heat - that tore the white paper tableclothes loose from the rustic wooden picnic tables to which they had been stapled, making them flop in the breeze. They were held down by the heavy flower pots with live flowers thatserved as a center piece on each table, and the wind draped them around the pots in a style that sometimes looked remarkably like a bridal veil.

President Carter sat at a table in shirt sleeves, enjoying a Louisiana creole meal - jambalaya, pecan pie and a salad - while some of his invited guests crowded around, snapping photos, saying hellow or just watching him eat. Some even stood on top of nearby tables to get a glimpse of the chief executive.

"I feel just like a tourist," said one White House regular, looking over the scene - unlike any other in the history of the White House. "I wish I had brought my camera."

Over in the corner The Young Tuxedo Brass Band from New Orleans was playing popular Dixie land numbers such as "Bill Bailey," "Down By The Riverside," and of course, "Jambalaya."

The jambalaya, after cooking for four or five hours in the enormous pots, was served in two varieties, with either chicken or pork added to the basic ingredients of rice, sausage and spices. It was distributed with miniature bottles of tabasco sauce - one eighth of an ounce per bottle, which was plenty for several servings.

The dinner was a prelude to what President Carter introduced as "the first White House Jazz Festival - I hope we have more in the future."

Held in honor of the 25th anniversary of the Newport Jazz Festival, which opens next week in New York, the jazz concert include performances by some of the finest living musicians in the history of this art form.Blake played two of his own compositions, "Boogie Woogie Beguine" and "Memories of You."

"I'm playing all my ASCAP numbers, said Blake. "That way I get paid royalties for it. He was followed by Mrs. Lewis, who played her father's "St. Louis Blues."

The remainder of the program included performers in old-fashioned and modern styles, ranging from Mary Lou Williams, Illinois Jacquet and Clark Terry to Ornette Coleman, Cecil Taylor, Chick Corea, Herbie Hancock and George Benson.

For the Carters, the jazz festival was one of their largest social events. 800 guests were expected. By comparison, "We had almost a thousand people tour the congressional promenade," said White House social secretary Gretchen Poston as she looked over the picnic tables and bandstand.

The guest list was drawn mainly from musicians, show business personalities and jazz academia. Gregg Allman represented rock. Lucile Armstrong, Harry Belafonte, Pearl Bailey, McHenry Boatwright, Mercer Ellington, Gil Evans, and Hazel Scott were among guests from the music world.

Also on the guest list were James Baldwin, artist Romare Bearden, Johnny Carson, Dick Cavett, Bill Cosby, record producer Kenneth Gamble, artist Sam Gilliam, Lena Horne, Quincy Jones, restaurateur Elaine Kaufman, writers Murray Kempton and Albert Murry, and Flip Wilson.

The concert was broadcast live by National Public Radio and recorded by the Voice of America.

According to Wein, almost all the musicians paid their own way. Two musicians whom Wein sought had to send regrets - Count Basie, who has a heart condition, and Joe Venuti, who had a conflicting engagement. But the evening promised a few reunions, one between Sonny Rolins and Max Roach, who together made some of the most memorable jazz records of the 1950s, and Stan Getz and Zoot Sims, who were sectionmates in the Woody Herman Orchestra in the late '40s.