President Reagan has selected four nominees to the National Council on the Arts, the White House announced yesterday. They are actors Celeste Holm and Robert Stack, author Allen Drury and musician and critic Samuel Lipman.

All four are expected to be confirmed by the Senate when it goes back into session after the November elections. Four more vacancies remain.

The 26-member council meets four times a year in Washington to advise the National Endowment for the Arts, the federal agency that makes grants to artists and arts groups.

Lipman, 48, is the publisher of The New Criterion, a magazine of arts and ideas, a professional pianist and the music critic for Commentary magazine. He is also one of the authors of a 1980 Heritage Report that was critical of the National Endowment for the Arts.

Holm, 63, has been in numerous films and won the Academy Award for her role in the 1947 film "Gentleman's Agreement." Stack, 63, is probably best known for his roles in the television series "The Untouchables" and "The Name of the Game." He was also nominated for an Academy Award for his role in the 1956 movie "Written on the Wind." Drury, 64, won the Pulitzer Prize in 1960 for his novel "Advise and Consent" and is the author of 12 other novels.

During its two-day meetings here, the council makes recommendations on new endowment policies and, in closed session, discusses grant applications that have been approved by review panels. However, the endowment chairman always has the final word on grants.

Council members are usually appointed for six-year terms. The eight whose terms expired were actor/singer Theodore Bikel, opera singer Martina Arroyo, film director Franklin Schaffner, businesswoman Geraldine Stutz, museum director J.C. Dickinson, businessman William Eells, museum director Willard Boyd and theatrical producer and director Harold Prince.