National Gallery of Art Director J. Carter Brown, who has been having a busy year with "The Treasure Houses of Britain" and "The New Painting: Impressionism 1874-1886," said yesterday there is no truth to speculation that he is being considered to take over as the director of the National Gallery in London. Such a rumor has surfaced on other occasions, and appeared recently in a major London newspaper.

Brown said a number of American names have "been ventilated" for the post but he couldn't imagine the British would ever select an American to head their national collection and be part of the British civil service. Brown, who said he has not been contacted about the position, said American names may be coming up because American architect Robert Venturi has been chosen to design the new addition to London's National Gallery. Brown hosted a farewell reception for the "Treasure Houses" show at the gallery last night. It's the gallery's most successful show ever and last week broke the "King Tut" exhibit record of 835,000. "Treasure Houses," which closes Sunday, ran about three weeks longer than the "King Tut" show. By Sunday, it is estimated that nearly 1 million will have toured enviously through the treasures of the great houses . . .

One of the celebrity tourists expected at the exhibit today is part-time "Tonight" show host Joan Rivers, who often has such fun with the royal family. After her tour she is to have tea with Lady Marjory Wright, the wife of the British ambassador. Why is Lady Wright having the sassy Rivers to tea? Lady Wright said yesterday, "You do everything Mrs. Thatcher and the queen ask you pause and your hairdresser." She said her hairdresser, Robin Weir, suggested it might be fun for her to have Rivers in for tea and Lady Wright agreed . . .

Washingtonians often forget what a good deal they have seeing the great exhibitions free at the National Gallery. About 407,000 viewed the Impressionism show from Jan. 17 through Sunday, when it closed here. It opens on April 19 at the M.H. de Young Memorial Museum in San Francisco, where it will have an admission price of $6 per person, making an expensive family outing, keeping art for a different economic class. End Notes

Rep. Beverly Byron, who has held the congressional seat of her late husband Goodloe Byron since his death in 1978, was married this past weekend to B. Kirk Walsh, an assistant secretary of housing and urban development in the Nixon administration. The couple was married Saturday in St. Ignatius Loyola Roman Catholic Church in Urbana, Md. The congresswoman is expected to retain the Byron name . . .

That was Nancy Reagan lunching yesterday at the Jockey Club with her pal Washington lobbyist Nancy Reynolds and Sylvia Morris, the wife of Edmund Morris, who was selected to write the official biography of Ronald Reagan . . .

"Long Day's Journey Into Night" star Jack Lemmon and his wife Felicia Farr had a private lunch Monday at Dominique's Restaurant. They walked in at 3:30, unaware the restaurant closes in the afternoon. When they were recognized, Lemmon and his wife were served and could eat without an autograph hunter in sight . . .

By unanimous voice vote, the House of Representatives voted yesterday to name the National Talented Teachers Fellowship Program after Christa McAuliffe, the teacher killed in the Challenger space shuttle explosion. The program rewards outstanding teachers with a year of paid sabbatical to pursue educational interests and projects outside the classroom . . .

Jazz singer Billie Holiday, whose tragic life was depicted in the 1972 film "Lady Sings the Blues," was finally honored Monday with a star on the Hollywood Walk of Fame, nearly 27 years after her death and on what would have been her 71st birthday . . .