A proposal to build a $4 million security building in which White House visitors could be electronically screened would detract from one of the nation's most historic buildings, the Commission of Fine Arts cautioned yesterday.

"There is no more important building in the nation . . . and we would like to see every possibility explored to avoid building" next to it, Commission chairman J. Carter Brown told a representative of the National Park Service, which maintains the White House grounds and was presenting the building plan for commission comment.

The proposed security building would be 24 feet by 34 feet -- large enough to contain at least two X-ray machines and four "arches" or metal detectors, Park Service landscape architect Ronald Dixon told the commission.

A large facility is needed because the White House has as many as 10,000 visitors a day in the summer and 5,000 a day during the winter, Dixon said. "It was the intent of Congress that some sort of facility be developed," he said, but exactly what and where was not discussed. The Reagan administration asked for $4 million to build the facility last May.

The building proposed to Fine Arts yesterday would go near the sidewalk of East Executive Avenue, just south of the existing driveway where tourists now enter. A new entrance gate for visitors would also be built.

Metal detector tests for White House visitors were begun by the Secret Service shortly after theassassination attempt against President Reagan on March 30. Close to 500,000 visitors to the White House this summer have been inspected by metal detector or X-ray machines set up in a temporary check-point under the East Wing portico.

The checks are similar to the routine screening given airline passengers, the Fine Arts Commission was told, but more than what White House tourists had experienced in the past when only handbags and briefcases were examined.

Fine Arts did not argue the need for increased White House security, just where the needed security screen should be done. The inadequacy of the polite purse and briefcase checks was seen twice within the past year when two women, one last December and one in January, were arrested after they pulled loaded pistols from their purses inside the White House. No shots were fired in either case and no one was injured.

If a new building is formally proposed by the administration, it must be approved by both Fine Arts, the federal city's esthetic watch dog, and by the National Capital Planning Commission, the federal planning agency for the Washington area.

The Secret Service and the Park Service were specifically asked to see if a permanent security check point could be built under the portico or porte cochere, where checks are now being made.