The most overwhelming consensus on Capitol Hill these days deals not with the budget, the Middle East or the nuclear freeze, but with a five-story high, 20-ton, $400,000 bronze "decorative frieze" just craned into place above the door of the James Madison Memorial Building of the Library of Congress.

Everybody hates it.

The screen--technically a sculpture of "falling books"--has prompted protests from such normally mild-mannered types as Daniel J. Boorstin, librarian of Congress, and the Congressional Research Employees Association, as well as from passers-by incensed at what appears to be a crude tar-papering of the building's white marble facade.

"It's ugly as hell," said Michael Gfoeller, researching Middle Eastern affairs at the library for a graduate degree at Georgetown University. "It ruins the appearance of the building."

Strong feelings seem to be the rule, rather than the exception. "I think it's the damnedest waste . . . ," said Rose Barquist of Frederick County, a free-lance researcher. "Spending government money on something like that leaves me speechless with rage."

Architect of the Capitol George White, pressed for some explanation of the sculpture by a subcommittee of aesthetically distressed senators last April, reminded them that "Beauty is in the eye of the beholder."

"It is the judgment of all who have looked at it, from the art standpoint," he said then, that the screen is "a very powerful piece of sculpture" designed "to add to the exterior decor of the building."

Unfortunately, it also covers most of the building's main front window, blocking for those inside a unique view of Capitol Hill and eliminating the street-level reflection of the library's old Thomas Jefferson Building across Independence Avenue.

Some of those who have lived with the sculpture almost apologize for the fact they hate it, apparently hoping that greater minds somewhere in the government have divined some value to it they just can't see.

"What's the purpose of it?" asked Marvin Reed, a Library of Congress policeman. "The building was looking pretty good before they hung that up there. That big window was just like a mirror."

Others are less hesitant.

"It looks like a junk pile," said Maggie Munro Newman, a researcher with the Commission on Security and Economic Assistance. Paul Miles, a tourist from St. Louis, peered at the sculpture and shook his head. "I don't really care for it," he said. "It looks like tar or tarpaper."

Architectural critic Wolf Von Eckardt has described the screen as a "book-hater's revenge . . . dimming the only sizable source of natural light in this entire 1.2 million square foot box."

Just how or why the screen has come to pass is not altogether clear.

White, in his testimony, one of his rare public discussions of his architectural role, said the frieze has been part of the design of the long-planned building for more than 50 years and has long since been bought and paid for.

Though the building was occupied three years ago, installation of the screen was delayed until this summer due to difficulties within the company making the bronze casting, White said.

Sen. Alphonse D'Amato (R-N.Y.) sought to have installation delayed or halted two months ago at the urging of the Congressional Research Employees Association, but discovered it would cost more to keep the frieze off the building than to let work proceed.

It was bad enough for Congress to appropriate money for the sculpture over the years, D'Amato said. To spend still more money to undo that action, he said, would make Congress "the laughing stock of the American people . . . . I don't want one of these Golden Fleece awards."

The issue is further complicated by the tripartite structure that governs the Madison building, involving a subcommittee of the House Committee on Public Works and Transportation, the Senate Committee on Rules and Administration, and the Joint Committee on the Library, chaired by Sen. Charles McC. Mathias (R-Md.).

Tony Harvey, clerk of the Joint Committee on the Library, said Mathias, prodded by a letter from the Congressional Research Employees Association, had come over to look at the screen one day and toured the library in search of some other place to put it.

"His conclusion was that given the fact that the building had been designed for it to hang there and given the multiple approvals of the design over the years and given the fact that it had been paid for and there was no good alternative place to hang it, the best thing to do was let it go up," Harvey said.

Did Mathias reach any conclusion on the aesthetics of the screen?

"No, sir, absolutely not," Harvey said.

Boorstin, in a rare public hint of frustration with his congressional employers, testified at the same hearing that he made "every effort" to prevent the screen from going up, but was told, "It was impossible to prevent."

"So I can simply say," he sighed, "that it seems to me that the two most difficult things in Washington are to build a building and to stop one from going up." CAPTION: Pictures 1 and 2, The James Madison building of the Library of Congress and a portion of the controversial $400,000 "falling books" sculpture. By Barbara Hansen--The Washington Post; Picture 3, Architect of the Capitol George White, pressed for an explanation, said: "Beauty is in the eye of the beholder." Picture 4, Librarian of Congress Daniel J. Boorstin was told "it was impossible to prevent" the addition of the art to the building.