Many U.S. General Services Administration employes who manage federal buildings in Washington have been demanding cash pavoffs from numerous contractors who are selected to repair, paint and maintain the buildings, federal investigators have found.

In return for the kickbacks, FBI agents have been told by both contractors and some building managers a number of the contractors have been paid by GSA for much more work than they have actually done in the buildings.

"The building managers set this up," said a source familiar with the widening federal investigation of GSA contracting fraud. "They told the contractors how much phony work would be done, how much the contractors would get paid for it, and how much would be kicked back" to the building managers.

THe building managers also have told investigators, according to another well-placed source, that they were instructed by higher officials in GSA to spend all the money budgeted for building maintenance each year so that Congress would not cut the agency's budget.

"Building managers have said they've been told by GSA they have $200,000 to spend by the end of the fiscal year and to spent it," this source said. "This breeds corruption."

GSA spends $4.5 billion each year to provide federal workers across the county with offices and supplies. GSA owns 157 buildings in the Washingtons area and has more than 50 middle-level employes managing those buildings at any one time.

Many more building managers have come and gone during the period under investigation, which dates back at least as far as 1970. Investigators also are checking GSA repair contracts for federal buildings in cities outside Washington.

The FBI which has brough in extra agents for the investigation, has moved into GSA's regional office at Seventh and D Streets Ws, where records of buildings repair and maintence contracts are kept.

In addition, GSA's own internal investigators, who have been cooperating with FBI, Justice Department prosecutors and a federal grand jury in the investigation, are looking at contracts for the repair and alteration of privately owned buildings that GSA leases for government use here.

Investigators have found that GSA regulations have frequently been violated in the awarding of repair, alteration and maintenance contracts, and that GSAs' contract records are far from complete.

"We've found the contracts frequently don't show were the work is to be done or who actually made the decision to award them," once source said. "When you ask why the regulations requiring the information aren't being followed, GSA people say it's common practice not to follow the regulations."

The investigation began when Lillian Gieseking, a supervisor in the accounts payable section of GSA's Washington regional office, noticed last June that a clerk approving bills from contractors was not normally authorized to do so. Subsequent inquiries revealed that many of the companies receiving payments from GSA did not exist.

The information was turned over to a GSA review team established a month earlier by John F. Galuardi, GSA's regional administrator, and James F. Steele Jr., the regional public buildings service commissioner.

In recent interviews, both men said they had not been satisfied with results of earlier FBI and GSA probes of written allegations by Robert J. Lowry, a Hyattsville painting contractor, that painters were being paid by GSA for nonexistent work in return for making payoffs to GSA's building managers.

"After I received the report (of the investigators), I said it looks like the investigation hadn't covered enough ground," Galuardi said. "The FBI did not do a really detailed investigation of the matter," Steele added.

A Justice Department source has said the FBI bogged down after having difficulty obtaining assistance from GSA.

Lowry began making his allegations two years before the current investigation began. He said recently he unseccessfuly sought in 1976 to persuade Kenneth A. Jacobson, director of repair and alterations, and other GSA officals to measure for themselves the areas where he said contractors has been paid for painting more wall space than existed in the buildings.

"Everybody said it was somebody else's job," Lowry said.

Jacobson said recently it was the building managers' responsibility to check repair work to insure it was done properly. "It's not my duty to inspect," he said, adding that he had turned Lowry's allegations over to GSA investigators.

The present criminal investigation, which began last summer, has turned up evidence of at least $2 million in cash payments collected by GSA building managers from contractors, according to sources.

GSA has since revoked building managers' authority for awarding contracts on their own. Each contract must now be personally approved by GSA's Washington area public building service commissioner. Work costing $10,000 or more must also now be approved by independent inspectors before it is accepted by the government.