The House of Representatives this week lost a secret effort in court to obtain a ruling that congressmen do not have to respond to federal grand jury subpoenas for House records, according to sources.

The legal battle -- carried on here behind closed doors and in sealed court filings in the U.S. District Court, the U.S. Court of Appeals and the Supreme Court -- grows out of an ongoing criminal investigation of Rep. Charles H. Wilson (D-Calif.), sources said.

The House request that the subpoena be blocked, apparently along with a similar request by Wilson's attorneys, is a marked departure in the House's response to similar grand jury subpoenas here in the past, sources said.

Wilson was reprimanded by the House Ethics Committee in 1978 for failing to disclose that he accepted money from South Korean businessman Tongsun Park. The criminal investigation against Wilson is reportedly related to the same series of events, but has not been previously made public.

Sources made it clear that the current House fight was under way before seven of its memebers were implicated last week in a massive FBI undercover operation known as ABSCAM.

The most recent action in the subpoena fight was a refusal by Chief Justice Warren Burger this week to postpone the enforcement of the subpoena, sources said.

Because of the strong federal court rules prohibiting discussion of matters a judge has ordered kept secret, attorneys handling the case refused to comment on the matter. It is unclear whether the House intends to comply with the rulings or press for further court action.

The House has previously resisted grand jury subpoenas aimed at telephone toll records of one member, and an appellate court agreed that certain of those records might be protected from a grand jury's scrutiny.

The subpoenas at issue in the Wilson investigation, however, are personnel hiring documents that have been routinely turned over to federal probers in the past after the chief judge of a federal court has ruled the material is relevant to a grand jury investigation.

In two recent cases here leading to criminal conviction -- one of Rep. Charles C. Diggs Jr. (D-Mich.) and the other of Rep. James Hastings (R-N.Y.) -- the same type of material was turned over without a fight after public notice of the existence of the subpoena was made in the Congressional Record.

No public record was made of the current subpoena, however. Instead, attorneys for Wilson and the House asked that the matter be conducted secretly because it concerned grand jury matters.

U.S. Attorney Charles F. C. Rudd, while refusing to discuss the current subpoena fight, said his office routinely agrees to closed hearings in grand jury-related matters "except in extraordinary circumstances." By statute, materials gathered in grand jury investigations will normally only be used in connection with court proceedings.

U.S. District Chief Judge William B. Bryant held the first closed hearing on the matter, sources said, and denied the House's request.

Attorneys for the House then asked the U.S. Court of Appeals here to delay enforcement of the subpoena, and that court also refused the House request in a secret order, sources added.

The House then went to the Supreme Court, where its last attempt for postponement reportedly failed.

Because of the secrecy surrounding the matter, details of the legal arguments could not be learned. However, reliable sources said the arguments reportedly focused on whether the mere hiring of an employe by a congressman was a "legislative act" that was protected by the "speech and debate clause" of the Constitution.

That legal issue was left open in another context in a Supreme Court decision last year involving a sex discrimination suit against a Louisiana congressman.

Several persons aware of the House's strong stand on the legal points in the case expressed surprise that the House would raise the arguments in closed-door, emergency hearings, instead of waiting until it could be aired publicly.

"I don't know why they are doing it," said one reliable source. "Maybe they were confronted with a subpoena, and with Wilson's request that they fight it and felt they had no other choice."

Others familiar with the general matter said it was ironic that the behind closed-doors wrangle would become public at a time when some congressmen are complaining about the Justice Department's refusal to allow Congress access to evidence in connection with the ABSCAM scandal.

The congressional request for evidence of the alleged misconduct of seven of its members has been made in public and has been an element in the developing ABSCAM investigation. Attorney General Benjamin R. Civiletti and Assistant Attorney General Philip B. Heymann have opposed the request, arguing that criminal probes should take precedence over congressional inquiries.

The Watergate Special Prosecutor's Office in the mid-1970s and the Justice Department since have consistently said grand jury secrecy rules do not allow material to be turned over to Congress for general investigations.

Judge Bryant has ruled in several cases that Congress cannot have access to such materials. One exception made by a federal judge here came in 1974, when U.S. District Judge John J. Sirica ruled that information gathered against then president Nixon by a grand jury could be sent to the House Judiciary Committee because it was planning to use the material in its impeachment inquiry.

Wilson could not be reached for comment late last night and one of his attorneys said it would not be "appropriate" to comment because of the secrecy surrounding it.