In a major concession ending a five-year-old lawsuit growing out of its antiwar intelligence activities a decade ago, the FBI has told the Washington-based Institute for Policy Studies that it will not illegally investigate the organization in the future.

The FBI, which also agreed to pay the expenses that the institute incurred in bringing the suit, did not admit that its massive spying on the think tank in the past was wrong.

However, it did agree to limit severely access to its files on the group that were compiled as part of the undercover probe.

The agreement filed in federal court this week brings to an end the long battle in which the research organization -- a sponsor of scores of radical and liberal activities against the Vietnam war and concerning other international issues -- accused the FBI improperly monitoring its political activities.

At one point in the suit, the FBI admitted that it used at least 62 informants and paid them at least $7,000 for information as trivial as publicly available brochures. The investigation continued for six years, despite the fact that the FBI had found no evidence of crimes on the part of the institute or its employes.

Robert Borosage, the institute's attorney who signed the stipulation ending the case, said the first major victory in the suit came almost immediately after it was filed in February 1974. It was only then that the FBI stopped its spying activities, he said.

"We felt we'd gone about as far as we could go" against the FBI in the suit, Borosage said yesterday. He said it had produced "six drawers-ful" of information and showed an FBI course of spying that included stealing the institute's trash and monitoring its mail.

The FBI steadfastly denied it had ever wiretapped the institute or its members -- as the institute had alleged -- but at one point in the litigation, the FBI conceded that it had set up a listening post across the street. The details of that listening post and its purpose were successfully labeled as a "national security" secret and were never disclosed.

"We were able to make it clear the bureau was willing to go after a research institute without evidence of criminal activity," Borosage said yesterday. He said that since the institute only could prove the bureau used admittedly legal investigative techniques against the institute -- such as informants -- he felt the chances for an ultimate court victory on damages were minimal.