American Management Systems Inc., the Rosslyn-based computer services and consulting company, returned to profitable operations in the third quarter but continued to post losses because of unfavorable developments in a lawsuit.

Charles Rossotti, AMS president, said the latest results, announced Friday, show that "we have done pretty well at turning it around" after a dismal 1981, in which the company lost money on operations.

The turnaround appeared to ease concerns that AMS might be forced into a merger. "We are not planing to merge, sell or be acquired," Rossotti said.

He said operating results in the third quarter were the strongest so far this year and indicate that AMS has righted itself, though it still is "not at the point of profitability where we would like it to be."

But the bottom line showed continuing losses because AMS was required to set aside a $3 million reserve to cover a possible write-off of claims in a lawsuit that the company had previously claimed it would win.

Without the reserve, for the three months ended Sept. 30 AMS would have posted a pretax operating profit of $513,000 compared with an operating loss of $731,000 in the same period a year ago, and a net profit of $278,000, compared to a profit of $9,000 last year. With the set-aside it actually had a loss for the latest quarter of $1,240,000 (76 cents a share).

Revenues increased to $17.8 million from $16.7 million, and for the nine months of 1982 reached $51.6 million from $48.1 million.

Pretax earnings for the nine months without the reserve would have totaled about $1 million against an $804,000 loss last year, and net for the nine months would have been $621,000 against an $81,000 loss in 1981. But with the set-aside, AMS had a net loss for the latest nine months of $897,000 (55 cents).

AMS issued a statement saying the reserve for losses in the lawsuit "has no effect on the company's cash flow or current busines operations." What the company did, in effect, was wipe out in the current quarter profits it had claimed in previous quarters. The action was taken after a federal magistrate in Boston recommended ruling against the company in a lawsuit over a computer contract.

In an interview Rossotti said 20 contracts AMS has received in recent months from the federal and state governments and private clients should further boost revenue and income next year.

Rossotti said there were three reasons for the improved results. First, "We made a big cut in the losses in our package systems division," which had been holding back the entire company. Second, AMS sold its computer center in Arlington to Martin Marietta Corp., and third, "we made a big push to increase revenues through new business."