The Securities and Exchange Commission, in its first lawsuit under a tough new insider trading law, charged yesterday that a Citibank vice president and another New York businessman reaped about $138,000 in illegal profits in connection with the acquisition of the Monchik-Weber Corp. by McGraw-Hill Inc. last summer.

SEC officials said yesterday that the case was significant because it is the first they have been able to file under the Insider Trading Sanctions Act, passed by Congress in July. Under the new law, which reflects growing SEC concern over insider manipulation of the stock markets, the agency may seek civil penalties three times the value of the profits made on the basis of inside information.

SEC officials said yesterday they were asking that Cesar Duque, a Citibank vice president, and Federico Ablan, a business partner who also serves as the chief executive officer of a firm called Unisoft Systems Enhancements Inc., be ordered to return the $138,000 and pay the maximum treble damage penalty -- or about $225,000 -- on that portion of the profits they allegedly gained from stock purchases after the law went into effect on Aug. 11.

The case is complicated, however, by the fact that Duque and Ablan, who are both Philippine nationals, are alleged to have begun their insider trading several weeks before that. U.S. District Court Judge Edmund Palmieri in New York yesterday granted an SEC motion to freeze the two partners' alleged profits on the Monchik-Weber deal.

According to SEC documents, Duque and Ablan were first told by Monchik President B. Leon Williams that McGraw-Hill had expressed a serious interest in buying the New York computer service company.

At the time, Duque and Ablan were acting as financial advisers to Monchik Director Leonardo Siguion-Reyna, another Philippine national who was Duque's uncle and who also had a personal and business relationship with Ablan, according to agency documents.

Acting on that information, the SEC said, Duque and Ablan bought 12,600 shares of Monchik between July 30 and Aug. 3 at prices ranging from $7.88 to $8.75 a share. The SEC further alleged that, on Aug. 14, Duque was told by Siguion-Reyna that an agreement in principle had been reached whereby McGraw-Hill would buy Monchik for $15 a share. That same day, Duque confirmed this information from Monchik President Williams and, on Aug. 15, he ordered the purchase of another 12,600 shares of Monchik stock at prices ranging from $8.25 to $8.75 per share, according to agency documents.

This second purchase came four days after the White House announced that President Reagan had signed the new insider trading act into law, an SEC official noted yesterday.

The McGraw-Hill-Monchik deal was publicly announced late on the afternoon of Aug. 20, and Monchik's stock price immediately jumped to $14 per share when trading resumed the following day.

Duque, who was described by a secretary as a Citibank vice president for new product development, was out of his office yesterday and could not be reached for comment, a secretary said. Ablan also could not be reached. Lawyers for both men declined to make any comment on the SEC's suit.