The Justice Department is negotiating with International Business Machines Corp. to cut the number of issues and possibly settle the government's 12-year-old antitrust suit against the computer concern, representatives of both sides said yesterday.

The negotiations were disclosed during a conference before U.S. District Court Judge David N. Edelstein in New York.

As part of the proceeding, a transcript was released of a conversation held Tuesday by Edelstein, Assistant Attorney General William Baxter and Thomas Barr, IBM's lead counsel in the case.

"We are discussing a procedure that would enable us to at least cut down the scope of this case, we hope, and perhaps, we hope, even lead to a settlement of it," Baxter said during the Tuesday telephone conference.

"We have not reached complete agreement on what that procedure should look like, although I don't think we're going to have a great deal of difficulty reaching agreement on a procedure," Baxter said. Barr told Edelstein in court yesterday that IBM agrees with Baxter's assessment.

Each side in the historic suit, which was filed in 1969 during the closing days of the Johnson administration, rested its case last week in the 6-year-old trial, with IBM saying that recent appeals court decisions on other antitrust suits raised serious questions about the basis of the government's case. Closing arguments are scheduled for January, with final briefs due in the meantime.

At that time, Barr asked for a conference with Baxter and Edelstein to discuss the decisions, which overturned antitrust rulings against American Telephone & Telegraph Co. and Eastman Kodak Co. The two cases pose pricing and marketing antitrust claims somewhat like the issues in the IBM case.

Barr claimed in a memo to Baxter that the two decisions and the changes in the industry since the case was filed "compel counsel for both sides to find a way to stop this unprecedented waste of public and private resources now or at least to narrow the issues."

Disclosure of the negotiations marks the first indication that Baxter, the Reagan administration's choice to head the Justice Department's Antitrust Division, has taken control of the case.

In earlier interviews, Baxter had criticized all sides' handling and slow consideration of the case, but had said he only had begun bringing himself up to date on the matter. In the telephone conference, however, Baxter said he and Barr had talked several times about the status of the case.

During the Carter administration, several series of settlement negotiations failed and the trial phase of the case continued.

The suit charges IBM with monopolizing important facets of the computer industry, and the government is seeking to split the giant concern into new independent companies.

Edelstein, who is unlikely to issue a decision in he extraordinarily important case for months, asked the two sides to report back to him on June 18 concerning their progress in setting up the negotiation procedures.