PORT WASHINGTON, WIS., JULY 27 -- White House spokesman Marlin Fitzwater charged today that "some members of the press" are attempting to "destroy" President Reagan over the Iran-contra affair, and he defended Reagan's decision last November to conceal information about the Iran arms sales on grounds that lives of American hostages and some Iranian contacts were at stake.

"I frankly think that some members of the press are so hungry to try to destroy the president that they've lost all perspective," Fitzwater said in response to a story in The Washington Post on Sunday reporting that notes of a White House meeting last Nov. 10 indicated that Reagan actively led the initial effort to hide details of the arms-for-hostages deals from the American public. Subsequently, other news organizations carried stories similar to The Post's, based on the same notes.

The notes were made by Alton G. Keel Jr., then deputy national security adviser, and made public in full for the first time last week by the congressional Iran-contra committees as an exhibit accompanying testimony by Secretary of State George P. Shultz. They recorded the comments made by Reagan and his top advisers at a discussion of the sale of arms to Iran a week after the sales were disclosed by a Beirut magazine.

Fitzwater said Reagan is "disturbed that this issue was so clearly reviewed by the Tower board and so openly discussed at the time and somehow the facts have been overlooked in this case."

The report of the Tower special review board, named by Reagan to look into the Iran-contra affair late last year, said: "The board found evidence that immediately following the public disclosure the president wanted to avoid providing too much specificity or detail out of concern for the hostages still held in Lebanon and those Iranians who had supported the initiative. In doing so, he did not, we believe, intend to mislead the American public or cover up unlawful conduct. By at least Nov. 20, the president took steps to insure that all the facts would come out."

The Tower board referred to Keel's notes in its February report, but said it did not have enough evidence to evaluate fully what went on in November. Its final report said, "Though the board reviewed some evidence on events after the {Nov. 3} exposure {of U.S. arms sales to Iran}, our ability to comment on these events remains limited."

In a speech today at the Broan Manufacturing Co. in Hartford, Wis., Reagan said again that he would not become a "lame duck" in his final months in office. Referring to "the legislative hearings going on there in Washington," Reagan borrowed a line from Brendan V. Sullivan Jr., the lawyer for Lt. Col. Oliver L. North, "who had to protest that he wasn't a potted plant."

Reagan declared, "I reject a potted-plant presidency."

Fitzwater's sharp response to The Post account of Reagan's actions last November marked a departure from the general approach the White House has taken of not commenting in detail on issues brought up in the congressional inquiry.

The White House initially reacted this way Sunday, saying there would be no comment on The Post account.

This morning, however, Fitzwater was seen by reporters conferring with Reagan on Air Force One as the president departed for a series of economic speeches in Wisconsin. Reagan did most of the talking to Fitzwater, gesturing forcefully.

Fitzwater then came to the rear of the plane and, referring to notes on a legal pad, told reporters: "The president's comments on Nov. 10 were appropriate, honest and consistent with all his public statements." Reagan has repeatedly said his administration tried to get out all the facts of the Iran-contra affair.

Fitzwater added, "What the president was doing in that Nov. 10 meeting was trying to protect the lives of those hostages."

"This is an old story dealt with in its entirety in the Tower report," he said. Fitzwater then distributed to correspondents several pages copied from the Tower report, with specific paragraphs circled pertaining to Reagan's actions in November.

Fitzwater pointed out that the Tower board had said, "Dr. Keel's notes suggest that the president felt that we had not dealt with terrorists or paid ransom and that one of the purposes of government was to protect its citizens. The president felt that a basic statement had to come out, but that we needed to avoid details and specifics of the operation; he urged that we could not engage in speculation because the lives of the hostages and the Iranians were at stake."

The reference to Iranians was to the contacts that the administration was using in its negotiations. There has been no public disclosure of any threat to those Iranians, and only one disclosed threat to the lives of the American hostages -- from Manucher Ghorbanifar, the Iranian arms dealer who was a middleman in some of the secret arms deals.

The Tower board report did not quote directly from Keel's notes. The notes showed Reagan playing an active part in a discussion of how to deal publicly with the issue. The president said, "We don't talk TOWS, don't talk specifics," a reference to the antitank missiles that made up the bulk of U.S. arms shipments to Iran, according to the notes. They also show Shultz asking, "Do we trade any more arms for hostages?" Reagan responded to Shultz by saying "Appreciate people saying you support policy -- will not comment on," Keel recorded.

Reagan used a series of speeches near Milwaukee today to decry "dangerous trends" in Congress on budget and trade legislation. He accused lawmakers of approving wasteful spending for mass transit and agriculture.

"Farm aid is meant to help the family farmer," he said. "But the way it is now, the government gives little or nothing to most family farmers. The bulk of the money goes to the big rich ones."

He said cotton, sugar and rice farmers are among these, but he did not mention dairy subsidies, which go to many Wisconsin farmers.