The White House said yesterday that U.S. strategic arms negotiator Edward L. Rowny has been the target of "unjustified attacks" because of a memo of "talking points" that was prepared for him and that criticizes his colleagues.

One day after Rowny was defended by Secretary of Defense Caspar W. Weinberger, White House spokesman Larry Speakes issued a statement reaffirming President Reagan's support for Rowny.

The memo, prepared for a meeting between Rowny and Kenneth L. Adelman, Reagan's nominee as director of the Arms Control and Disarmament Agency, reportedly includes a "hit list" of arms control officials and comments critical of their performance.

Speakes said the "current discussion" of the "talking points" prepared for Rowny's use in the meeting with Adelman "is a disservice to all concerned."

"Unjustified attacks on Ed Rowny--a dedicated, experienced, competent public servant who has devoted his life's work to our nation's security--do little to serve the country," the White House statement said.

Speakes also said in the statement that Rowny's "skill and wisdom" have been "critical factors" in the preparation of Reagan's Strategic Arms Reduction Talks (START) proposals and "in presenting them at the table in Geneva."

When the memo came to light recently, Rowny said it did not represent his views, although he did not dispute its authenticity.