The American ambassador to Moscow tonight told millions of Soviet television viewers that the United States would like to arrest a steady decline in Soviet-American relations.

In an address marking America's Independence Day, Ambassador Arthur Hartman said, "Recently our bilateral relationship has deteriorated. We would like to arrest that decline but we must say frankly that this cannot depend only on us. It will be necessary for both of our countries to act with restraint in the world."

The ambassador's 4-minute speech followed a cool message from Soviet President Leonid Brezhnev to President Reagan. The Soviet leader, who departed yesterday for his summer holiday on the Black Sea, merely asked Reagan to convey to the American people "congratulations and wishes of peace" on their holiday.

Hartman's speech appeared designed to counter the widespread popular perception here that the Reagan administration is warlike.

The goal of the United States is peace, Hartman said. "The president has asserted many times that the United States will never use any arms--nuclear or non-nuclear--except in defense against attack," he declared.

Hartman appeared eager to dispel concern about what the public considers to be the militaristic nature of Reagan's policies and about allegations that U.S. officials believe the United States could win a nuclear war against the Soviet Union.

He quoted Reagan as saying that "in a nuclear war there can be no winners."

The ambassador alluded to the Soviet military involvement in Afghanistan when he said that "not a single American soldier is fighting anywhere in the world."

U.S. sources said that Soviet authorities were not happy with parts of Hartman's speech and suggested some deletions. The ambassador would not make them, however, and the speech was broadcast in full.

Two years ago the U.S. ambassador, Thomas J. Watson, was not allowed to make the customary July 4 television appearance because his speech referred to the invasion of Afghanistan.