Navy Secretary John F. Lehman Jr. and Rep. William J. Coyne (D-Pa.) have swapped insults over the Navy's plan to name a new nuclear attack submarine for the city of Pittsburgh.

The imbroglio began when Coyne wrote Lehman on Oct. 22 suggesting that "if this administration wants to do something for the city of Pittsburgh, it could do so by taking steps to reduce the double-digit unemployment which the metropolitan area is and has been experiencing."

Coyne protested that the Navy plan proved that the Reagan administration "seems to have its priorities backwards" while "displaying a great insensitivity to the real needs of the citizens of the city of Pittsburgh."

Lehman, never one to duck a scrap, fired back at Coyne on Nov. 5 in a handwritten rebuttal scrawled atop the congressman's neatly typed note:

"Thank you for your snide, tasteless letter of Oct. 22, 1984, concerning our new submarine, the USS Pittsburgh, the fourth naval ship to bear that proud name.

"As a fellow Pennsylvanian, I know your extremist views do not represent those of the people of Pennsylvania or of Pittsburgh.

"Sincerely, John Lehman."

Coyne was elected to his third House term on Nov. 6.

Two years ago, the administration agreed to modify the name of another attack submarine after church groups, members of Congress and a radical Christian activist protested the selection of USS Corpus Christi. The Texas city's name means "Body of Christ" in Latin.

After months of skirmishing, the 360-foot vessel was renamed USS City of Corpus Christi on orders from President Reagan.

But Navy officials give no sign of intending to rename the Pittsburgh, which was launched yesterday in Groton, Conn.