Senate leaders urged President Reagan yesterday not to give in to pleas from Prime Minister Yasuhiro Nakasone to lift trade sanctions against Japan until there is "irrefutable evidence" that Tokyo has begun abiding by a semiconductor trade agreement.

Four key senators sent a letter to Reagan in Venice amid reports that the president might order at least a partial lifting of the $300 million in trade sanctions as a good-will gesture to Nakasone during the seven-nation economic summit that starts there next week.

But the letter -- signed by Majority Leader Robert C. Byrd (D-W.Va.), Minority Leader Robert J. Dole (R-Kan.), Finance Committee Chairman Lloyd Bentsen (D-Tex.) and Sen. John C. Danforth (R-Mo.), an influential member of the GOP minority on the Finance Committee -- warned that any lifting of the sanctions "could damage future U.S.-Japanese trade relations and prospects for enactment of a sensible trade bill" this year.

The Cabinet is deeply divided over whether there should be any lifting of the sanctions. The sanctions were imposed in April after the Reagan administration said Japan had failed to abide by provisions of the pact that called for it to stop dumping semiconductors in other countries at less than their cost of production and had not increased its purchases of U.S.-made computer memory chips.

The Cabinet-level Economic Policy Council failed to reach agreement on the issue Monday and sent the president the latest information on dumping and sales without any recommendation.

Secretary of State George P. Shultz and Treasury Secretary James A. Baker III, who will be with Reagan in Venice, were reported by administration officials to favor lifting the sanctions. Commerce Secretary Malcolm Baldrige and U.S. Trade Representative Clayton K. Yeutter, who will not be in Venice, remain opposed to lifting them now, administration sources said.

In Venice, White House spokesman Marlin Fitzwater said a decision on the sanctions seemed unlikely during the summit, but "the president can decide any time he wants to," The Associated Press reported.