His president couldn't budge him, the chairman of the Senate Finance Committee suggested that he was the tool of greedy bankers, and his majority leader indicated that he had some harsher expletives in mind.

But Robert W. Kasten Jr. (R-Wis.) had been having the time of his life anyway, fighting his Senate leadership to a week-long standstill over a $5.2 billion emergency jobs bill to which Kasten kept trying to attach an amendment to repeal the law requiring the scheduled withholding of income taxes on dividends and interest beginning July 1.

Despite entreaties from the White House and the determined efforts of the Senate leadership to put him in his place, Kasten didn't relent until he got the repealer attached to another bill scheduled to come up April 15 and, Senate sources said, because of impending defections by his supporters.

Kasten is delighted by all the attention. At 40 he still looks like a choirboy, but he also has a thick skin. His resolve was bolstered by the unaccustomed interest of the national press, an appearance on public television news and huddles with prominent allies like Sens. Russell B. Long (D-La.) and Barry Goldwater (R-Ariz.) that left him starry-eyed.

"Whenever you try hard for something, and do it well, it's an exhilarating experience," he said after the midweek cloture votes that left him just one tantalizing vote short of success. In Kasten's view, the withholding plan is "a dumb idea" that the Internal Revenue Service has been trying to foist on the public for more than 40 years.

The trouble is that the jobs bill also contains $5 billion to continue the unemployment benefits that are expected to run out today in 27 states and the District of Columbia. In the caustic view of Senate Finance Committee Chairman Robert J. Dole (R-Kan.), opponents of the withholding plan--the American Bankers Association and its allies--unlike their less fortunate fellow citizens can afford to wait for another legislative vehicle.

"Not many bankers are out of work," Dole observed. "I have checked. They are still in their tellers' cages. Not a single banker showed up in the cheese line the last time they had one."

Kasten has been in the midst of controversy before. After a term as a state senator, Kasten was elected to Congress in 1974 and served on the original House Intelligence Committee where he became a vocal critic of the CIA and voted with the committee's Democratic majority to cite then-Secretary of State Henry A. Kissinger for contempt of Congress.

Kasten's climb upward was temporarily derailed in 1978 when he lost the Republican gubernatorial primary. But he came back in 1980 with a startling upset of Sen. Gaylord Nelson (D-Wis.).

Elected by a 40,000-vote margin on a Reaganite platform of less government and taxes, Kasten likes to show a bit of "independence" from time to time, a characteristic that has always gone down well with Wisconsin voters. But he has also had an embarrassing moment or two.

It came out after his Senate election, for example, that Kasten had filed his 1977 federal tax return two years late and hadn't filed any state tax return for 1977, when he was still in Congress. He said that he was "told my taxes were sent in," and attributes the lapse to "some errors by one of my staff."

The senator also was named last year in a civil lawsuit concerning some questionable real estate deals in which he had been a partner.

He said "some of my former partners did some things after I left" for the Senate and "I have indemnification provisions against some of their actions."

The senator brushed aside suggestions that he is dancing to the American Bankers Association's tune on the withholding tax repealer.

"I'm not in close touch with the banks," he said in an interview. Some critics point out that Kasten's uncle used to be president of the First Wisconsin Bank Corp., but Kasten said his uncle, now retired in Florida, hasn't said a word to him about his amendment.

"It's a savers' issue, a people issue," he said. He said he regrets that "some of the initial advertisements" put out by the banking lobby were "distorted," but contended that the outpouring of mail to the Senate reflects much more than the criticism of the banking community.

What Kasten didn't mention was that he, too, has been taking part in the campaign that depicts the withholding plan as "a new tax," which it is not. Mailings from his office carry the headline: "How to Stop the New Tax on Your Savings." A spokesman attributed this to a staff error.

Kasten argued that it is President Reagan who has departed from conservative doctrine on the subject. The new withholding law, Kasten said, means new regulations, new exemption forms, new deadlines.

Some think he went beyond the pale Tuesday in refusing the initial offer by Dole and Senate Majority Leader Howard H. Baker Jr. (R-Tenn.) for a vote on the issue on another bill.

"What he did was question the integrity of the leadership and the chairman of the Finance Committee," one Senate veteran said. "That really isn't done up here."