Outgoing Treasury Secretary Donald T. Regan said yesterday it wasn't likely that all of a tax simplification plan proposed by the White House would get through Congress this year, the first administration admission that parts of its high-priority tax revision may be delayed.

Regan, in a half-hour farewell address to Treasury Department employes, also said the administration hoped to halve the current inflation rate by the end of President Reagan's second term.

Regan is scheduled to swap jobs with White House Chief of Staff James A. Baker III on Monday.

Regan said that his tax reform plan, unveiled last November to mixed reviews, would become a landmark document that will be the subject of debate this year and next.

Baker and other administration officials have said that pushing tax simplification will be the top priority of the second Reagan administration along with deficit reduction. However, leading Republicans in Congress have said that tax simplification will have to take a back seat to reducing the deficit, which they consider to be more important.

The administration has not yet embraced the Treasury tax reform plan and intends to consult with members of Congress who have introduced similar proposals and with business groups before presenting its own proposal.

Regarding an administration proposal, Regan said, "Now it may not be that we can accomplish all of it in the Congress this year, but I think we'll do an awful lot."

Regan said the administration was "well on the road now" to begin debate on how the tax code should be restructured.

According to Treasury Department officials, Regan, the 66th Treasury secretary, served four years and eight days in that post. Only 12 other Treasury secretaries have served that long, the officials said.

Regan, the former chairman of Merrill Lynch, also said he traveled 300,000 miles, giving 300 speeches, during his four years in office, and testified before Congress 80 times.

More than 300 Treasury Department employes gave Regan a short standing ovation at the beginning and end of his address, in which he enumerated various first-term successes such as a strong economic recovery, completion of the tax simplification study, improvement from the recession of the world economy and the reduction of inflation.

"Inflation, which was the major topic on everyone's tongue when we came into office, has certainly been stilled," Regan said. The Consumer Price Index last year was 4 percent, with other inflation measures growing less slowly, Regan said.

When the Reagan administration began, it was facing consumer prices rising at more than a 12 percent rate.

"We're not going to rest on that" inflation reduction, Regan said. "Even though that's a major accomplishment, I think it can come down even further and my goal is to see that it's cut at least in half again if not more over the next four years."

Although the Reagan administration has said it is trying to reduce the federal budget deficit, Regan listed among his achievements the fact that under his tenure he has raised more money in the financial markets than any other Treasury secretary.

During his four years in office Regan said he raised $3.63 trillion to cover the federal deficit. "Maybe we shouldn't have headlines on that," Regan said. "No other administration has raised anywhere near that amount of money. I don't think in so doing we have disturbed the marketplace too much."

Many economists contend that the financing of the large federal budget deficit has caused real interest rates to stay at historically high levels.