Moments before he was shot and wounded outside the Washington Hilton Hotel yesterday, President Reagan appealed to 3,500 union construction workers to support his spending and tax cut program, already under intense attack from other sectors of the U.S. labor movement.

The speech before the annual legislative conference of the Building and Construction Trades Department of the AFL0CIO was Reagan's first address to a major labor group since his inauguration.

Though he was welcomed inside with a standing ovation, the president apparently was aware of a warning delivered earlier yesterday by trades department president Robert A. Georgine that "labor will not be a scapegoat or a whipping boy" in the administration's drive to right the nation's errant economy.

Reagan said he came to the conference "to communicate," to dispel the notion that his proposed $48.6 billion in federal budget cuts was aimed at the American worker.

"When people ask me why we have to cut down on the budget deficit, I think the answer is pretty clear," Reagan said, preparing to touch on a matter dear to the hearts of the nation's builders.

"If we don't get control of the budget and stop wild and irresponsible spending, we will repeat past, intolerable prime interest rates of more than 20 percent, rates which have played havoc with the lkives of your fellow workers," the president said.

Nearly 660,000 construction workers are jobless today, largely because of soaring interest rates that have slowed or killed building projects, according to Georgine and other building trades officials.

The union leaders said yesterday that one of their major goals this year is to muster enough political clout to force the Federal Reserve Board to revamp its "tight money," high-interest monetary policies.

"Every American, and especially all the working people of our country, has an enormous stake in what we do," Reagan said, building on his theme.

"You pay the most taxes. You believe in the work ethic, but subsidize a government that does not. You who have traditionally saved to provide for your futures today cannot. You who most want to work are likely to be laid off," Reagan said.

"You, through taxes on your hardearned wages, pay for what could be as much as $25 billion each year in federal waste, abuse and outright fraud in government programs," the president continued.

In a line that drew on of the four rounds of applause that interrupted his speech, Reagan said there are "just too many people" in the federal bureaucracy who believe "the billions of dollars raised from taxpayers . . . belongs to the government."

"If we do nothing else in this administration, we're going to convince this city that the power, the money and the responsibility in this country begins and ends with the people and not with some cinder-block building in Washington, D.C.," Reagan said.

Several of the delegates interviewed after the address expressed disappointment that the president "skipped over," as one put it, matters such as federal funding for mass transportation and energy construction projects and the administration's plans to limit unemployment compensation benefits.