House Republican leaders agreed yesterday to punish Rep. Richard Kelly of Florida, the only Republican involved in the Abscam bribery net, by withholding campaign funds and kicking him out of the party conference.

Minority Leader John J. Rhodes (Ariz.) said after a 1 1/2-hour meeting of a dozen party leaders that the conference of all House Republicans will be asked this morning to expel Kelly and to recommend that the congressional campaign committee withhold any further contributions.

Rhodes said the leadership decided to move against Kelly now -- though no criminal charges have been brought against him -- because he has admitted accepting $25,000 from the undercover FBI agents. He contends he did so to protect an investigation he was making.

A Republican candidate in good standing could expect $5,000 in primary-election campaign funds and $5,000 for the November election campaign from the congressional campaign committee. Kelly has already received $4,000, said Rhodes. If the committee cuts him off, it means a loss of $6,000 in campaign aid.

Expulsion from the party conference would be largely symbolic, saying to the country that Republicans are quick to clean up their ethical problems. In practice, it would mean only that Kelly could not attend the occasional meetings of the conference where issues are debated and occasionally voted on.

Party leades acted after a number of members, especially newer ones, demanded that the party act to punish Kelly, so it could then campaign against the otherwise wholly Democratic Abscam scandal.

The leadership decideed against trying to strip Kelly of his committee assignments on Agriculture and Banking-Finance-Urban Affairs or his seniority there, which would require action by the entire House.