House and Senate negotiations over legislation that would greatly increase the number of immigrants allowed into the country have become seriously jeopardized, with some sponsors of the measure and a coalition of supporting groups threatening to walk away from the package.

The sticking point, according to congressional aides and lobbyists, has been refusal by Sen. Alan K. Simpson (R-Wyo.) to accept a compromise position between the House bill and the more restrictive Senate measure. Simpson has threatened to filibuster the legislation if it does not meet his demands.

"Here you have a guy who comes in and single-handedly is trying to undo everything," said a Democratic House aide.

Aides to Simpson did not return phone calls yesterday.

It appeared that the two sides have come close to agreement on an overall figure of 675,000 immigrants annually. That figure represents a compromise between the House cap of 775,000 and the Senate figure of 630,000. The current limit is 540,000.

The biggest point of contention that remains is the number of "family preference" visas granted to foreign relatives of U.S. citizens and permanent residents.

Aides said Simpson has refused to budge from the position adopted in the Senate bill, which would subtract the number of visas granted to immediate relatives of U.S. citizens from a cap on the number of visas available for other family members. Under current law, there is no limit on the number of visas for immediate relatives of citizens.

Hispanic and other groups argued that the Senate bill would allow no improvement over the current situation, which they say forces many family members to wait as long as 10 years to join relatives in this country.

"The political consensus around family immigration is overwhelming," said Cecilia Munoz of the National Council of La Raza. "It's unacceptable for one man to blockade improvements in family immigration from becoming law."

Simpson has also insisted that the conference bill contain provisions not included in the Senate and House measures. Among these are "border enhancements," such as fences, that would be constructed to control the flow of illegal immigrants.