White House officials have rejected a long-debated compromise pension plan for new federal workers, saying it would cost too much and allow workers to qualify for retirement benefits too early.

As a result, the take-home pay of 300,000 civil servants hired since January 1984 will drop 5.7 percent starting today, when the government begins taking full retirement fund deductions.

Congress and the White House were attempting to meet a Tuesday midnight deadline to agree on a compromise worked out by Sen. Ted Stevens (R-Alaska) and Rep. William Ford (D-Mich.), but White House officials said they needed additional time to negotiate.

The compromise plan would have based retirement benefits for new workers on Social Security, a modified civil service pension and a voluntary tax-deferred investment plan.

But the White House said the plan was too costly and would have allowed workers to retire as early as age 59. The administration has long sought to raise the federal retirement age, now 55 after 30 years' employment. Officials say that early retirement features of the current plan are a costly benefit not available to most other workers in the United States.

The increased deductions, which will be reflected in checks issued later this month for most employes, also will affect the paychecks of most political appointees -- as well as about half the members of the House and Senate.

Newer federal employes have for the past two years contributed less to the pension program than longtime employes. Under an interim program Congress set up more than two years ago, new hires were allowed full civil service coverage but had to pay only 1.3 percent of their salaries for it. They also pay 7.15 percent for Social Security coverage.

Federal employes hired before 1984 are covered only by the Medicare portion of Social Security, for which they pay 1.3 percent. They contribute 7 percent of their pay to the civil service retirement fund.

The interim agreement, which allowed new hires to pay only a portion of the civil service retirement contribution, was due to expire earlier this year. But it was extended by Congress to allow time to come up with a plan agreeable to the Senate, House and White House. The second extension expired at midnight Tuesday.

Senate sources said the increased deductions will hit the paychecks of all Reagan administration appointees no matter when they came into government, 198 of the 434 House members and about 45 of the 100 senators. Members who elected not to be covered by the civil service pension plan will not be subject to the new deduction.

Congressional sources say they anticipate another attempt to revise the retirement system for new workers.

Yesterday, federal and postal union leaders unanimously denounced the White House for its last-minute rejection of the compromise, saying that it was a sound plan that had been developed over more than two years. Pay Proposal Skepticism

Federal union leaders are skeptical of a Reagan administration proposal to extend throughout the civil service a performance-based pay system that has been tested in two Navy facilities in California. The so-called China Lake project gave managers more power to reward employes than the regular federal pay structure does.

Red Evans of the National Federation of Federal Employees said his union is not very happy with the idea of broadening the program.

"China Lake is a unique facility with high-tech personnel, and it was developed without any input from unions," he said.