The policy-making arm of the federal judiciary yesterday proposed major changes in federal law that would radically curtail the flexibility judges have in sentencing criminals and would eliminate parole as a means of reducing time in prison.

Under the legislation proposed yesterday by the U.S. Judicial Conference, the sentence imposed on a criminal after a trial would generally be the sentence actually served. For the first time, however, either the defendant or the government could appeal sentences deviating from prescribed guidelines.

The proposal, which would apply only in federal courts, becomes one of many before Congress that are designed to reduce the wide disparities between punishments imposed on different criminals for the same crime. Studies have shown differences as dramatic as 20 years in sentences handed out under almost identical circumstances.

Congressional sources said the conference's position would be "helpful" in finally getting something enacted after years of debate. The conference, headed by Chief Justice Warren E. Burger, is composed of the chief judges of each of the nation's 13 federal circuit courts, plus one district court judge from each circuit.

Judges now enjoy almost unlimited discretion in imposing minimum sentences. Like other recommendations, yesterday's would require judges to stay within guidelines established for each type of defendant.

Judges could depart from the guidelines in special circumstances when they believe "the purposes of sentencing will be best served . . . . "

Parole would be eliminated entirely as a means of reducing time served, although a parole commission could increase the time for post-conviction misconduct.