Members of the House Ways and Means Committee expressed doubt yesterday that American taxpayers would trust the Internal Revenue Service to calculate their taxes for them, as proposed in President Reagan's tax-simplification plan.

In his speech last week outlining the proposal, Reagan said more than half of all taxpayers could quit filing returns, a change he compared to the distance between the Model T auto and the space shuttle.

Yesterday, with Internal Revenue Service Commissioner Roscoe L. Egger Jr. testifying before the committee in favor of overhauling the tax code, there was little sign Congress was interested.

"I hate to say this to you . . . but the perception is that you can't trust the IRS," said Rep. Bill Archer (R-Tex.). "When they get that bill from you, the average taxpayer will say, that's not a bill you can believe."

Egger responded that, because the new system would be voluntary, the IRS would not offer it if no one signed up. The tax calculation could be done quite simply, he said, by adding up salary and investment-income information provided by employers, financial institutions and other third-party sources. Taxpayers could just sign the form they received and send it in, with a check if necessary.

Committee Chairman Dan Rostenkowski (D-Ill.) wanted to know why the Treasury Department's original plan predicted 66 percent of taxpayers would be covered by the return-free system, while the Reagan version would cover only half. Egger said the various tax preferences restored to the plan meant more people would be itemizing deductions.

Rep. J. J. (Jake) Pickle (D-Tex.) was skeptical about whether the new computers the service would have to buy for additional processing work would pay for themselves or be less balky than this year's new hardware. That $103 million computer has slowed processing and fouled up refunds and taxpayer accounts this year.

Another witness, former IRS commissioner Sheldon S. Cohen, warned that "to establish an elaborate system which most people choose not to use would be expensive. I hope the IRS will do careful surveys before it moves ahead on this type of program."

Other witnesses, all former Treasury Department or IRS officials, raised several concerns about the Reagan plan, though several endorsed its intent or its broad outlines.

Georgetown University professor Daniel I. Halperin criticized the proposal for benefiting the very wealthy disproportionately and for doing away with the deduction for two-income families. Washington lawyer John E. Chapoton cautioned that the plan should include well-thought-out transitions from current law.

Edwin S. Cohen, also a lawyer here, said chances for passage might be improved if some of the controversial items that raised little revenue to offset rate reduction were left out this time and pursued later. Almost all emphasized the importance of lowering tax rates to give people less incentive to put their money into tax-shelter investments.