President Reagan's Cabinet council on domestic policy has approved a new system of Medicare payments to hospitals for capital outlays, administration sources said yesterday.

Under the proposal, which must be approved by Congress, the basic payment received by each hospital for treating a Medicare patient would be increased by an estimated 6 to 6.9 percent through add-on payments.

But all other special Medicare reimbursements to hospitals for capital costs and return on investments would be ended.

The use of an add-on differs sharply from the current system and is designed to even out and eventually limit the growth of hospitals' investments in new plants and equipment.

Medicare currently uses different systems for reimbursing hospitals for costs of a treating a Medicare patient and for capital outlays.

For treatment, payments are sharply limited under a system that was enacted in 1983. Under that system, a single payment for each illness and hospital stay is fixed in advance.

If the hospital's costs exceed the payment, which can happen if a patient is hospitalized longer than expected, the hospital receives nothing extra.

This is designed to discourage hospitals from keeping patients longer than necessary.

For capital investments, however, there are no fixed repayments. Medicare simply reimburses the hospital for Medicare's share of what the hospital spends, which critics contend has encouraged wasteful spending.

The new system would eliminate this unlimited reimbursement.

Instead, a hospital would receive an extra 6 to 6.9 percent on each payment for treatment. The exact amount of the add-on payment would be determined later.

The Cabinet council has agreed that this add-on should not include any allocation to cover returns on equity for investor-owned hospitals.