The House Ways and Means Committee voted 20 to 17 yesterday for payroll tax increases that would nearly triple the Social Security tax for the highest paid workers by 1986.

The financing plan, pushed through the committee by Democrats over the opposition of the committee's Republicans and a handful of Southern Democrats would raise the tax rate on employers and employees from the current 5.85 per cent each to 6.05 per cent each next year (this increase was scheduled under current law), 6.45 per cent each from 1981 to 1984, 6.9 per cent by 1986 and 7.45 per cent in 1990.

At the same time, it would raise the maximum wage on which the tax is levied, from the current $16,500 to $19,900 next year, $22,900 in 1979 and in stages to $37,500 by 1986.

The wage base increase would apply equally to employers and employees.

Rep. Jim Guy Tucker (D-Ark.), Abner J. Mikva (D-Ill.) and other sponsors of the plan said it is needed to prevent the exhaustion of the Social Security old-age and survivors and distability trust funds, which will run out of money in the next few years unless there is a radical strengthening of financing.

Social Security outgo is exceeding income by substantial amounts because high unemployment in recent years has reduced the number workers paying into the fund, while inflation has brought automatic increases in benefits.

In addition, the aging of the population means there will be more retirees to support in relation to the number of workers paying taxes in the future.

"This will give us sound financing to the year 2000," said Tucker after the vote.

At present the highest amount paid by employees in Social Security taxes is $965.25 a year - the sum paid by a worker making $16,500 a year or more.

By 1986, as a result of the proposed changes in the rates and wages base, a worker making the maximum taxable amount of $37,500 would be paying $2,587.50 a year in Social Security taxes and his employer the same.

Tucker warned the committee that the tax rate will have to go still higher, 0.15 of 1 per cent starting in 1981, if the committee's decision to bring federal government, state and local government and non-profit employees under Social Security on Jan. 1, 1982, is defeated on the floor or in conference. The committee agreed on a 31-to-5 vote yesterday to leave that 0.15 of 1 per cent out for now, but Tucker said that if government employee coverage is deleted on the floor it will reduce system income and require the extra rate increase.

The committee also agreed on a 20-to-17 vote yesterday to increase the amount a Social Security beneficiary can earn from a job without loss of benefits to $4,000 in 1978 and $4,500 in 1979 - and apply the increase only to those 65 or over.

At present, a retiree can earn $3,000 without loss of benefits regardless of age, and Republicans had pushed through the Social Security subcommittee a plan to raise the figure to $4,500 in 1978 and $6,000 in 1979. However, Tucker's financing proposal pared the increases back because of the high costs to the government. An amendment by William A. Steiger (R-Wis.) endorsing the GOP plan failed, on the 20-to-17 vote.

The bill, which still has several undecided provisions, allows the old-age, survivors and disability funds to borrow from the Treasury whenever their reserves fall to under 25 per cent of total annual benefits. On a 25-to-12 vote, Rep. Joe D. Waggonner Jr. (D-La.) won approval of a requirement that automatic Social Security tax rate increases be imposed in two years to repay such loans whenever made.

Steiger offered a Republican financing counterplan which called for no tax rate increase beyond that scheduled in current law, until 1985 and much slower wage base increases than Tucker - but payment of half the costs of Medicare from Treasury general tax funds. Democrats beat it on a voice vote.

The Senate Finance Committee is at work on a somewhat different financing plan and President Carter has proposed still a third.