The House yesterday shrugged off objections from senior Democrats and ordered its tax conferees to accept a version of the Republican Roth-Kemp tax cut plan that has been adopted by the Senate.

The vote was a top-heavy 268 to 135.

Although the instructions are not legally binding on the House conferees, the action makes it unlikely that the conference committee will drop the Senate provision, as had been expected.

The Senate plan was drafted by Sen. Sam Nunn (D-Ga) as a Democratic adaptation of the essentially Republican Roth-Kemp. It would cut taxes $142 billion or 5 percent a year, between 1980 and 1983, but only if the government succeeds in restraining spending far more than it has in most recent years - and also more than most experts regard as likely.

The vote on the Nunn plan came as the conferees began their work on the bill - and as they continued their end-of-session game of chicken with President Carter over the controversial college tuition tax credits, which both houses have approved and which the Senate has also tacked onto the big tax cut bill.

Carter told conference leaders yesterday morning he would veto the tax bill if it contained tuition credit legislation, although he hinted he would be willing to compromise on most other issues.

In return, the conferees decided informally to allow the two houses to send Carter the tuition credit legislation separately and guarantee the law-makers a chance to override him if he vetoes the legislation.

At the same time, however, Long promised that the tuition credit measure would be retained in the overall tax bill until it became clear whether there were enough votes to override a veto.The outlook now appears uncertain.

The conference is to reconvene this morning to try to reconcile 126 separate provisions in the overall tax bill including 50 added by the Senate and never considered by the House.

At the suggestion of Sen. Russell B. Long (D-La), the panel recessed in mid-afternoon to give House conferees a chance to camcus to decide which of the Senate amendments they would accept.

The tuition tax credit legislation apparently was the only provision specifically cited by Carter as certain to lead to a veto. Although the measure passed both houses as a separate bill as "veto insurance."

Carter appeared to be more flexible on other key tax issues. White House press secretary Jody Powell told reporters that the president had agreed to accept a $2 billion cut in capital gains taxes, if Congress did not dilute the "minimum tax" on high-income investors.

It was not clear how Carter would react to the Nunn proposal. Treasury Secretary W. Michael Blumenthal said the issue did not come up at yesterday's meeting. But Long said he thought Carter does not favor the provision.

The conference committee is trying to reconcile a House version of the tax bill, costing $16.3 billion, with a $29.1 billion Senate version, which all sides agree would bust the budget.

Although the Senate measure technically would meet congressional budget targets approving it would leave no more room for other pending tax legislation, such as energy tax credits and tax breaks for Americans working abroad.

Long and Rep. Al Ullman (D-Ore.), chairman of the House Ways and Means Committee, both have been trying to avoid a veto, which would bring Congress back for a lame-duck session. But many lawmakers appear not to take the president's veto threat seriously.

Rep. Barber B. Conable Jr. (R-N.Y.) told the conference yesterday that "the president has not been that relevant a force in the development of this bill up to this point," and, "I don't think we should allow him to dictate to us."

Conable was the key sponsor of the push to get the House to go on record in favor of the Nunn provision. Ullman and other key Democrats spoke against it, calling the measure a Rube Goldberg scheme. They warned that it would "straitjacket" the government and bring of a recession.

In all, 132 Democrats supported the Nunn proposal, while 134 voted against it. On the GOP side, 136 Republican endorsed the measure and one - Rep. Charles W. Whalen Jr. (R-Ohio) - voted aginst it.

All local area representatives voted for the Nunn proposal except three Maryland Democrats, Reps. Clarence D. Long, Barbara A. Mikulski and Parren J. Mitchell.