Establishment of a House Select Committee on Congressional Operations to be headed by Rep. Jack Brooks (D-Tex.) was approved by the House Rules Committee yesterday, but the new committee will have considerably less power and scope than Brooks originally sought.

Last week Brooks asked the Rules Committee to set up a commission, which he would have headed, which would have had an unlimited life span, unlimited funding, subpoena power, counterpart funds to travel abroad, jurisdiction over changing the House Committee system and the right to study Congress' operation and court cases of interest to the House, and the right to pay two staff members more than any other House committee staffers could be paid.

Brooks said then he wanted the commission formed because a recent committee reorganization proposal, passed by the Senate, would put the House-Senate Joitn Committee on Congressional Operations out of business later this year. Brooks co-chaired the joint committee with Sen. Lee Metcalf (D-Mont.).

But members of the Rules Committee and others objected to the broad powers Brooks was seeking for the commission, the lack of control by the House over the commission and the fact that it trampled on other jurisdictions.

A few members speculated that Brooks was being given the commission by House Speaker Thomas P. (Tip) O'Neill in return for Brooks' cooperation on government reorganization. Brooks opposed President Carter's requested procedure for reorganization authority and it was tied up in the House Government Operations Committee, which Brooks also chairs, until last week when a compromise was worked out with Brooks' concession and the bill was reported out of committee.

In the form that it was approved yesterday, the commission was made into a House select committee, which means that Brooks can head it even though he chairs another standing committee. The committee's life ends in 1978 at the end of this session of Congress unless it is reapproved by the House.

Other curtailments imposed by the Rules Committee yesterday require the committee to seek funding from the House Administration Committee, as other House committees do, prevents travel abroad and fixes the pay of staff at the rate for other committees. While it has power to look into reorganizing the House committee system, the select committee can't do it if some other committee wants to do it.

The Rules Committee also passed an amendment saying Brooks' new committee will have no control over any operation set up for televising the House.

The House recently began experimenting with televising floor action, following a procedure Brooks had long advocated, something some thought was another concession to Brooks for cooperation on reorganization.

Under the form passed yesterday, the main functions of Brooks' new committee will be to look out for court cases of interest to the House, and operation of a placement office that helps screen applicants for jobs in House offices.