Senate cafeteria workers voted 146 to 35 this week against affiliating with a union.

The vote, conducted by the Senate Rules Committee, which has jurisdiction over the cafeterias, came after a protracted effort by the Capitol Employees Organizing Group to unionize the 210 employes who serve the Senators, their staffs, and the public in the dining rooms and cafeterias of the Senate and its office buildings.

As an alternative to permitting a union, the rules committee considered a bill that would permit a private company to run the cafeterias. Under such an arrangement, workers could then belong to a union, without the Senate having to decide the issue. The cafeteria workers also rejected that idea in a poll conducted this week by the committee, which asked the workers to consider a variety of alternatives.

While laws passed by Congress guarantee workers' rights to organize unions at private companies, they do not require Congress to guarantee such rights to its employes.

According to the poll results announced yesterday, the employes voted 149 to 26 against contracting with a private company to run the cafeteria.

They also rejected a proposal to give control of the restaurants to a government authority other than the Architect of the Capitol, even if current wage levels could not be guaranteed.

Finally, in a poll on the quality of the restaurant management, 65 rated it average, 40 said it was poor, 37 said it was excellent, 25 said it was above average, and 13 said it was below average.

Out of 210 eligible workers, 187 voted.

Stuart S. Smith, a government public relations officer who helped organize the effort, said the issue of whether the workers wanted a union was settled in 1980, when 130 of the 223 people then employed by the cafeterias signed up for a union.

"Any discrepancy between the union's majority status in 1980 and today's election results can be attributed to more than 3 1/2 years of unfair labor practices that continued right up to the day of the election," Smith said.

He said his union was not consulted before the election was held, did not have an opportunity to review the questions before they were asked, and was not allowed to monitor the polling.

"The Senate would not put up with such abuses among other private or public employers," Smith said. "It should not tolerate them within its own confines."

A rules committee spokesman declined to comment.

In the past, union organizers and some cafeteria workers have complained that there is no procedure for airing grievances, that they have no job security and that they are not treated with respect.