Presidential pollster Patrick Caddell joined the administration's who-needs-enemies club yesterday as Republicans gleefully circulated a Caddell poll indicating that Americans have some problems with President Carter's labor law revision bill.

The public opinion survey was taken by Caddell's Cambridge Reports Inc. for the American Retail Federation, one of many business groups fighting Carter's labor bill in Congress.

The poll, which was immediately attacked as slanted by some of the bill's proponents, shows that most people favor changes in current labor laws. But the changes they want are not always the changes that are proposed in the bill.

The legislation, which passed the House last year and is now being filibustered by Republicans in the Senate, would generally make it more difficult for employers to thwart union organizing and bargaining efforts by setting deadlines for representation elections and stiffening pernalties for illegal anti-union activities.

But the survey, based on 1,500 interviews between April 20 and May 15, showed that "the largest single 'reform' desired by the most respondents was a reduction of labor union power," according to a summary of its findings. The second "controls on strikes."

The survey also showed a 41-to-33 margin against changes to "make it easier for unions to recruit new members" and a 48-to-22 margin against easing the "rules by which unions gain the right to represent employes."

The Caddell poll was the latest skirmish in what has become a battle of the polls over the labor bill. The AFLCIO struck last week with a poll by the Public Interest Opinion Research Corp. showing strong support for the bill's purposes. It showed 73 percent saying they favored "making it easier for workers in large companies to vote on whether or not they wish to join unions" and 67 percent saying they favored the same for small companies.

Sen. Harrison A. Williams (D-N.J.), chief sponsor of the Senate bill, cited these examples in a tactfully worded rebuttal to the Caddell poll, noting that "many polls are commissioned and may be structured to elicit responses desired by the organization paying for it."

The survey, which is cost $15,000 and was labeled "strictly confidential," was given by the retail federation to Senate Minority Leader Howard H. Baker Jr. (R-Tenn.), who happily shared it with reporters yesterday morning. "We mentioned it to him, and the next thing we knew he wanted the whole thing," said Don White, vice president of the association.

White said Caddell's Cambridge, Mass., research firm has been doing work for the association for two years."We though it had creability and was pretty relevant in Washington these days," he said.

Caddell was not the only White House confident involved in the latest exercise in lobbying-by-polls on the labor bill. The federation's president is Loyd Hackler, a presidential aide during the Johnson administration who is close to Carter special assistant Robert S. Strauss and attended a recent Strauss-sponsored skull session on how to improve White House operations.

Caddell was traveling and unavailable for comment. John Gorman, vice president of Cambrige Reports, said Caddell, who is president of the firm did not work personally on the poll. Added Gorman: "If all our clients had to agree on everything, we'd probably have no clients."