Prime Minister James Callaghan is widely expected to call a general election here for Oct. 5 but all he would tell his trade union allies at their annual conference yesterday was that he will announce his plans "very shortly."

He did, however, bring the union chiefs to their feet with a slashing attack that pictured his Tory opponents as apostles of prejudice, division and heartlessness towards the poor. His labor government, Callaghan insisted, was a model promoting tolerance and unity with a heart.

If this sounded like the warmup to a campaign, it is not surprising. Aldes say Callaghan has all but decided on the Oct. 5 election date. There remain, however, powerful men in his Cabinet, notably Denis Healey, Chancellor of the Exchequer and Michael Foot, the government's leader in the House of Commons, who are urging a delay until spring.

Callaghan, however, is understood to be impressed by a poll produced yesterday by the Labor Party's American pollster, Robert Worcester. It shows that 71 percent of those sampled and with an opinion want an election next month.

Callaghan is said to fear that a postponement will make him look like a man clinging to straws, hanging on to power for its own sake. This might undermine the dignified image he has created.

The Worcester poll, however, also shows Labor trailing the Conservatives 47-45. That could be one reason for Callaghan telling political commentators yesterday at Brighton: "Don't count your chickens before they are hatched."

Callaghan's political arithmetic is said to run like this: The third party liberals now hold only 5 or 6 percent of the electorate. They will go up to 10 percent on election day, drawing largely from Tories for whom Margaret Thatcher, the Conservative leader, is too mucs of a right-wing idealegue.

In addition, the Labor optimists reason, Callaghan campaigning, against Thatcher will add another three or four percentage points to the government's total.

In effect, the argument goes, a presidentially-style campaign of personalities will bring victory to Callaghan and Labor in a narrow squeak. This assumes that Thatcher will not be able to exploit economic Issue - inflation, unemployment, three years of falling living standards - to become Britain's first woman prime minister.

The prime minister gently chided his faithful allies yesterday for refusing to back a fourth year of anti-inflationary wage restraint, for declining to endorse the government's target of 5 percent pay increases. He warned them, "If you exert your muscle and secure wage settlements higher than 5 percent," inflation will rise above the present 8 percent level.

Callaghan's ready acknowledgement of the differences between himself and the Trade Union Congress also suits him politically. His strategy is predicated on the notion that he is a prime minister of all the people and Thatcher represents a narrow few. He saved his heaviest political fire for an attack on Thatcher's call for curbs on Caribbean and Asian immigrants.

"I serve notice," Callaghan said, "that we shall match tolerance against prejudice, policies against slogans, cooperation against conflict, unity against racialism and sectarian divisions.'

Conservative leadership, he said, was "hard, uncaring, abrasive." on the issue.

Those on the floor enjoyed Callaghan's performance but youngsters in the gallery and outside the hall were less impressed. "Right to work" demonstrators dramatized the fact that unemployed here is counted at 1.5 million, or 5.7 percent of the registered work force.

When Labor took power nearly four years ago, the jobless total was only 500,000. This will be a big Tory talking point.

About 50 of the several hundred demonstrating youths jostled Callaghan as he entered the hall and heckled him throughout his speech. He angrily challenged one, "If you want a job now come and see me afterwards and I'll find you one."

In fact, unemployment, particularly heavy among the young, will cost Labor among its own ranks in an election now.