The United Auto Workers and General Motors Corp. late tonight broke off talks aimed at an unprecedented agreement that would have lowered car prices through reducing labor costs.

Both sides had hoped that such an accord would halt the sharp decline of car sales for the nation's No. 1 automaker and block the march of more autoworkers into unemployment lines.

"We tried hard and we think the company tried hard," a downcast, sweater-clad Douglas A. Fraser, the UAW president, told reporters at 11:30 tonight, a half-hour before the union-set deadline to come up with an emergency agreement ran out. The new contract would replaced the current contract, which expires Sept. 14.

"It's all over . . . .It just didn't go together," Fraser said of the three-week, off-and-on effort. He said the talks broke down over three key issues: job security, use of outside contractors--particularly foreign firms--and how GM would carry out its promise to reduce sticker prices through labor cost savings.

Tonight's impasse marks the second time in eight days that the talks have broken off. Fraser said the decision to end negotiations tonight means GM will have to wait until regularly-scheduled bargaining in July to try to win the labor cost savings it says it needs to compete with foreign-car makers.

The collapse of the talks also means that both the union and GM have lost face with customers who were expecting to purchase GM cars at anticipated lower prices resulting from a wage and benefit concession contract.

"Everybody has been holding back" waiting to buy the cars at lower prices, Fraser said. "I think the consumer will be angry with both GM and us . . . GM is worse off today than it otherwise would have been" if the consumer rebate proposal had not been announced on Jan. 12.

Alfred S. Warren, GM vice president of industrial relations, also expressed deep disappointment. "We had hoped the union would recognize the reality of the problems facing the American automobile industry and the impact of those problems on the job security of the people they represent. The agreement to pass through lowered labor costs to consumers could have been a stimulus to the economy and a benefit to everyone--our employes, customers, dealers and stockholders," Warren said.

The GM executive said he is "hopeful that the forthcoming September 1982 negotiations will lead to a long-term agreement that addresses our mutual concerns."

Fraser has conceded that his union's negotiating position will be greatly diminished in September. Now, he said, the union is in a stronger position because it has something GM needs--the power to grant an early settlement that would allow the car-maker to save millions of dollars in labor costs before September. Tonight, he said, the union lost that chip.

Both Warren and Fraser said tonight that the union could also lose more jobs in the interim. Some 139,000 GM workers have been laid off indefinitely. An estimated total of 229,000 UAW workers are laid off throughout the industry.

Warren said tonight that he is not certain how many more layoffs will occur as a result of failure to reach an early settlement. But he said that his company will have to review its plans to shut down plants which have fallen below production.

Similar talks have been under way with the Ford Motor Co., which lost more than $1 billion last year, Fraser said that tonight's breakdown of GM's negotiations will not affect the Ford effort.

GM, faced with steadily declining car sales, says it no longer can afford the current contract, which carries an average hourly labor cost of about $20 per covered worker.