The bankruptcy judge overseeing the reorganization of Manville Corp. refused to hear objections yesterday to the plan announced by the building manufacturer last week to set up a $2.5 billion trust fund to compensate asbestos victims.

Instead, Judge Burton K. Lifland told lawyers to return to the bargaining table to try to negotiate the details of the complicated proposal, the completion of which would enable the beleaguered company to emerge from bankruptcy proceedings. He told the lawyers to report back to him on Aug. 13 about their progress.

Under the proposal announced by Manville, victims of asbetos-related diseases would be compensated from a trust that would be funded by more than $1.6 billion in cash from a bond sale, $600 million in insurance proceeds and 50 to 80 percent of the company's common stock.

Plaintiffs could sue the fund, but not the trust, if they believed the settlement too low, according to the terms of the agreement.

The proposal was drawn up by New York lawyer Leon Silverman, who was appointed by the court to represent the interests of future asbestos victims.

While lawyers for current asbestos plaintiffs said they approved of the broad outlines of the plan, other parties to the dispute have raised objections.

A lawyer for 30 state governments that have filed $40 billion in property damage claims yesterday tried to protest that he was not consulted, but he was cut off by the judge.

Although more than $80 billion in claims in property damage claims have been made against Manville, the proposal sets aside only $50 million to deal with them.

Manville's shareholders, who could see their common stock diluted by as much as 80 percent, constitute the other party that has objected to the proposal.,

Although it is a financially healthy company, Manville filed for bankruptcy in August 1982 to protect itself against an onslaught of lawsuits.

At the time, more than 16,000 claims had been filed against the company, and the rate was increasing by 400 to 500 per month.

The company said it faced potential liabilities of more than $2 billion.