The four Democratic mayoral candidates yesterday agreed to debate on prime-time television the week before the Sept. 14 primary election, but only in the informal setting preferred by Mayor Marion Barry and his aides.

Barry and his three Democratic opponents confirmed yesterday that they would appear together on a WDVM-TV Channel 9 prime-time talk show on Friday evening Sept. 10, four days before the election, to be interviewed by an anchorman and a reporter in an informal newsroom setting.

However, after a morning of brainstorming, Barry's aides turned down an invitation from WJLA-TV, Channel 7, for the mayor to take part in a more formal one-hour televised debate with the other Democrats the evening of Sept. 8. WJLA's public affairs director said the offer remains open but the debate would not be held without all candidates' participation.

The Channel 7 offer was made shortly after Patricia Roberts Harris, the mayor's chief Democratic challenger, told reporters Tuesday that it would be a "tragedy" if the four candidates did not debate on television before the election.

Harris failed earlier this month to coax Barry into a one-on-one televised debate with her. She and City Council members John Ray and Charlene Drew Jarvis, the other two Democratic candidates, accepted the Channel 7 offer. But Lea Adams, the mayor's press secretary, turned it down yesterday.

Sharon Pratt Dixon, Harris' campaign director, accused Barry of dodging the proposed televised debate for fear of stumbling and jeopardizing his lead in the polls -- a charge denied by Barry's aides.

"I can't fathom how anyone who is ahead in the polls and who is so far ahead in raising campaign contributions has any disquiet about a format that should be to his advantage if he's as good as he says he is," Dixon said.

Dixon said that voters would derive far more information from a highly structured debate -- similiar to the 1980 presidential debates -- in which candidates discussed issues in detail and were subject to rebuttal, than from a free-wheeling interview show.

Aides to the mayor said yesterday that Barry has performed well in a wide range of public forums and that he has no fear of meeting Harris on television.

"The mayor does fine in any forum," said Ivanhoe Donaldson, Barry's campaign manager. "He's able to articulate his position with a great deal of clarity."

However, Donaldson and other Barry aides indicated they were peeved with the Channel 7 proposal because it seemed to be a direct outgrowth of Harris' proposal for a televised debate. They said they didn't want to appear to be reacting to a Harris proposal.

Also, the aides said they had been dissatisfied with Channel 7's handling of a recent program in which the mayoral candidates were interviewed and that they believed the proposal for a televised debate had too many loose ends.

Barry and his three Democratic challengers will be interviewed on the Sept. 10 WDVM TV program by anchorman Gordon Peterson and reporter Bruce Johnson in an informal newsroom setting, where coffee and doughnuts will be served.

The show will run from from 10 to 11 p.m. and then break for a half hour before resuming at 11:30 p.m. for an additional 1 1/2 hours, according to Betty Endicott, the station's news director. Donaldson said yesterday that that Barry has not decided whether he will remain for the second 90-minute segment of the show.

In an interview yesterday, Ray said a televised meeting of the candidates shortly before the election is likely to have a significant impact because many voters are still undecided.

"Most voters, even those thinking of voting for a particular candididate, will be impacted by the debate," he said.