In the late afternoon the vacant building looms over Mrs. Edward B. Burling's garden. Its shadow falls on the antiques in the library, on the oriental rugs in the center hall.

It was built by a Vanderbilt-turned-countess, but now it belongs to Saudi Arabia. The ambassador, who lives next door on Massachusetts Avenue, would like to use it for office space - if the law allows.

"I'm quite irate about it," says Mrs. Burling, who has lived in her home on Benton Place since 1928. "Why should we have another chancery in the best part of town."

Last week she and her lawyer Thomas Corcoran (formerly of Franklin Roosevelt's "Brain Trust") spent four long hours at a zoning, hearing in the District Building's drab room 11A. They were there with their neighbors - some of the wealthiest and most influential people in the city - to fight what several feared would be a last, losing battle in a long war.

The controversy about where embassies and especially chanceries (their offices) will be allowed in the District of Columbia has plagued Washington almost since its founding, involving as it does political issues from the ward level to the international arena. Now with home rule, a federally established National Capital Planning Commission, and the almost 60 new foreign missions established here since World War II, the issues are more complex than ever, and they are still just as emotional.

Saudi Arabia is only one of several countries, from Finland to Bangladesh, now looking for new chancery space. But it has rather more resources to bring to the fight.

One of Saudi Arabia's lawyers. Fred Dutton, was an assistant secretary of state in the Kennedy administration, he is now on a retainer of $200,000 a year to help the Saudis with such matters as the new petrochemical contracts with five major U.S. companies. But he went last week to the same zoning bearing in 11A, the first he had been to since he was fresh out of law school 25 years ago.

On the other hand, former senator William Fulbright is on a Saudi retainer of $50,000, but he sent a letter to the District Building opposing the rezoning. The new chancery would be in his neighborhood.

What the National Capital Planning Commission, the State Department and the Municipal Planning Office are proposing, (once they iron out several jurisdictional problems) is to create a kind of super-zoning - literally an overlay of the present zoning map - that would allow chanceries in certain residential as well as commercial areas of the city, including those along Massachusetts Avenue.

In December the State Department tried to have a "state of emergency" declared which would put at least part of this plan into immediate effect, thereby allowing Saudi Arabia and Bangladesh to move offices into buildings they already own along Massachusetts Avenue. But the neighbors fought the plan and the zoning commission postponed decision until the hearing last week. It did not make a decision then either, and won't meet again until Feb. 27.

As anyone driving along the stretch of Massachusetts immediately realizes, a number of embassies and chanceries are already there.

"If you can't have a chancery along Massachusetts's Avenue," Dutton asked, "where can you."

But homeowners fear that what is left of the residential character of their neighborhoods will be further eroded or destroyed if anymore of new chanceries are allowed to open, full of offices and junior diplomatic personnel, with cars to park, and diplomatic plates that allow them to ignore many traffic regulations.

"Nobody really minds the residences," says Elaine Dym, vice chairperson of the Sheridan-Kalorama Advisory Neighborhood Commission. "Of course you don't like it when ambassadors with nine children move in and throw their garbage all over the street, but basically they're all right. It's the chanceries - the offices - that everyone objects to."

The State Department, on the other hand, objects to the objections of the residents. "For God's sake," says State Department lawyer Harold Burman, "here we are trying to conduct diplomatic relations with these countries, and if we give them a tough time, then they have to wonder. "Does that mean the U.S. really doesn't care about us?"

There is fear of various subtle retaliations overseas. "It may come to a question of an we get cement for our contractor. You had problem where before there might not have been problems. If all they encounter here is long drawn-out zoning fights and angry citizens it doesn't help, to put it mildly."

In the case of Saudi Arabia, the U.S. has recently been looking for all the help it could get. Not only is there a need for Saudi oil, there is also an increasing dependency on Saudi assistance in international diplomacy.

With development of what President Carter has called "a deep sense of cooperation," Saudi Arabia's present chancery has become too small. Even people who oppose its move to Massachusetts Avenue have called its present facility at 1520 18th St. NW "woefully inadequate."

Moreover, the United States is itself facing a need to build several new chanceries and embassies overseas in the next few years - including one in Riyadh, Saudi Arabia, and Dacca, Bangladesh. Hence some of the urgency felt in Foggy Bottom.

The proposed International Center on the south side of Van Ness Street had been looked to as a partial solution. It was to offer chancery space to at least 10 nations, from Bulgaria to Israel. But the University of the District of Columbia still occupies two buildings that need to be cleared from the site, and the State Department cannot proceed until they are vacated.

The center is meant to be used by poorer (or, as the State Department call them, "lesser funded") nations. But even if it were available now, nations which could find a location on Massachusetts Avenue, residents fear, would make their chanceries there.

"They like to have a status address," says real estate broker Charles Pardoe. "They don't want to be stuck up there on Van Ness or The Avenue of the World or whatever the hell they call it."

Bangladesh for instance, is unquestionably a lesser funded nation with a per capita income of less than $100 a year. At present its embassy/chancery (actually the ambassador lives in Bethesda) is in a rented house at 3421 Massachusetts, far above "Embassy Row" and a couple of doors from a Church of Scientology.

The house that Bangladesh has bought, however, is at 2501 Massachusetts, near the Islamic Center and the Iranian Embassy, across California Street from Venezuela, just a couple of blocks away from Pakistan.

"It's a nice little house," says Ambassador Mustafizur Rahman Siddiqi, "not too big, not too small - just right for us."

It cost, according to the ambassador, $550,000. Another $127,080 is being spend remodeling it. The intention is to use it as a chancery, but in the meantime it is being redone with a building permit for a home.

The north side of the 2500 block of Massachusetts is primarily residential and the opposition to Bangladesh is especially vehement. The building is wedged between the private homes of lawyers Sidney Zlotnich and Kenneth McKinnon, and they have already gone to court twice in attempts to stop the constructions work.

Zlotnick calls the means used by Bangladesh to continue remodeling "subterfuge." and says he fears Bangladesh will use the building as an office regardless of the zoning decision.

"They have diplomatic immunity and they just ride roughshod over people," says McKinnon.

Siddiqi says that anything Bangladesh does will be in accordance with the law.

In the case of Baugladesh, however, the fear of some residents goes beyong questions of law or zoning.

"You can't really say this," said one resident of the immediate neighborhood, "but the truth of the matter is that it makes a difference which country it is that you're talking about. There are other chanceries around here Nobody bothers about the Japanese across the street for instance. But well, certain countries are more are more equal than others (nervous laugh). But really these emerging nations are, shall we say, a problem."

The problem of their chanceries, at least, is something that many people in the Sheridan-Kalorama neighborhood thought they had solved 14 years ago.

In 1964 Sen. William Fulbright - a long-time resident of the area - sponsored a bill to prohibit establishment of any new chanceries in residential neighborhoods (The French chancery is next door to Fulbright's house on Belmont Road, but he insists there is "nothing personal" about the law.) It is still in effect.

What was not clear at the hearing last week, and is not clear now, is whether the zoning commission has the authority to implement the designs of the National Capital Planning Commission and the Municipal Planning Organization insofar as "Embassy Pow" is concerned.

Fulbright, Corcoran, and the Sheridan-Kalorama Neighborhood Commission believe it does not.

"I was a member of the United States Senate in 1964 when the Congress enacted the Chancery Act," wrote Fulbright in his letter opposing the rezoning, "and am familiar with the purpose of that act." It is his opinion that the zoning proposal would "evade and defeat the clear intent of the law without repealing it. It would create confusion and instability in the areas concerned."

"You tend to protect your own little house and home first," said Dutton of the vehemence with which the homeowners are fighting.

Opponents of the rezoning describe their stand as a matter of principle. "If this plan is adopted," says Corcoran with a stamp of his cane, "the chanceries may invade anywhere.

"If I have to take off my pajamas to go to a downtown office, why don't they have to take off their pajamas to go to their downtown offices?" He vows to take the issue to court if the rezoning passes.

In the face of all this at least one nation looking for office space along Massachusetts Avenue, Finland, has been advised by its legal counsel to look elsewhere or be prepared to wait as long as four years for the legal right to move in.

If the new zoning passes and the house at 2929 Massachusetts is used for a Saudi chancery. Mrs. Burling says she will move out. "When my husband built this house," she says, "this was the country. I don't want to live among offices."