Little seems to come easily any more for O. Roy Chalk.

The head of the D.C. Transit System, Inc., which used to own the city's buses, has long been one of the Washington area's most picturesque business figures. For more than 16 years he drove the bus and streetcar network with the panache of a test pilot. At his elaborate Walnut Hill estate he wined and dined congressmen, candidates and even presidents.

But in the eight years since Chalk's control over the bus and streetcar system ended, he has had to battle legal and financial headwinds.

Costly lawsuits accompanied the switch from private to public ownership of the transit system. One of the heaviest blows came in 1973 when the U.S. Court of Appeals ruled that D.C. Transit's last fare increase in 1970 had resulted in excess profits. The court also held that the sytem's riders -- rather than its stockholders -- should have benefited from increases in the value of various streetcar rights-of-way that it owned.

Chalk's Annandale estate, Walnut Hill, which was virtually a private country club, became a financial albatross. It went on the block in 1972 for more that $1.5 million, remained unsold and was converted into a luxury condominium development.

But of the 80 town houses he planned to build there, and sell for around $225,000, only nine have been finished and three sold. Chalk -- who is offering to finance purchases himself -- recently dropped the interest rate from 13 1/2 percent to 10 1/2 percent. He says he's trying to "set an example" for the beleaguered housing industry.

And now Chalk finds himself in another touchy suit -- one he filed himself -- that could determine the viability of a town house development he wants to build near Georgetown University.

Chalk says that a tiny plot of ground -- less than a tenth of an acre -- is virtually all that stands in the way of his development, which his attorney says would be built at the southwest corner of the university on a strip of former streetcar right-of-way above Canal Road NW. Chalk hopes to build at least 15 town houses, priced at about $500,000 each, on the wooded tract, which D.C. Transit still owns. He said he also is interested in building town houses on the right-of-way west of Foxhall Road -- and eventually as far up the eight-mile former trolley route to Glen Echo as there is demand. The area west of Georgetown has been the scene of considerable luxury-house construction in recent years.

The problem with his plans for the university area is that the U.S. Park Service owns adjacent land Chalk needs for a road, and it doesn't want to give the property up.

Chalk, who declines to discuss the case for publication, has appealed to President Reagan. In a letter to the president in March, he wrote that the park parcel is "the key entry" to the property he hopes to develop because it is essential for a water supply line and an access road.

The disputed plot had been part of the planned Glover-Archbold Parkway, but District officials decided not to build the road and gave the land along its route to the Park Service between 1967 and 1969.

Chalk and his lawyers, from the Washington firm Wilkes & Artis, say that the District made a mistake in deeding the plot to the Park Service. Late last year Chalk filed suit in U.S. District Court here, arguing that the section of the parkway in dispute should have been granted instead to D.C. Transit because it owned the land on either side.

The Park Service officials say they are determined to block Chalk's challenge. In a filing made last week, lawyers for the service argued that the District had the authority to give the disputed plot to the federal government and that even if Chalk had a right to the property back in 1967, he has lost it now.

The Park Service has objected to Chalk's development plans all along, mainly on aesthetic grounds. Manus J. Fish, director of the National Capital Region, worte Chalk in 1978 that the planned development "would impinge on the natural and unspoiled character of the Potomac gorge" and was "not in the best interests of the United States."

Chalk complained in his letter to the president about "environmentalists who are overplaying the concept of protecting Mother Nature in preference to protecting people." Washington, he wrote, should be a place "where the focus is on people and not wildlife."

But he may be facing a challenge by the neighborhood as well.

"The Park Service is concerned with protection of the pallisades area, and that's what we would support," said Paul E. Smith, president of the Foxhall Community Citizens Associaiton. Smith said it's too early to guage the impact of Chalk's development plans, baut said the association already has a grudge against D.C. Transit over an abandoned trolley trestle in the neighborhood that Smith considers "an accident waiting to happen."

Community approval could prove important if Chalk needs to change the property's zoning or if he runs into trouble with any of the multitude of review boards that govern building in the city.