They used to call Olney the crossroads of upper Montgomery County.

That was back when the small, rural community was just a country ride from Washington. When the historic Oleny Inn Restaurant - dating back to 1835 - was a fashionable stop for a Sunday brunch. And when the old blacksmith and tin shops and corner grocery and drug stores were prominent buildings in the partof town.

It's all gone now. Just a fading memory eased out by bulldozers, shopping centers and housing developments.

For the first time in more than a century Olney is facing an identity crisis.

It has no mayor, no town council, no structures to conjure up the past, and more rapidly than some residents would like, it is becoming an invisible part of Montgomery County's urban sprawl.

The town, which is not incorporated and was once centered at the intersection of Georgia Avenue and Sandy Spring Road (Rt. 108), has grown from a population of about 10,000 in the 1960s to more than 20,000 persons now and is still growing, according to county planners.

In the meantime, Olney has become the home of the largest Safeway store on the East Coast; it has a shopping center and a shopping mall under construction and five banks within walking distance of each other.

This growth, according to county planners, has caused tremendous traffic jams on Georgia Avenue - the two-lane road that has snaked through the middle of Olney for more than a century.

As a result, Georgia Avenue is being expanded to a four-lane highway, and the intersection in the middle of town will be widened. The expansion has already prompted demolition of the rustic blacksmith shop, the tin shop and the corner drug store.

The real blow came last March when the Olney Inn burned down. Fire officials said the fire was due to arson. The property is now for sale.

The rapid changes that have brought an army of bulldozers to Olney have received mixed reviews by many of those who call Olney home.

"As far as I am concerned they have ruined the old town . . . It's terrible . . .," said Francis Hawkins Jr., the 65-year-old owner of the corner grocery store at the town crossroads, which has been in operation since 1934.

Hawkins, who was married at the Olney Inn in 1933 and bought land for his store from the owner of the Inn, has strong feelings about the way Olney has developed. He said he believes his small community has fallen victim to the county because the town was never incorporated.

He also said county planners should have developed a road around the town instead of expanding the intersection and widening Georgia Avenue.

Hawkins is so upset by the way the county has developed Olney that he has filed a suit in the county circuit court for damages to his business. He says the expansion of Georgia Avenue and the intersection has hurt business. He also said the county is now trying to acquire his property in its effort to modify the intersection.

Dudley Finneyfrock, the town's 65-year-old blacksmith, whose shop has been in Olney ever since his grandfather first tied horses to the maple tree in back of the old establishment, has become philosophical about the changes in Olney.

"There has to be progress . . . and we still have our memories," said Finneyfrock.

Finneyfrock is building another blacksmith shop just behind the spot where the old one stood. He said he is conducting a poll to determine whether he should cut down the old maple tree that has been on the property for more than a century.

Another businessman, Clyde W. Unglesbee, the owner of Olney Electric Co., said: "I sort of feel the main thing that happened is that we lost out identity when Olney Inn burned and when the old buildings and landmarks disappeared . . . The greatest job ahead is building a new identity for Olney."

Unglesbee added that he is optimistic that people will adjust to a new Olney when the widening of Georgia Avenue is completed.

Others are not so optimistic. Like 26-year-old Karen Collins, who recently moved into an Olney subdivision. "It's soon going to get to the point that Olney will look like Wheaton and Silver Spring," she said.

The recent changes in Olney have not been ignored by Montgomery County planners. They wrote in the draft of the Olney Masterplan: "Within the last several months, many events have occurred which have altered the landscape of Olney. The loss of historic structures has been most unfortunate. Coupled with the rapid pace of development, Olney is in a period of 'future shock.'"

The planners, who see Olney's future as a "satellite community" bordered by open space and low-density development, are concerned that Olney must again develop an identity.

Lyn Coleman, the Montgomery County planner responsible for Olney, said although Olney's housing stock doubled between 1970 and 1976 - and is continuing to grow - she hopes the county will limit the growth to about 32,000 people, or 5,000 new homes by 1996.

One of the key elements of the master plan is the development of a town center that would clearly identify Olney as a town. A town center, according to the master plan, would be a shopping area that provides theaters, restaurants, offices and public open space.

"Not everything has to be cutesy," according to Robert Sokolsky, president of Fordham Development Company, which is developing a 35,000-square-foot shopping center in Olney.

Sokolsky, who noted that "some things are antiquated in the 20th century," contends that old buildings are not the only way to identify the town.

He said the old town buildings could be replaced with "well thought out alternatives," such as the planned town center.

The developer said a building moratorium, which was lifted recently, had delayed construction that ordinarily would have have been natural for Olney. As a result, he said, building has been carried out in the last few months with "gusto."

In this town where residents say it is "prestigious to live" and where homes range from $80,000 to $100,000, Sokolsky said he has been so charmed by the residents that he is considering moving to Olney himself.

But this bedroom community is no longer the crossroads of upper Montgomery County, according to residents. The tiny intersection has been lost in the growth of expensive sub-divisions and the new shopping centers that seem to mirror the ever-growing country.

This growth has been particularly noticeable to Harry Musgrove, who - until 10 years ago - carried all the mail for Olney, Brookville and the Brighton Dam area, by himself. Now, he said, it takes more than 10 mailmen to serve the same area.