Chestertown is the kind of town where a white-haired octogenarian named Miss Harriet Welch sits in the late afternoons on her porch until a 76-year-old black maid named Nellie "Belle" D. Jackson calls her in for supper; where students from the town college run through the streets every May Day without benefit of clothing and where the residents of the ritziest street are primarily Episcopalian and have been since the late 17th century.

It is a town where last year there were no murders, rapes or robberies and where the worst tragedy in years transpired this spring out on Smithville Road when a county commissioner's son was killed in a car wreck.

It is also the kind of town where black slums and white mansions stand within a block of each other, where whites pass blacks in the street and smile and ask after their babies, and where Fannie Wilson recalls the exact year (1935) when the "fourth black bathroom in the county" was installed in her house.

Chestertown, population 3,300, is not on the main road to anywhere. Located on Maryland's Eastern Shore, it is a two-hour drive from Washington, three and a half hours from New York. But few people pass through town en route somewhere else.

The 276-year-old town, 21 miles upstream on the Chester River from Chesapeake Bay, survives not as a quaint suburb but as a living specimen of a seminal American entity--the small town. Its inhabitants exist somewhere between "Our Town" and "Main Street," between idyllic neighborliness and claustrophobic provincialism. Norman Rockwell would find plenty to paint and Sinclair Lewis would be busy taking notes.

If you don't want to go to Chestertown, you probably won't. The Bay Bridge is 20 miles to the south, and weekend hordes from Baltimore and Washington pass unnoticed to the Maryland and Delaware beaches.

Townspeople want to keep it that way. They live in fear of a second Bay bridge that would connect the Upper Eastern Shore with Baltimore. That city, just 13 miles west across the Bay, is now a miserable 80-mile drive. Locals criticize ticky-tacky resort development near the Bay Bridge. They breathe easier, they tell each other at dinner paties, when they cross the Bay Bridge and drive north toward home.

From the south, the only way to drive into town is over the aging Chester River Bridge. From there, the town presents itself as a row of Georgian and Federal mansions, sprawling clapboard manor houses and tidy brick townhouses. Between the houses and the wide, mud-colored river are carefully tended lawns, white picket fences and flower gardens with pergolas and wrought-iron love seats. The view from the bridge is of Water Street, the oldest and richest street in town, the street where most everyone is an Episcopalian.

For about a century, from its founding in 1706 to shortly after the Revolutionary War, the port prospered. Merchants built opulent riverfront homes and founded, in 1782, Washington College, the oldest college in Maryland. George himself granted use of his name.

Then the same geography that still isolates the present town killed the port. Shipping went to Baltimore, and Chestertown became what it remains: a farming town with a few small industries, a court house and a hierarchy of town leaders--many of them descendants of original settlers--whose influence seems to derive from their proximity to the river.

Chestertown is the county seat of Kent County, the smallest county in Maryland in both in square miles and population. In 1910, there were 16,957 people in the county, which is 57 more than there are now. The dearth of new blood allows Chestertown to cling to a way of life that has changed little in half a century.

For example, Harriet Welch, weather permitting, has been sitting at cocktail time among relatives, friends and potted plants on her porch for more than 40 years. Miss Harriet (as she is known in town) is 83 years old and was born on the same street where she now lives--Queen Street, one block from the river.

"We have lived here in this house for 65 years and we moved here in 1918 from right across the street where we were born," says Welch, an amiable, sweet-voiced spinster who holds her hands in her lap as she speaks. For 25 years, she ran a private preschool for children of Chestertown's established families. She retired 15 years ago.

Miss Harriet drives a baby blue 1950 Plymouth sedan. When she goes shopping at the A&P, her black maid, Nellie Jackson, rides in the back seat. In the summertime, when her sister comes up from Florida with her husband, Welch spends many of her days this way:

"Usually I'm out on the porch part of the morning before lunchtime," she explains. "Lately I've been reading one of those paperbacks, a romance I think you call them, Spring Girls. I watch the soaps ("Days of Our Lives" and "Another World") after lunch. Not till about 4 o'clock or so do I go back on the porch. My sister usually comes out and we visit with each other and with the people who pass on the street. It is always pleasant. We stay out on the porch until dinner is ready, which is usually just about half past six. Nellie gets our dinner and she calls us in."

Nellie Jackson, whose nickname is "Belle," came to work for the Welches in 1943. She has been with them ever since. She cooks and cleans four days a week and makes $20 a week.

Jackson is a wispy, voluble woman who paints her fingernails red and wears a red wig. She claims to make the best sweet potato pie in town. She plays jazz piano by ear, snapping her fingers and dancing at the stool of the upright piano in her house as she plays. She went to work as a domestic for white families in Kent County 66 years ago, when she was 10 years old.

She lives just two blocks but a world away from Miss Harriet, in what locals call a "slum." Jackson's house, which she has lived in for 20 years and now rents for $16 a month, borders on an old coal yard. The tiny house is unpainted, has a leaky roof and doesn't have a bathroom.

"I have to go out in the coal yard to go to the bathroom or I go to the neighbors or when I am at work, I use Miss Harriet's," says Jackson.

She says she likes working for white people, but that in recent years a few of her black neighbors have accused her of letting white people use her. "They don't like me because I've got too many white friends," says Jackson.

There are 600 black people in Chestertown. They have one-third the income of whites and three times the unemployment. Not one black person in town, according to the 1980 census, earns as much as the average white family income of $24,356 a year. Segregation ended abruptly in the town's stores and restaurants in the early 1960s, but changes in racial attitudes have come much more slowly.

"Racism here is subtle because everyone's a friend. It is hard to put your finger on it because it just sort of oozes," says the Rev. John F. Holden of Janes United Methodist Church, an all-black congregation. Holden, who is white, has been at the church for a year and a half.

"Many of the blacks in this town have functioned at one point or another in their lives as servants in white homes," says Holden. "Those relationships last throughout people's lives. It is not the relationship of employe and employer, but of servant and served."

The major liberalizing influence in Chestertown is Washington College. The small private school spices the town with public lecture programs, antics of college students (ritual May Day streaking) and the annual hooha that surrounds the giving of the $35,000 Sophie Kerr Award, said to be the richest literary prize in America.

The town's survival, however, depends much less on the college than on the silt- loam soil of Kg ent County, of which Chestertown is the county seat. Farming is the county's most important industry, generating about $70 million a year.

The greatest change in the county in 200 years involves the sale of farm land. More than 10 percent of the tillable land has been sold in recent years to foreigners, a foreign- ownership rate 10 times higher than the national average. Much of the land has been sold to Germans and South Americans at prices up to $4,000 an acre. According to the Kent County News, foreigners have paid more than $28 million for 51 farms.

Wealthy New Yorkers and Philadelphians, known in town as "to-do people from the North," have also bought and restored several million- dollar, waterfront plantations in the county.

Despite the change of ownership, little of the farmland has been subdivided and the county continues to have the largest farms (averaging 360 acres) in Maryland.

Black or white, skilled or unskilled, there are few jobs in Chestertown or Kent County for young people, including graduates of Washington College.

"Almost all of them move away," says Elmer E. Horsey, Chestertown's mayor. A new industrial park on the edge of town has, thus far, has attracted no new major employer.

While lacking jobs, the town has no shortage of wealthy residents, both natives and newcomers. Many of them are devoted to historic restoration. Dozens of 17th and 18th century dwellings have been restored. Concrete sidewalks have been ripped out and replaced with bricks. Georgetown-like shops, such as Mrs. Kelley's Tea House, have opened and more are planned.

H. Hurtt Deringer, editor of the Kent County News, fears Chestertown could pickle itself in its own history and become a museum.

"There is a tendency to push people out of town, for the area to become too pristine," says Deringer, referring to the recent demolition of crumbling in-town black housing and the erection of housing projects outside the city limits.

Yet even without a growing business base, Chestertown in recent years has become increasingly attractive to a growing number of well-educated (mostly white) adults who fled the town after high school.

"When I was a kid, you were considered a loser if you stayed here. Now it is just the reverse," says Bill Ingersoll, 33, the town's housing administrator and a recent returnee. "For those who can find a way to come back, this is real wood around here and everything else seems like plastic."