DELAWARE IS A hardly noticeable blip beyond the concrete boundaries of I-95 winding its way from Philadelphia to Washington. Sneeze and you've missed the whole state, to say nothing of the conspicuous signs for the little city of New Castle. And that is all too bad because New Castle is a largely undiscovered delight. New Castle is a Williamsburg without the hype and the people in costume, a small town settled in 1651 whose ancient courthouse dome lit by night against a star-filled sky surely would stir the coldest heart. Nothing much goes on in this courthouse anymore, not since Wilmington's burgeoning development siphoned off most of New Castle's commerce, but never mind -- we are the gainers. New Castle looks today much as it must have looked when the village green was laid out under the direction of Peter Stuyvesant.

This one-time colonial capital is a town for walking, and round every corner in the historical district is a reward. The Green itself at Delaware and 3rd is lined by handsome old houses between two and three centuries old, whose marble steps are worn by the feet of long-ago generations. Half of 3rd Street is still cobblestones. New Castle has bowed to the automobile and repaved some of the old streets but left this to show us how it must have been.

History sits lightly on the residents of New Castle, who are accustomed to living in a page from a history book. On Sundays they walk up the road to Immanual Church, built in 1707, or to the old brick Presbyterian church across the street which was built in the same year. Intent on their business, they scarcely spare a glance at the old arsenal on 3rd which their forebears built in 1809 to store arms against the coming war with England. In a 300-year old town, the past is just part of the neighborhood.

The graveyard at Immanual is kept locked these days since the church was badly damaged by a recent fire. Pieces of the grand old steeple lie scattered about the cemetary, even over the place where George Read, who signed the Declaration of Independence, is buried. But Immanual has survived a long time and work on the restoration is already underway. New Castle is where William Penn first set foot in the New World, stepping from his ship moored in the river to take possession of the land in the ''Livery of Seizen'," a traditional ceremony in which he was presented with a bit of soil, turf, twig and water. Legend has it he spent the night in the old house across from the Court House, now the William Penn Guest House, though this is not well documented as is George Washington's appearance at a wedding at the corner house down the street.

The historic part of New Castle is small -- ''seven streets of people,'' said the young man who served us breakfast at the bake shop across from the courthouse -- but it's so perfect one is surprised to see automobiles. The brick sidewalks have accommodated themselves to the ancient tree roots and there are diagonal parking slots here and there, but nobody has come round in the name of Progress to remove the hitching posts and the marble slabs for ladies alighting from carriages.

Delaware was the first state to ratify the Constitution but later in the Civil War, things got less clear. The cupola of the courthouse on the Green is the exact center of the Mason-Dixon line. The young men of New Castle were conscripted into the Union army, but the town's sympathies and money flowed south.

The pace of New Castle will soothe your soul. Walk about the town and poke into the bricked alleyways from which the handsome old boxwood gardens are so visible. Best not to filch any stray bloom from the cutting gardens, though, for even if nobody appears to be looking, New Castle is full of ''busybodies'' -- arrrangements of mirrors attached to an upstairs window which enable householders to see who is at their door and what is happening up the street. Many a New Castle homeowner has ignored a visit from a bore, forewarned by these handy lookouts.

In 1824, New Castle had a bad fire that swept up the strand by the river from the stables in the rear of the Jefferson House. A bucket brigade to the widow walk atop the home of George Read II spared this magnificent mansion, which is now open to the public.

Account books of the time it was built show a $1 entry on Sept. 14, 1797, for rum to treat the workman laying the cornerstone. Maybe it was worth while because in the end, under the direction of Peter Crowding, the master Philadelphia builder, they produced a house with the best of everything from the period, from Palladian windows to silverplated hardware on the solid mahoghany doors.

Part of the charm of New Castle is the small scale. The historic district lies within a town not much different than other small trowns on the Atlantic seaboard, but inside these limits everything looks as if it should be framed. Old gentlemen lean on canes catting on the benches of the Green, neighbors stop to gossip a moment outside the antique stores or the pub, children stand on tip-toe to drink from the fountain that once also served the carriage horses. New Castle is the antidote to the urban jungle of glass concrete.

At number 206 Delaware St., a discreet sign ''Guests'' is bolted to the front wall. This is the house where William Penn is supposed to have slept, the property now of Richard Burwell and his wife, turned innkeepers. It's a 1682 gem, standing as it has stood for hundreds of years between its neighbors facing the courthouse. If you can get one of the four bedrooms for a weekend, you have a treat in store.

Guests are really guests at number 206. When we arrived, a note under the beautiful brass knocker instructed us to get the key from the curator of the courthouse and make ourselves at home in the back bedroom. The owners, said the note, were attending a weeding and wouldn't be back till late. Like any weekend guests, we had been given the run of the house.

Our bedroom was small and cozy, equipped with a fireplace and hook rugs and brightened with an antique pitcher full of greenery of the bureau. Our bathtub was claw-footed and the price per night was $18.

We peered into the living room before we went out for dinner and found handsome ancestral portraits, federal furniture, chinoiserie and an extensive collection of antique glass. It's hard not to collect antiques in New Castle. The whole town is an antique and outlets for its artifacts are everywhere.

The Arsenal on the Green, where we had dinner, is pine-paneled and attractive, specializing in the crab and fish so handy to come by here. It is soon to have competition when Brock's of Philadelphia takes over the old arsenal building on 3rd Street as a restaurant; it's due to open Nov. 20. Renovations are already underway, but nothing will be done to disturb the patina of aged charm.

Breakfast Sunday morning is chatty and companionable at the Bake Shoppe, number 204. Everybody knows everybody else and sitting at the counter, you get to hear local talk and advice on what is not to be missed on a visit. They'll also sell you some good sticky buns, doughnuts and Danish made daily by their own baker.

Walk it all off afterward in the pretty park by the river, the waters of which -- sad to say -- are polluted but still beautiful. It's all so peaceful and bucolic, it's almost startling to meet an occasional jogger wearing sneakers. One comes to expect parasols and gowns sweeping the ground. wNew Castle is a time warp.

We sat on one of the benches and opened the guide book we had bought, stumbling on a letter written in 1804 by Elizabeth Lees of Liverpool, England, to her sister who had recently married and settled in New Castle:

"I expect your next letter will . . . again go over the history of New Castle, leaving out neither . . . births, deaths or marriages; whether G. Rr (George read) has got his elevated house, as the last I heard . . . he and his carpenters were at law together; how the hotels, inns, wharfs and shop-keepers come on. I should like to know something of American fashion. The style in which you dress . . . I imagine . . . differs . . . from the English; if you have good wigs and handsome ones. George, I am told, wears a black wig without powder."