The paddle-wheel ferry from Norfolk, docking at Olde Towne in Portsmouth, Va. (James F. Lee/For The Washington Post)

“There’s a story in every brick you trip over.” Barbara Early was talking about Olde Towne, the historic heart of Portsmouth, Va., a city of 97,000 on the banks of the Elizabeth River.

Early’s enthusiasm for this small city is infectious. “In one square mile there’s so much to see,” said Early, an Olde Towne resident and history/docent coordinator at the Hill House Museum there. “There’s not any one reason to visit. There’s a lot of reasons. We’re a hidden gem.”

If you go: Portsmouth, Va.

My wife and I were on a walking tour of Olde Towne, a compact grid north of Queen Street and east of Green Street. We’d arrived earlier in the day, crossing the river on the paddle-wheel ferry from Norfolk. Looming on our left as we’d approached Portsmouth were three huge naval vessels, including an aircraft carrier undergoing repairs in dry dock. From the vessels came clanging and rumbling and the smell of diesel fumes. Tugboats and a military patrol craft cruised the river, while several helicopters flew by overhead. The Elizabeth is a working river, home to the Norfolk Naval Shipyard in Portsmouth.

Portsmouth’s shipyard history goes back to 1767, when Andrew Sprowle, a well-to-do Scottish merchant, established the Gosport shipyard just south of Olde Towne. In the ensuing years, succeeding generations of merchants, ship designers and builders, whose livelihoods depended on the ever-growing shipyard, built homes and businesses in Olde Towne.

Starting our walk on North Street, we strolled past Federal, Victorian and Romanesque houses, former taverns, lodging houses and businesses lining the tree-covered streets. Crape myrtles flowered with lovely pink and white blossoms, reminding us that this is a Southern city, as did the Confederate Monument to fallen soldiers at Court and High Streets.

The Grice-Neely House on North Street, an 1820 structure with a wrought-iron second-floor balcony, reminded us of New Orleans. Francis Grice was the last master builder of sailing ships for the U.S. Navy. He designed the flagship that Commodore Matthew Perry sailed on his 1854 voyage to Japan. Across North Street, the Patriot Inn, built in 1784, was once a boardinghouse and until recently a bed-and-breakfast.

Perhaps the best place to visit for the most complete sense of Portsmouth’s history is the Hill House on North Street. This three-story building is the home of the Portsmouth Historical Association and contains the furnishings of four generations of the Hill family, who lived there from 1820 until 1961. The Hills were a prominent Portsmouth family of businessmen and politicians. Hill House entertained four U.S. presidents over the years.

Among the interesting pieces in the house are two gorgeous rosewood secretaries, with ivory inlays, from Italy; an unusual cylindrical music stand; a Knabe piano, a famous 19th-century brand; and the family silver and china.

Portsmouth was occupied by the North during much of the Civil War, and several houses have Civil War connections. One of the most interesting is the Pass House, so named because that’s where the Union army issued passes to anyone wishing to leave Portsmouth. The Macon House, now a private residence, served as a hospital during the Civil War.

At the end of London Street, we boarded the Lightship Portsmouth Museum, an actual lightship that used to anchor at sea for months at a time to warn ships, via a light mounted on its mast, of navigational hazards. It was extremely dangerous duty for the crew of up to 15 men. Sometimes the lightships were rammed by the very ships they were trying to protect. And conditions on board were crowded. The tiny crew’s quarters held bunks stacked three high, while the captain’s quarters had a single bunk and just enough room for a small desk and chair.

At the Portsmouth Naval Shipyard Museum on High Street, naval artifacts, ship models and various display cases illustrate the history of Portsmouth and its shipyard through the Revolution and the Civil War and into the 20th century. My wife’s favorite display was a scale model of Portsmouth in 1774, showing the streets of Olde Towne and the early shipyard.

There are models of the Monitor and the Merrimack (a.k.a. the Virginia), the ironclads that fought their historic Civil War battle in the waters of nearby Hampton Roads. The Confederate navy converted the wooden frigate Merrimack to the ironclad Virginia right in the shipyard. I looked at an actual piece of armament from the Virginia, a small piece of metal that revolutionized naval warfare.

Corey Thornton, curator of the shipyard and lightship museums, said that one of the most popular exhibits at the shipyard museum is the yeomanette, a model, in full uniform, of the first women on active duty in the U.S. Navy during World War I. Most of these women worked around shipyards and other naval installations doing clerical work, although some were machinists or did medical work. They were precursors, in a sense, of the World War II-era WACs and WAVES.

“It’s one of our favorite pieces,” Thornton said. “It stands for something when women didn’t have the rights that they have today. It was the first real effort by the government to say to women that we need your help.”

For me, the most fascinating story at the museum concerns the German village, constructed at the shipyard in 1915 by German sailors whose ships were impounded before the United States entered WWI. This was a complete village with houses, shops, a church, a post office and even its own newspaper. A photo shows bearded and mustachioed men clutching beer mugs around a piano, one man cradling a dog in his arms. Another photo shows the village windmill.

Curiosity seekers paid admission to tour the village, whose residents, living in a sort of political limbo, were neither prisoners of war nor completely free. When the United States entered the war in 1917, the sailors became prisoners and were shipped to POW camps in Georgia. The village was torn down to enlarge the shipyard.

Just another story found in the bricks of Portsmouth.

Lee teaches journalism at Bucknell University.