Sam Hall, administrative assistant to a string of Norfolk's mayors, could be as polite as any Virginia gentleman to the city's politicians. But privately he was furious at their inability to capitalize on Norfolk's biggest asset: miles of waterfront.

His hand would sweep across the Elizabeth Riverfront panorama outside his 11th-floor City Hall office, and he would fume. "Why, do you realize," Hall would lecture his visitors, "that there is no place in this city, no park, where you can go and sit and look at the water?"

It has taken almost two decades, but Norfolk has discovered what Sam Hall long ago realized: Few spots on the East Coast are better than downtown Norfolk for watching a rich parade of aircraft carriers, cruisers, destroyers, tankers, tugs and sleek oceangoing yachts.

"Gradually it became apparent to Norfolk's leadership that the water is a our primary asset," says Mayor Vincent J. Thomas. "It's why the Navy is here. It's why the port is here. We know we should now look to the water for our future because it was our past."

His city, which now dubs itself "Norfolk by the Sea" and stages an annual three-day harbor festival, is in the throes of transforming its moribund downtown waterfront into what Thomas and others hope will become a major tourist attraction.

On June 1 Norfolk's civic leaders will raise their glasses once again in hopes they have found the answer: a precise replica of one of Baltimore's Inner Harbor buildings and an eight-acre public park on the waterfront. Sam Hall should be delighted.

But will Baltimore work at the other end of the Chesapeake?

Norfolk officials literally are betting millions that it will. According to some estimates, the city has invested nearly $45 million in public money in rebuilding its waterfront and has placed $9.8 million alone in a two-story pavilion called Waterside that looks as if it were plucked out of Baltimore's Harborplace.

The man who inspired that gamble is none other than developer James Rouse, who has convinced Norfolk it can duplicate what he did in Baltimore and at Boston's Faneuil Hall Marketplace, albeit on a smaller scale.

To Rouse, however, the downtown project is not so much a roll of the dice (relatively little of Rouse's money is involved) as it is a matter of capturing a new American spirit--"that kind of yearning for a central festival marketplace."

Tick off all the reasons why Waterside won't work and Rouse will shoot them right back. "Not an expense-account city, not a rich city, not a tourist city? It's the same thing they said about Baltimore . . . but that's all changed."

The thousands of sailors who live on the ships here and wander aimlessly around on weekends are seen not as a handicap, but an asset because "there are not always enough opportunities for entertainment." The proximity of Williamsburg and Virginia Beach--the state's premier tourist attractions--are also pluses: "The critical masses of activities," as Rouse puts it, "each will help the others."

"No question, it is a smaller city," Rouse says, "and we don't have expectations as high." Waterside, he notes, will have five major restaurants, compared with 10 at Baltimore's Harborplace, a total of 80,000 square feet of floor space vs. 140,000, and should pull 5 to 6 million visitors a year vs. the 18 million Baltimore draws.

Still, for Norfolk, a city of 270,000 which is seeking a cure for the sharp population losses suffered in the 1970s, Rouse's plan is big-time. City officials who have longed for restoration of Norfolk to its once-dominant position in Hampton Roads are ecstatic, even though the first oyster has yet to be cracked at Waterside.

"It's finally taking off. It's happening and it's exciting,"says David H. Rice, who came to Norfolk 20 years ago as a city planner and now heads the local redevelopment authority. Rice, who has seen at least two other major projects planned for the downtown never get off the drawing board, acknowledges that "there is a certain amount of risk" in Rouse's project.

". . . But what are you going to do: sit back and do nothing?"

Sitting back has never been the Norfolk "establishment's" style. When a Virginia governor scoffed at its efforts to create a medical school, the city dug in its heels and built one anyway. When others questioned whether the school could ever achieve prominence, it attracted a pair of Baltimore doctors who opened an in vitro clinic that led to the first "test-tube baby" in the nation.

Now it's Norfolk's turn again. Waterside will open during a week-long celebration and will be followed by the annual three-day "Harborfest," an event patterned after he bicentennial "Tall Ships" parade in New York harbor. Its scale again is smaller but the event has drawn, city officials say, crowds of close to a million with seafood booths, sailboat parades, fireworks, and concerts in the Waterside area

Waterside will depend on a well-developed marketing plan to pull those tourists away from the Loch Ness monster in Busch Gardens and the surf at Virginia Beach. Double-decker buses will shuttle between downtown Norfolk and the oceanfront, and a ferry service is planned with downtown Portsmouth, just across the Elizabeth.

Williamsburg, a tourist market Norfolk has longed to tap, may be more difficult. Rice and others speak of snagging what they call "rainy day" tourists: families that will turn to Norfolk, perhaps in frustration, when week-long vacations at the beach or Williamsburg are spoiled by bad weather.

If Norfolk has its way, those gray clouds could have green linings.