When Conrail decided to abandon the rail route along the Eastern Shore two years ago, the future of this quiet town looked about as bright as the peeling paint on its aging Victorian houses.

But Cape Charles is back on track now as a railroad town with modest future, thanks to the Virginia & Maryland Railroad Co.

By national standards, it's not much to pin a town's hopes on - 47 employes, four locomotives and a single freight car.

But the V&M nontheless has made staggering achievements since it began operations April 1, 1977.

"The V&M is a fine example," said general manager J. Paul Carey, "(of) alternatives for the nation's railroads short of pulling up the rails and ties and creating bikeways and hiking paths across the United States."

In its first year the V&M cut Conrail's $6.6 million operating cost on the money-losing line by more than half to $2.8 million.

Freight now moves in a quarter of the time or less than it took Conrail or its predecessor, the bankrupt Penn Central Railroad, to get boxcars from one end of the line in Norfolk to the other in Pocomoke, Md.

The V&M increased freight traffic by 20 percent last year - the first time the line has not registered a decline in traffic since World War II.

The tiny line possibly could break even this year, relieving the need for what adds up to millions of dollars in federal operating subsidies that come from the taxpayers' pockets.

And all of this is happening on a railroad once given up for dead.

"We're not out to change the world," cautioned V & M President Tony Hannoldt. "We just want to keep (the railroad) alive."

Area businessmen are more blunt. "The line is essential to the economy of the Eastern Shore," said an executive of the Bayshore Concrete Corp., the largest employer in Northampton County with about 200 employes.

"People just don't realize how important this railroad is," said county supervisor Ed Parry, whose local lumber yard relies on the V&M for deliveries. "If they close this line, the entire East Coast would feel the pain," he said, noting that most of the V & M's freight is through-traffic that just passes by on its way north or south.

The history of railroading in this largely agricultural and isolated peninsula dates ot the 1890s, when the coming of the old New York, Philadelphia and Norfolk Railroad Co. led to the founding of Cape Charles. The town, which brugeoned and sagged with the railroad's fortunes, became the northern harbor for enormous barges that still carry the line's freight cars 26 miles across Chesapeake Bay to the southern trackage in Norfolk.

As business declined after World War II, the railroad's importance didn't. Unlike the main line through Richmond, Washington-Cape Charles route is relatively unobstructed and is therefore one of the few ways to move bulky, oversized freight, such as gun mounts the Navy uses in Norfolk.

It is the most economical route for moving combustible gases, now barred from the Chaespeake Bay Bridge and Tunnel to the Eastern Shore from the South. And rail service to southern Delaware depends entirely on the road's success.

V & M executives are frank about what separates them from the problems faced by Conrail: The railroad isn't bound by expensive manning clauses in the labor contracts Conrail inherited from the Penn Central. The Penn Central, for example, ran each of the two tug boats that haul the freight-car barges with a 14-man crew. By contrast, Portside Marine, a subcontractor that runs the bay operations for the V&M now, handles its entire operation with only a dozen workers - and has still found time to salvage the equipment from the shocking state of disrepair it was in.

"This boat looked like a pig pen," said Danny Baum, a weathered tugboat pilot for Portside. "I was ashamed to come aboard. Even the floor was all to pieces." Today, however, both tugboats, still leased from Penn Central, are beginning to take on some of the look they had when they were built in the 1940s.

The road to progress, railroad executives say, has not always been a smooth one. Carey recalls his first day on the job in July 1977, when the docking equipment in Norfolk broke, threatening to put the V&M out of business entirely. The next day, a 10-car derailment near Cape Charles shut the line down half a week.

Carey refers to the early months as the V & M's "ride 'em, cowboy days," when the trackage was so poor there was a derailment a day. Chief engineer Alan D. Szaks will show a visitor a stretch of track where date nails in the ties did major maintenance: 1953.

Today, 60,000 new ties later under a two-year federal repair program, the line is down to a dereilment a month. "And they're tame ones, usually just a matter of one wheel," Carey said.

There are still questions about the railroad's survival, however. One reason the line operates relatively cheaply is that it pays nothing for the right of way, which is leased from the Penn Central by Accomack and Northampton counties through a public transportation authority. The $500,000 annual rental will become an impossible burden on the local governments unless the federal government helps finance the purchase of the real estate, officials say.

Further, although the line has cut its federal operating subsidy from $900,000 two years ago to a projected $130,000 for the first six months of this year, there is still no guarantee that the V & M will break even by 1981, when federal subsidy programs for carriers like the line expire.

"About 3,000 more cars a year would put us into the black," Hannoldt said. Current annual traffic is about 10,000 freight cars. An increase hinges on Conrail, which has to provide good service on the line's northern end to make it competitive with other East Coast routes - such as Conrail's own.

Conrail has promised better service, but a similar promise last year never came true, and many shippers testing the V & M went elsewhere afterwards.

"We're really dependent on this service promise Conrail has made," Carey said. "We can become the fastest route from the Southeast to the North. Or Conrail can sneeze and blow us away." CAPTION: Picture, A barge hauls some of V&M's freight cars from cape. By Regan Kerney-The Washington Post