AS THE BUS pulled out of the Harborplace driveway, guide Linda Segal told riders, "You're not in the town of Baltimore. You're in Bawlmer. That's with no T." And so began another of Baltimore's now famous Insomniac Tours, a 41/2-hour late-night odyssey through a town hip-high in history.

Some 60 night owls had paid $24 each for the excursion, billed "A Different Time." They boarded at 10:30 p.m. intent on seeing the city in a different light, with running commentary about everything from Baltimore's Washington Monument to the star-spangled siege at Fort McHenry.

Inside the bus, the scene was reminiscent of a grade school field trip. Passengers peered out the giant tinted windows and chattered about the city at night. The bus slipped through the narrow streets of the city's oldest section, Jones Town, past the famous Shot Tower, an ammunition factory that took less than six months and more than a million hand-made bricks to build. Until the Civil War, it was the tallest building in the United States.

The facts and figures continued to fly as the bus chugged by the Carroll Mansion, home of Charles Carroll, a signer of the Declaration of Independence; the Flag House, where the "star- spangled banner" was sewn; and the Washington Monument on Charles Street at Mount Vernon Place. (Baltimore's monument, started in 1815, was the first begun in the name of the president. But because of funding problems, it was not completed until 1829, several years after another Washington Monument, in western Maryland, was completed.)

The bus cruised through Fells Point and Federal Hill -- two waterfront communities renovated and populated by yuppies over the past 15 years -- and Otterbein, where rowhouses sold to homesteaders for $1 in 1976 now begin in the $100,000 range.

Passengers also got a whiff of the McCormick spice factory and passed one of the nation's oldest daily newspapers, the Baltimore News-American, which carried battlefield reports of the Revolutionary War.

But getting off the bus was best, and here were the spots worth breaking for:

* At 2 a.m., the gates at Fort McHenry swung open and footsteps echoed along unlit paths leading to the courtyard where Old Glory waved over the battle that inspired the national anthem. In the darkness, the visitors stood in two rows, face to face, and stepped slowly backward for 20 paces, unfurling a giant, 30-by-42-foot flag to its full size. Before the stars- and-stripes was back in its packing bag, a park ranger had told the tale of Francis Scott Key and the battle of Fort McHenry.

"Unfurling the flag was nice," said Valerie Duncan, a head teller for Union Trust in Catonsville. "It was neat to find out just how large that flag was. I thought that was a nice touch."

* Although Babe Ruth never played any baseball games under the lights, he is one of Baltimore's brightest memories. The Babe Ruth House at 216 Emory Street where the Babe grew up is now a fascinatng museum of childhood and baseball memorabilia. It's also the home of the Maryland Baseball Hall of Fame.

* It may be a morning paper, but the finishing touches are added in the middle of the night at the Baltimore Sun. The roar of the rolling presses is enough to keep anyone from napping, and there's the chance to get a newspaper hot off the press.

* What better way to celebrate the middle of the night than with the dark poet Edgar Allan Poe. The group sipped sherry at his tomb in the crowded Westminster Churchyard, backlit by the streetlights from Greene and West Fayette streets, as "Annabel Lee" was read aloud. Poe, who died while passing through Baltimore in 1849, was originally buried along with his wife and mother-in-law in the back of the cemetery, but was moved to graveyard prominence after a "Pennies for Poe" campaign by locl schoolchildren produced half of the nearly $2,000 cost of transfer in 1875. The campaign continues today, helping to defray the cost of the grave's upkeep. * No midnight ride would be complete without a snack. At 2:30 a.m., the group returned to Harborplace for coffee and dessert at the American Cafe.

But why would someone want to stay up all night for something they could easily see in daylight?

"Because I've never done it before," said Barbara A. Ward, a Catonsville music teacher. "And doing it with a whole lot of people made it fun. I wouldn't have done it by myself. Going at night gave it a whole different flavor." NIGHT PROWLING

Since the Insomniac Tours began in 1977, Baltimore Rent- A-Tour's operator Ruth Fader has discovered that there's just too much of a good thing to see in one Baltimore evening. She has mapped out six different Insomniac Tours, with two left this season. Reservations are a must. Fader will also customize tours for groups of at least 40. Call 301/653-2998 for further information.

OCTOBER 27 -- Halloween Insomniac Tour. $30 per person. Begins at 1:30 a.m. (Saturday night to Sunday morning; remember this is the weekend that clocks turn back an hour at 2 a.m., but riders will remain on Daylight Saving Time until the tour is over.) This is the most popular of the Insomniac Tours; it includes, among other things, a costume contest for passengers, magic show, the catacombs near Poe's grave, trolley ride at the Baltimore Streetcar Museum, and a view of dawn from the top of the World Trade Center.

NOVEMBER 15 -- "City At Work" Tour. $24 per person. Begins at 8 p.m. WJZ-TV anchorman Jerry Turner leads a behind-the-scenes tour of the station followed by a stop at the new Animal Health Care Center at the Baltimore Zoo. Dessert and coffee are next at Baltimore's famous German restaurant, Haussner's, plus a look at Haussner's bakery operation. The evening winds up with a stop at the Baltimore Sun, just in time to pick up the first edition as it rolls off the press. D.C. BY NIGHT

Washington offers plenty of "Illuminated Tours," excursions that hit the lighted memorials -- Jefferson, Lincoln, Iwo Jima -- and breeze through the Library of Congress. They begin at 7 or 7:30 p.m. and last roughly three hours. But there aren't any scheduled after-midnight tours.

"Maybe it's because we work so hard in Washington," says Irma Greenspoon, president of Guide Service of Washington Inc. Another tour company spokesman pins the lack of late- night tours on "the grey flannel, pin-stripped mentality," which for years has been credited with ending Washington dinner parties by 10:30.

In any event, a number of tour companies say they'd be happy to arrange a red-eye tour of the Federal City for groups of 40 or more and, in some cases, even smaller groups. For a complete listing of companies, write: Washington Convention and Visitors Association, 1575 I Street NW. 20005; or call 789-7000.