THE SIGN hanging on the country club fence is unmistakable: "No Jews, Dogs or Coloreds Allowed."

For Ben Kurtzman (Ben Foster), Sheldon (Evan Neumann) and Murray (Gerry Rosenthal), this is another rude reminder that they are Jewish kids in Baltimore during the segregated 1950s.

But in Barry Levinson's affecting, bittersweet "Liberty Heights," the times are changing. Desegregation is on its way, albeit slowly. And Ben, who develops a crush on his high school's first black student, is about to get a personal sense of what it means to cross racial and cultural boundaries.

His family members, too, will find themselves dealing with a changing America. Ben's father (Joe Mantegna) and his illegal numbers racket lead to desperate dealings with a black petty gangster (Orlando Jones); and Ben's older brother Van (Adrien Brody) becomes infatuated with Dubbie (Carolyn Murphy), a blonde gentile who hangs around exclusively with WASPs.

You will find directorial craftwork in Levinson's commercial successes, such as "Rain Man," "Good Morning, Vietnam" or "Wag the Dog." But you won't find the personal conviction of "Liberty Heights," the fourth in a series of semi-autobiographical, Baltimore-set films, which includes 1982's "Diner," 1987's "Tin Men" and 1990's "Avalon." After watching "Liberty," you leave with a deep sense not only of his wit but his humanism.

Nate Kurtzman (Mantegna) and business partners Louie (Charley Scalies), Charlie (Richard Kline) and Pete (Vincent Guastaferro) run a burlesque club. But the real money comes from their numbers racket. Unfortunately their new, get-rich idea -- to leverage their profits with greater winning combinations -- gets them into trouble when Little Melvin (Jones) wins big.

Without a hope of paying Little Melvin the $100,000 he just won, Nate and his cronies offer everything they can, from his prized Cadillac to a cut of their business territory.

The financial fate of Nate's family (including Bebe Neuwirth as his wife) hangs in the balance as Nate continues tense negotiations with Little Melvin. His sons -- blissfully unaware of Dad's grim situation -- get into their own kind of trouble. No one is happy. But somehow, just about everything is funny, touching or guilty fun.

Ben, who has never met an African American before, falls deeply in love with Sylvia (Rebekah Johnson), the intelligent, elegant girl in his class.

Little by little, the two become friendly, until Ben is learning about her world, which includes her father (James Pickens Jr.), a wealthy doctor, and her love for a certain wild, hot new sensation called James Brown. Ben goes crazy for this "other music" and even crazier for Sylvia, despite admonitions from all sides.

Van, his brother, has a harder time of it. Dubbie, it turns out, has a boyfriend called Trey (Justin Chambers), a charming, reckless partyer who seems to be intentionally pushing Van into Dubbie's arms. Passionate for Dubbie, he's not sure how to consolidate his love for her with his growing friendship with the easygoing Trey.

"Liberty Heights" demonstrates what writer-director Levinson does best: evoke the sights, smells and atmosphere of his youth with intelligence, humor and a keen sense of social perspective.

As with "Diner" and "Tin Men," he animates his characters with unforgettable charm. Whether the subject is the seedy stuff of locker rooms, the insidiousness of anti-semitism, or the social differences between Ben and Sylvia, the conversations in this movie are priceless. Whether it's the gold-toothed, swaggery Little Melvin; the sweet-natured Ben; or Van's amusing pal Yussel (David Krumholtz), who dyes his hair blonde, pretends to be Scandanavian and goes out dancing with shiksas, you want to spend limitless time with every one.

LIBERTY HEIGHTS (R, 122 minutes) -- Contains obscenity, racial epithets, sexual situations and violence. Area theaters.