THE Kool Jazz Festival will again take over the Kennedy Center Sunday with multi-act afternoon and evening sessions full of stars.

Perhaps the festival's best kept secret is the impressive roster of Washington and Baltimore-based artists who will perform. The Eddie Phyfe Quartet, Vaughn Nark Quintet, Lady Rebecca Trio and Deater O'Neill with Mike Crotty are scheduled for the afternoon session. Southern Comfort, Three Saxes for Lester, the Arnold Sterling Quartet and Shirley Horn Trio with Richard Rodney Bennett are in evening slots. Solo pianists John Eaton, Eddie Hayter, John Malachi, Ron Smith and several others will take turns throughout the day and evening.

Southern Comfort leader and trombonist Al Brogdon says the decade-old group is not "strictly traditional Dixieland" and explains why. "I tend to choose tunes that are musically good that the band can learn and present well. To me Dixieland is a style, not a given list of tunes, if a tune lends itself well to that style, it's fair game. We pick up music from here, there and everywhere." He cites Duke Ellington, Count Basie and Willie Nelson as some of the group's sources. The group's first album, released earlier this year, boasts compositions by Fats Waller, Horace Silver and Jimmy Noone, "That's A Plenty" and other Dixieland warhorses, and several vocals by bassist Mike Pengra.

"Music has always been a very important hobby with me," said Brogdon who earns his living as a technical editor. "My dad was a high school band director and I started playing when I was 6. It was cornet at first and then baritone horn." Brogdon grew up in Tennessee, where he listened to "hillbilly music" and the pop sounds of the day and, beginning around 1950, traditional jazz on WWL from New Orleans. He is a self-taught trombone player, first picking up the instrument "at a very advanced age"--when he was 22.

Brodgon settled here in 1970 and began subbing in local bands. By 1973 he had inherited a group whose leader had left the area. "Nobody wanted to be leader so I took it by default," he says. The seven-piece group changed the name to Southern Comfort, and although the band has undergone inevitable personnel changes, for nearly eight years it has held down a Friday night gig at Shakey's Pizza in Rockville, becoming a virtual institution in the process.

Pianist Eddie Hayter began his present six-night-a-week engagement with his trio at the Skylights Restaurant in Crystal City six months ago and says, "That's when it all came together." Hayter came to Washington in 1976 and "started really thinking about a career in music." He had known for a long time "that there was something I wanted to do and just couldn't put my hand on it." Actually, he had been making his living as a musician off and on since his teens, but for long periods was siderailed by drug problems.

Hayter grew up in the Sugar Hill district of Harlem and was taught piano by his mother, who was a dance teacher and performer with the Rhythmistic Brownies. She often took him with her to clubs to hear jazz greats and his father, a vocalist, had his own group. By his early teens Hayter was playing piano and percussion in Latin bands ("my people were basically West Indian").

"I fell into that trap, using drugs, and I was a forger. I never stuck up anyone or broke into anyone's home. I cashed checks to support my habit," he says. He served jail sentences in New York, and later served two years in Danbury, the federal prison in Connecticut.

"That's when I really got my music education," he says. By the time he "graduated" from Danbury, "I could read music, I could write and I could arrange. I had a 13-piece band in Danbury and one show we put on was all original music I wrote."

A program for former drug addicts helped him get on his feet after his release, and a bass-playing friend from Danbury persuaded him to come to Washington. "So I said maybe that's good that I got away from New York and the temptation of falling back into the cesspool of drugs again."