What began as a jam session 11 years ago in the back yard of Bill (Pigfoot) Harris, a jazz guitarist, has evolved into a Labor Day tradition that drew 500 people into his Northeast Washington neighborhood yesterday.

The event was called the "Labor Day Jazz Fantasia," and it kept the relaxed audience in a grove of oak trees in the Langdon Part East Recreation Center until a sudden downpour sent most spectators scurrying for their cars.

Sponsored by the D.C. Recreation Department, the program was designed to provide entertainment for people in their own neighborhood as well as to provide a showcase for young jazz musicians, according to Carolyn Mills, the program coordinator.

"It's a nice way to end the summer," said Lionel Hough, whose summer job at the adjacent Langdon Park swimming pool ended yesterday. He will attend classes at the University of the District of Columbia.

Yesterday's program featured several musical groups, including "The Three of Us," students at Howard University, and Noble Jolley, a guitarist who was last year's winner in a contest sponsored after a workshop conducted by Harris and Kenny Burrell, another noted jazz guitarist.

Burrell was the headliner on yesterday's program, but the rain came before he arrived. Several in the audience said it was his presence that attracted them to the performance.

Harris,who owns a nightclub featuring live entertainment, said the annual jazz concerts began on Labor Day 1968 in his back yard, just up the hill and within sight of where the Recreation Department set up its portable stage yesterday.

With Burrell as one of the players in what was then a sort of jam session, "we used to sell tickets," Harris recalled. But the event began to draw crowds, chiefly of neighborhood youngsters, who ringed and sometimes jumped the back yard fence.

"That's when we enlisted the help of the Recreation Department to make it a public event," Harris said. "It's something big for the people who don't like to drive out of town on the weekend."

Richard Wilson, a business researcher, said he liked the idea. "We need more of 'em," he said.

Another member of the audience, Elsa Malloy, said she wished such programs could be held the year around. "People always like to hear good music," she said.