The real action at Africa's culture festival is at this dusty, sand-blown and far-from complete high-rise complex housing participants from nearly 50 countries.

In theory the singers, dancers and actors should be saving their best efforts for the national stadium, the new theater or four other sites located miles away from where they are scheduled to perform.

But such is the official confusion bedeviling the second world and African Festival of Arts and Culture - FESTAC for short - that no one is quite sure when, where and how any play, dance group, poetry reading or concert is supposed to take place. Or how to buy tickets.

Guarded like maximum security prisoners by nervous Nigerian police and army troops, the performances put on a nonstop show at the village challeging neighboring delegations to dancing and singing competitions and otherwise comparing notes.

Much of the spontaneity would seen to stem from the fact that unlike its illustrious predecessor held in Dakar. Senegal in 1966, FESTAC boasts few superstars. A French-speaking dancer from Senegal remarked, "This time we are all illustrious unknowns." An American actor insisted, "Superstars wouldn't mingle - they'd stays in the big hotels in the city. Here we have a chance to learn from the Africans."

In general, the Americans are young and involved in community cultural life rather than showbiz, albelt perhaps out of necessity. As one young American performer said, "We're just regular people riding the train - would-be celebrities still paying our dues."

The American delegation which at Dakar included Louis Armstrong and James Baldwin is making do with the likes of the National Black Theater of New York, the Denver-Ensemble, the Flying Soul Trapeze of Los Angeles and the Milford Graves Music Ensemble.

Wandering around the village's three-story, low-cost housing is like what one musician called "the black olympics." Gospel singers give forth in one building, a lonely saxophone is heard at the next while dancers practice their rutines in the dusty parking lot further down the line.

The iffy water supply and electricity - part and parcel of even the most gracious Lagos living - the unfamiliar food, the pickpockets and the lost luggage all tend to be forgotten in the thrill of what some American performers call "coming home." And as the head logistics man for the National Black Theater, a New York bus driver named Areoye Fields, said: "At least I haven't seen any roaches." Echoing his good-natured approach another American displayed a handwritten sign on his room door announcing: "Welfare Office."

For saxophonist Hugo Glover, a graduate of both the Mannes and Juilliard schools of music, his first trip to Africa at 39 is "better than being in the Library of Congress" listening to its extensive music collection.

"Hearing all that black man's music - through the gut, the senses, the psychology - is total immersion in black musical expression. It will bliss you unless you keep your feet on the ground," he said.

Wandering around FESTAC village, listening to the sounds was "like a continual block party in Harlem between 125th and 140th Streets," he said. "Listening to all that music, the ear works like a scanner - telling me to stop thinking and start feeling, to intuit, to sense."

For Glover, the best music was being made at the village - "practicing and rehearsing is far more rewarding than the overly formal performances."

Like many other American participants, he was moved by the opening ceremony last Saturday in which he and thousands of other performers marched in National Stadium to the delight of more than 60,000 Nigerian spectators.

"It was so much on the money, that stadium filled with masses of black faces, the mixture of chants, drums and dancing, the sound of the crowd's response, what a dynamic exchange of emotion.

"I couldn't have been more dissipated, more dehydrated or about to faint," he recalled, "but at the same time I felt vibrant, like running around the track despite the heat and humidity. I had to detox out all my classical music training."

For the six teen-age boys and six girls of the Harlem Children's Theater, FESTAC Village has proven to be just the kind of enriching experience their teachers envisaged when they were granted two weeks off from classes to attend the festival.

Hayes Fields, a tall, 13-year-old from Manhattan said, "We want to bring a little enlightenment here because there is so much stereotyping about America, to explain what the United States really is."

He and his colleagues are proud to be the only all-children group in FESTAC. And in return the American children are learning about Africa. "Imagine," Hayes said, "the Upper Volta group which lives next door woke us up one morning singing and dancing. We went down and danced with them. They had all their instruments, not just drums."

Both director Reggie Life and his assistant Aduke Aremu are putting their past African experience to work - explaining the various performances and arranging visits to Lagos and its environs and hopefully also up-country, where they could experience rural Africa. The security-conscious FESTAC authorities have yet to approve the out-of-town jaunt.

But the theater management is careful to avoid the romantic total identification which many black Americans sought to their eventual bitter disillusionment when they came to Africa in the late '50s and early '60s.

"There's a line in the show we are going to put on which says "illusions, illusions create confusions," Aduke said."Coming to Africa sparks the same excitement for black Americans as landing in Israel does for a Jew - you're totally elated. But there are lots of problems in Africa. And if a Ghanaian feels foreign in Nigeria, there's every reason a black American would, too."

Savvier, but no less thrilled by Africa, American performers thanks to FESTAC village are savoring the variety of the continent during the month-long festival without having to face the rigors of living in Africa in less heady surroundings.

For Pat Phipps, a New York photographer, coming to FESTAC has been an exercise in "coming home." But she shrewdly noted that alone of all the delegations at the opening-day ceremony the Americans slouched around in a variety of gear in contrast to the disciplined Africans "all five in a row." It was perhaps her way of also saying "you can't go home again."