Callie Diggs, a 77-year-old woman from Queens, N.Y., who had waited all night to fly to Africa in search of her ancestry, sobbed and shivered yesterday morning outside Gate 20 at Dulles International Airport.

Around her filed 240 fellow passengers bound for Dakar. Senegal, all victims of 12 hours of delays caused in part by President Carter's daughter-in-law Annette and in part by a bungling airport employe who had smacked their plane with a truck.

The passengers, many flying to Africa for the first time, had been transformed from excited tourists at 9 p.m. Thursday to a cursing crowd at 9 a.m. yesterday.

They were herded on and off their plane, kept ignorant of the reasons for the delay and surrounded at one point by security police. In the terminal, they tried to sleep on window sills and in straight-back chairs. They argued over rumors through the night and nearly everyone - passengers and airline personnel - got angry.

"Dulles stinks," said one man as he stumbled wearily into the mobile lounge that would take him to the Trans-International Airways charter plane. "Dakar or bust," said another. Gallie Diggs wiped a tear from her eye with a trembling finger, said she was "nervous to death" and boarded the plane.

So ended a nightmare for Dulles that led frustrated passengers to call newspapers in the middle of the night and say. "We'r going to tear this airport apart if we don't get our money back." And that forced Dulles security officers to surround shouting passengers at the ticket counter and that kept airline employe up all night saying that were sorry.

Rumors spread that a door had fallen off the plane, and that the pilot refused to fly because of "extensive damage" to the plane's wing. Confusion reigned among the angry passengers, some of whom had waited months for the NAACP endorsed flight. "Frankly, I'm not surprised [at the uncertainty] because I don't know what is going around," said Gaye Mohammad, a Senegalese physics professor on his way back to the University of Dakar.

Twelve passengers chose not to take the round-trip flight [including seven nights' accommodations in Dakar] for which they had paid about $435. Some said they were afraid that the plane was not safe. Some were just too angry.

"They are out their money," said an employe of International Weekends, the Boston charter company that booked the flight. The employe said he did not want his name printed for fear of reprisal from passengers.

As Trans-International flight 85014 boarded yesterday morning, one of those who chose not to fly, Tina Short of Washington, stood near Gate 20 and made an impassioned speech to a television news camera: "I slept on a window sill last night. This is a chance of a lifetime to go to Africa. They didn't talk to us one single time like we were human."

The delay began on Thursday when the DC-8 airplane that was to fly to Senegal from Dulles was held up for 26 minutes in West Berlin, Germany, for the birthday celebration of President Carter's daughter-in-law Annette, wife of 26-year-old Jeff Carter.

Annette Carter, who turned 26 on Nov. 5 and celebrated her birthday on Thursday, was in Germany for 10 days as part of a "Friendship Force" exchange of residents of West Berlin and Northern Virginia, according to the White House. She held a press conference before the flight and the plane, which was flying what is called a "back-to-back charter" between continents, waited for her, according to Bill Novichuk, an agent for Trans-International Airlines.

The delay in Germany forced the plane to fly into foul weather that was moving eastwood across the Atlantic. Novichuk said. The plane had to make an unscheduled fuel stop in Bangor, Maine, for 30 minutes and landed at Dulles four hours late.

Passengers, meanwhile, who had arrived at Dulles around 7 p.m. Thursday did not known the reason for the delay. "They didn't tell us what was going on," said Dr. Irvin Bromall, who works for the federal Urban Mass Transportation Administration. "We were trying to get information. I wouldn't say we grew unruly, just inquisitive."

At 4 a.m., as passengers were finally boarding, an airport employe hit the tip of one of the plane's wings with a baggage truck.

TIA agent Novichuk, who had flown to Washington early yesterday morning to try to deal with the delay, told group members to grab their carry-on luggage because they were going back to the terminal. He said there would be a wait while the airline found a qualified maintenance man to check the wing for damage.

While calls went out in the early morning to wake up a mechanic, passengers with bleary eyes and growing anger complained that the plane not only would not fly, but that it smelled of urine and was dirty. "The odor was caused by dampness and rain," explained Novichuk.

The damage to the plane "didn't even break the skin," according to Novichuk. It was easily repaired, but passengers had to wait while Civil Aeronautics Board inspectors made sure the plane was fit to fly.

Passengers yelled "Liar" and "What makes you think we should believe you now?" when a Trans World Airline's spokesman who was handling ground operations for Trans-International Airlines said the plane was ready to board.

The charter company and the airline involved in yesterday's incident are not responsible, under CAB law, for compensating those forced to wait. "There is a certain risk required in charter flights," said an employe of Weekends International.