While few Americans disapprove of the Reagan administration's massive civil defense evacuation plans, most feel the government should give higher priority to providing food, water and shelter in case of nuclear attack, according to a poll done for the Federal Emergency Management Agency.

The poll, the first of three public opinion surveys conducted for FEMA by the Gallup Organization at a cost of $18,900, was based on telephone interviews conducted from May 24 to June 13 with 1,023 adults.

President Reagan wants to spend $4.2 billion over the next seven years to upgrade the nation's civil defense system. The centerpiece of the program is the administration's plans to move millions of residents from areas thought to be prime targets for a nuclear attack to rural "host communities," sometimes hundreds of miles away.

Critics have criticized the "crisis relocation" plans as unworkable, in part because the program assumes that there would be warning of a nuclear attack well before the 30 minutes it would take a Soviet missile to reach the United States.

The poll found 50 percent of adults approve of Reagan's plan to increase civil defense spending, and 34 percent disapprove. Sixty-seven percent of adults agree that the United States should have evacuation plans, while 19 percent disagree.

But when asked what they thought the government should provide as part of a civil defense program, 70 percent of those polled said safe food and water and 35 percent said in-place shelters. Other responses included information in advance (7 percent), medical help (6 percent) and a warning system (4 percent). Only 3 percent said a relocation plan.

If Reagan asked them to evacuate an area, 41 percent said they would be very likely to follow his directions, 23 percent said somewhat likely, 18 percent said very unlikely and 10 percent said they would be somewhat unlikely to go. Eight percent said they didn't know what they'd do.

Twenty-four percent thought it would take a considerable amount of time--five days or more--for the government to evacuate an area. Others thought it could be done in one day (19 percent), two days (15 percent), three days (14 percent) or four days (6 percent).