Officials responsible for President Reagan's Strategic Defense Initiative, popularly known as the "Star Wars" program, have begun to look for flaws in the various "countermeasures" that critics have said the Soviet Union could use to defeat United States anti-missile defenses.

Speaking before the annual meeting of the American Association for the Advancement of Science here yesterday, Gerold Yonas, SDI's chief scientist, said this effort had found reasons the Soviets might choose not to pursue at least one widely touted countermeasure: the fast-burn booster.

Star Wars critics have suggested that the Soviets could avoid having their missiles destroyed during launch by developing rockets that accelerated more quickly, escaping the atmosphere and releasing their warheads before there was time for U.S. beam weapons to attack.

Critics and supporters of SDI agree that if rockets cannot be destroyed before they release as many as 10 independently targeted warheads, it would be much harder to blunt a Soviet attack.

Yonas said preliminary analysis has shown that if the Soviets went to fast-burn boosters, they would sacrifice accuracy in aiming the warheads. This drawback, he suggested, might mean that the Soviets would be unlikely to employ such a countermeasure.

Much of the criticism of SDI has been based on assertions that it would be easy and cheap for the Soviets to penetrate a difficult-to-develop and costly defensive shield.

Yonas said the effort will examine Soviet political and social circumstances to decide "not what they can do but what they will do."

Star Wars' most prominent critic, Richard Garwin of IBM, brushed aside any optimism that such analyses might leave a potential missile-defense system looking invulnerable.

Speaking at the same session, Garwin granted that although it might be possible to develop space-based anti-missile systems in 20 to 30 years, there are so many different and inexpensive countermeasures that there is no way the system could overcome them all.

Garwin said those who support SDI on the basis of technological optimism should realize that similar optimism must underlie an analysis of the Soviets' ability to develop countermeasures.

Also on the panel was Donald Hafner, a political scientist at Boston College who specializes in the legal aspects of anti-satellite arms.

Although ASATs, as these devices are called, are not formally a part of the SDI effort, Hafner warned that their development and deployment, which is not prohibited by treaty, could be used as a cover for development of outlawed anti-missile systems. The Antiballistic Missile treaty forbids development of anti-missile systems but not research on them. It is a foregone conclusion that, if SDI moves from pure research to weapons development, the ABM treaty would have to be abrogated.

The United States is developing an ASAT of the "smart rock" type. It is a small device carried on an air-launched missile that can home in on a satellite and destroy it by simply crashing into it.

SDI officials have suggested that a similar device could be used against incoming warheads.