The decision to use the space shuttle as the launching platform for future missions in far earth orbit and in deep space is turning out to be much more costly and time-consuming than the space agency and the Pentagon had foreseen.

The latest unexpected delay is the trouble Boeing Co. is having in developing a solid-fuel rocket engine called the Interim Upper Stage (IUS) for use in the shuttle by both the Air Force and the National Aeronautics and Space Administration. Boeing is developing a two-stage IUS to boost large Air Force payloads from low earth orbit to high earth orbit and a three-stage version to send NASA's Galileo spacecraft to Jupiter and Solar Polar spacecraft around the sun.

The Air Force version of the engine has already risen in cost by $144 million to $430 million, so steep an overrun that the Air Force has renegotiated its contract with Boeing and put a ceiling on it. The NASA version of the engine is understood to have escalated $100 million, forcing the space agency to reconsider using the IUS as a shuttle-based launch platform.

"I suspect that if we have to switch from IUS to something else we will have to postpone Galileo and Solar Polar," NASA Administrator Robert A. Frosch said in a recent interview. "I do know we're staring at more than $100 million in cost overruns on IUS and that's too expensive to take."

Frosch said that Boeing was about to propose reducing the overrun, but that he did not know what the scaling-down would entail or cost.

"We're going to sit down and see if we can make some sense out of it," Frosch said. "If they can only build it for so much money that we can't make sense out of it, then we have to revisit something we explicitly rejected last year."

What Frosch meant was the possibility of using the liquid-hydrogen fueled Centaur engine in the shuttle to launch spacecraft to the planets. The Centaur has been for years the mainstay upper-stage engine for most of NASA's large spacecraft and all of its planetary missions, but was rejected a year ago as a shuttle platform.

Agency planners thought it would cost too much to adapt the Centaur engine to the shuttle and would divert too many engineers whose job it was to get the first shuttle ready for its first flight.

"Well, it's a year later and maybe we're ready to have these people look at other things," Frosch said. "What makes Centaur somewhat more attractive is that it looks as it it can be done a lot simpler and a lot cheaper than we once thought."

The main difficulty with the Centaur is its plumbing. Burning liquid hydrogen that must be kept supercold, the Centaur engine would require major changes in the shuttle's construction to accommodate the specially insulated fuel tanks.

Whichever course NASA takes on the shuttle launch platform, it promises to force new delays on the Galileo and Solar Polar missions. The launch of Galileo was originally planned for 1982, then put back to 1984 because the shuttle itself was two years behind schedule. Solar Polar had been planned for 1983 and has also slipped to a 1984 launch.

What concerns many space scientists is that continued delays in developing a shuttle launch platform will cause the White House Office of Management and Budget and Congress to refrain from backing any new planetary missions beyond Galileo. At stake are possible missions to Venus, Halley's Comet and Saturn, all of which would be launched from the shuttle in earth orbit.

The only one of these missions approved by OMB is the Venus mission, which would take a huge radar into orbit around Venus to map the planet through its permanent cloud cover. Congress has approved none of these missions; all three would require more than $1 billion.

Frosch worries that the incoming Reagan administration and the new Congress may be reluctant to approve new planetary missions until the shuttle launch platform problems are solved.

"There may be a legacy of continuing political problems with planetary exploration accentuated by the cascade effect of the shuttle delays because we opted to do all these things by shuttle," Frosch conceded. "It certainly makes things tough."