The Challenger disaster is forcing private companies that were counting on the space shuttle to launch communications satellites this year to look for other ways to get their payloads into orbit.

The National Aeronautics and Space Admininistration's shuttles had scheduled 13 more trips into space this year and were to launch at least four commercial communications satellites for clients including GTE Corp., Western Union Corp. and the communications networks of Britain and Indonesia.

NASA officials yesterday were unable to estimate when the shuttle will fly again, leaving their commercial customers a choice between waiting or finding another route into space.

Commerce and State Department officials suggested that other countries might have to launch U.S. satellites. "The only two realistic possibilities are France and the Soviet Union," said one State Department official.

But U.S. officials are reluctant to put even nonmilitary satellite technology in the hands of the Soviets. The French have been plagued by accidents and already have a two-year waiting list for trips into space.

The launch of commercial U.S. satellites by the Soviet Union would meet with "heavy resistance" from the Pentagon and export officials, the State Department official said.

The only other commercial alternative to the shuttle is Arianespace, the French-backed European consortium. That organization, which lost two satellites -- one belonging to GTE Corp. -- in an accident last September, cannot squeeze additional satellites into its tight schedule of eight 1986 launches.

"We are basically fully booked this year and next year," said Jacqueline Schenkel, a spokeswoman for the consortium.

Transpace Carriers Inc. of Lanham has bought the rights to use outmoded NASA Delta and Atlas Centaur rockets to launch satellites privately, but so far has signed up no customers.

Some communications company officials privately chided the Reagan administration yesterday for spinning off the older rockets -- the only other American commercial alternative to the shuttle -- to the private sector.

"The user community will have to turn to the expendable-launch-vehicle industry," said Rick Endres, director of corporate development for Transpace. "This puts real pressure on many users to get access to space and makes us a reality faster."

He said Transpace has "financing fairly well on track" to make commercial rocket launches a reality. Nevertheless, the earliest the company could complete a commercial launch would be 1987, he said. General Dynamics Corp., which has taken over the rights to the Atlas Centaur, had no comment yesterday.

A Commerce Department official did not give the shuttle's rivals rave reviews. "If you turn off the shuttle program, you turn the debate back in the favor of expendable launch vehicles. . . Ariane has its own problems. Periodically they blow up satellites; their track record isn't super."

Officials at GTE Corp.'s Spacenet subsidiary in McLean as well as at Satellite Business Systems, the MCI Communications Corp. business communications company, said yesterday they would come up with something.

Spacenet, which plans satellite launches using Arianespace over the next few months, was due to have another launched by the shuttle in November. "We want to use the space shuttle for this launch but we have customers and they have needs," said C. J. Waylan, president of GTE Spacenet. "The alternative would be an Ariane or Delta launch."

But Waylan was not altogether optimistic about unmanned expendable launchers. "Transpace Carriers Inc. doesn't have the depth of commitment from an organization standing behind them . . . it takes a large amount of commitment and dollars.

"I personally continue to believe we have made an error not to have an expendable-launch-vehicle industry parallel to the shuttle -- we have been driven to one program because of financial considerations," he said.

GTE Spacenet has "less pressure" to figure out a solution, he said, because of the planned Ariane launches that will tide the company over. The November shuttle launch was to provide service to Federal Express Co. among other clients.

Satellite Business Systems, which has a 1987 launch scheduled on the shuttle, said it could use terrestrial fiber optic cables to transmit voice, video and data signals.