The Olympic flame began its 3,040-mile journey today to the controversial Moscow Summer Games from the ancient site here where the ideal of the sports competition was born 3,000 years ago.

The ceremonies coincided, however, with the reports that three more countries previously reported as having accepted invitations to the Moscow Olympics have decided not to attend.

The decisions by Niger, Gabon and Panama raised to 42 the number of nations not going to Moscow this year. The United States has tried to organize a boycott of the Games to protest the Soviet military invasion of Afghanistan.

The International Olympic Committee in Geneva said 83 nations would attend, 42 have refused, while another 20 countries have failed to respond to invitations.

But a number of sports federations for specific events in countries scheduled to attend the Summer Games have announced that they would not be going to Moscow. The latest announcement today came from Stockholm where the Swedish Equestrian Federation said its team would not attend because of the expected poor quality of equestrian competition.

In Melbourne, the Australian Olympic Federation, in a vote termed by Prime Minister Malcolm Fraser as representing "a sad day for Australia," reaffirmed its decision to go to Moscow. Some of Australia's best athletes have decided to boycott the Games, however.

In London, the British Olympic Committee, which earlier decided to send a team to Moscow, today voted to deny permission for the playing of the national anthem or the raising the British flag in the event of British victories at Moscow.

Committee chairman Dennis Follows said that the decision was in line with a stand adopted by representatives of the Olympic committees of 18 European nations at a meeting in Rome last month, presumably as a way to demonstate that their presence in Moscow does not constitute tacit support for the invasion of Afghanistan.

The Olympic flame ceremonies here were conducted at the altar of Hera, the most sacred spot in ancient Olympia where the first recorded games were held in 776 B.C.

The flame was lit with a special parabolic mirror by Greek actress Maria Mscholiou, who was dressed as a priestess of Hera and escorted by 12 maidens clad in pleated tunics.

The Soviet-made torch was carried in relay, with the first stop at the monument to Pierre de Coubertin, the man who promoted the modern Olympic Games. About 1,170 Greek youths would carry the torch through northern Greece to the Bulgarian border.

In Bulgaria, officials said that runners, cyclists and horsemen would alernate carrying the flame from 581 miles to Romania. Romanian organizers said 2,000 athletes would share the job on the 368-mile-long stretch to the Soviet border.

Muscovites are scheduled to meet the torch on July 18 in Red Square. It will be carried to the Olympic stadium the next day when the Games are to be formally opened.