After nearly two weeks of sustained effort, Sweden's great submarine hunt has failed to turn up the intruder. Military sources -- chagrined that all the hoopla has come to naught -- now say the vessel or vessels probably escaped.

The affair, Swedes acknowledge, is becoming an embarrassment. One naval spokesman attributed an early difficulty in maintaining sonar contacts to the seamen's hearing having been damaged by overexposure to disco music.

"We are going to have to look deeply into what really happened -- physically and mentally," a senior officer said in a telephone interview today. But there is no longer any doubt, he said, quoting Supreme Commander Lennart Ljung, that at least one submarine of undetermined size spent perhaps a week deep inside Sweden's 12-mile zone of territorial waters -- and was not caught.

That such a thing was possible is a serious matter for Sweden's defenses, and by extension, those of other coastal countries. Despite a naval task force, mines, depth charges, advanced antisubmarine technology, divers and air cover, the intruder was able to outmaneuver its pursuers.

Nor was this the first incident. The last major submarine search was in August. In late September, the Defense Ministry issued a detailed statement concluding that in 1982, the "number of violations" has been considerably higher" than in previous years and that intruders "act a great deal more provocatively."

Sweden's Conservative Party, the country's second largest, said recently it has confirmed information that 20 times since 1980, at points all along the shoreline, foreign submarines have penetrated antisubmarine positions. One Western source said that Swedes now believe the intruders have even placed and withdrawn small landing parties as part of penetration exercises.

Western diplomats in Stockholm say that the Soviets, whose submarines are thought to be the main intruders from bases across the Baltic, appear to have made a tactical decision to harass neutral Sweden. This, they speculate, is partially in retaliation for last November's humiliating episode in which a Soviet submarine ran aground in a restricted military area and was expelled.

Moreover, the Swedes say ships of the "foreign power," a euphemism used in official documents for the Soviet Union or Warsaw Pact, are conducting reconnaissance in Swedish waters for possible operations in wartime.

With all that in mind, the Swedes recently adopted a tougher strategy for dealing with intruders. A new weapon called "Marlin" was developed that would magnetically fix itself to a submarine and detonate an explosive charge sufficient to force the craft to surface. Commanders were given greater latitude for using weapons, and naval exercises were specifically designed to stress antisubmarine defenses.

None of these measures was to take effect until next year. But when, in the midst of an exercise on Oct. 1, a periscope was sighted near the country's most sensitive naval installation -- about 25 miles from Stockholm -- a major search operation was launched.

Initial communiques, referring to the intruder as being trapped and predicting that it would soon have to surface, are now conceded by spokesmen as having been "too optimistic." The scale of the hunt, the presence of hundreds of reporters from all over the world and the prospect of catching a Soviet submarine red-handed on an espionage mission apparently generated greater expectations of Swedish success than in retrospect were warranted.

In fact, as early as last Thursday, Swedish officers put out the word to selected foreign diplomats that something crucial had happened in the hunt, but they were unauthorized to say what it was. That was the day spokesmen reported an escape attempt by the intruder and later suggested there may have been more than one submarine.

Sources now say the pattern of sonar contacts, the condition of blockades designed to prevent an escape and visual evidence increasingly seemed to indicate that the escape attempt had been successful. That evening, the task force commander gave a gloomy forecast to reporters. In particular, the Swedes backed away from predictions about when the submarine might be seized.

The only alternative to the escape theory that remains is the possibility that the submarine is still in the vicinity but no longer functioning. With each passing day, that seems less likely.

Hors Bay, the focus of the search, provided better cover for the submarine than the Swedes had thought and their own equipment, officers said, was proving inadequate. What had started out as a bold attempt to demonstrate Sweden's resolve in dealing with intruders became a case study of how hard it is to stop them.