Three of the world's greatest swimmers -- U.S. teenagers of 17, 16 and 15 -- were forced to reach emotional depths tonight in the Pan American Games that were deeper than any swimming pool.

In the crowded, beautiful outdoor Piscia Escambron, Jesse Vassallo, 17, a native of Puerto Rico who has seen himself called a traitor by the local press because he swims for his adopted U.S.A., almost broke his own world record in the showcase 400-meter individual medley, then heard the crowd burst into the Puerto Rican national anthem as he stood on the victory platform.

Next, Sippy Woodhead, 15, a tiny high school freshman, broke her own world record in the 200-meter freestyle by a tenth of second in 1:58.43.

In her moment of triumph, Woodhead did not speak of childish pleasures or teen-age motivations.

Instead, she talked of how upset she had been to hear the Star Spangled Banner jeered at opening ceremonies., and told how she had dedicated her race to a former coach who had just suffered a miscarriage. Her young face grim, she said, "I told myself that I could endure as much pain in my race as she went through."

Moments later, Tracy Caulkins, at 16 the most prominent swimmer in the world and America's current amateur athlete of the year, was beaten in the 100-meter breastroke by Tami Paumier, 15, a high school sophomore from Columbia, Md.

Instead of being delighted to win a silver medal in the hemisphere Olympics Caulkins who just had her brwaces removed two weeks ago, had to explain to microphones and cameras how she possible could have lost.

"I would liked to have won," said Caulkins, apparently unruffled, "but I'm happy for Tami,"

Certainly the mot strained American athlete has been Vassallo, a virtual hermit for a week, who has been driven into silence and brooding by constant harassment from his countrymen.

The Mission Viejo [Calif.] High School senior, whose brother, Victor, swims for the Puerto Rican national team, admitted that, "I've wondered all day how I would be welcomed home."

The strain on Vassallo whose family moved to the U.S. when he was 11, has existed for years. At least one relative requested on his death bed that both brothers swim for Puerto Rico

The weight of an island was removed from Vassallo's shoulders tonight. As he touched the wall in 4:21.63, barely missing his world mark of 4:20.9, the full chanted Vassallo, Va-ssa-llo.

The crowd erupted even louder when Vassallo accepted a small Puerto Rican flag from his brother and waved it on the victory stand as the Star Spangled Banner was played.

Finally, after the U.S. anthem had ended amid cheers, the crowd broke into an impromptu chorus of "La Boringuena."

The prodigal son, practically denounced for treason in San Juan's yellow journals, had been embraced by his people.

What can I say?" asked the handsome Vassallo with his wispy mouthstache as he twisted the small flag around and around in his fingers. "How would you feel if your people broke into your national anthem. I have waited years to come home and swim."

Vassallo, born in Ponce, tried to swim for Puerto Rico in the 1976 Olympics, but was met by a wall of red tape.

"I asked them and they didn't let me," said Vassallo. "Well, not exactly. It's more complicated than that. They said I had to live in Puerto Rico for 12 months.That means I couldn't have competed internationally for a year. They just demanded things I could not do.

"I doubt," said Vassallo, smiling weakly, "if my friendss in the United States know what it means for me to swim for my new country."

Woodhead sat down next to Vassallo rolling a program tighter and tighter in her hands as Vassallo twisted his flag. They looked at each other and patted each other on the knee, both knowing the problems the other had gone through.

"Jesse's really a neat person," said Woodhead. "As nervous as he's been I'm glad his people loved him."

Woodhead has been upset since Sunday's opening ceremonies.

"I came into the [Hiram Bithorn] stadium before the teams," she said. "It bothered me to hear them boo our national anthem. I thought I really don't know what's going on down here."

In fact, the whole teen-age U.S. swim team was bothered. Tonight they all carried Puerto Rican flags and, as Vassallo took the victory stand, they all rose together, waving the flags and chanting, "Puerto Rico, Puerto Rico."

"We wanted to show them that we like it here," said Woodhead. "This was so important to Jesse that we thought it would be nice if they'd like him and maybe like us, too. It was our way of reaching out."

The State Department has seldom fared as well in Latin America. CAPTION: Picture, Jesse Vassallo, born in Puerto Rico, living in California, salutes his victory in 400-meter individual medley. UPI