U.S. Olympic Committee executive director Dr. Harvey Schiller abruptly resigned yesterday after just 16 days on the job, and was immediately rehired to his former position as commissioner of the Southeastern Conference. Almost as immediately, the USOC announced assistant executive director Baaron Pittenger would replace Schiller, an upheaval that comes just 3 1/2 weeks before the Winter Olympics.

Schiller, a 48-year-old former chemistry professor and retired Air Force colonel, resigned after a conference call yesterday morning among most of the 10 school chancellors and presidents who make up the powerful SEC. An undisclosed majority of the presidents voted to invite the former commissioner back to the job he vacated on Jan. 4, when he moved to the USOC offices in Colorado Springs.

Schiller said his decision was based on his strong ties to intercollegiate athletics and the SEC, where he served as a prominent and effective commissioner for the last 13 months prior to accepting the USOC position. He also said he felt he could be more effective as a USOC volunteer than as a bureaucratic administrator for the USOC.

"I'm disappointed I couldn't carry out the duties they elected me to," Schiller said from the SEC offices in Birmingham last night. "But it feels right to be here. You've got to be at peace with yourself."

It was the second time in six months a USOC executive director has stepped down, leaving the organization leaderless with the Winter Games opening in Calgary on Feb. 13. In August, Gen. George Miller resigned without explanation after serving for 2 1/2 years. But yesterday the USOC responded swiftly when it named Pittenger to succeed Schiller.

In Pittenger, the USOC has named a director who appears to offer stability, having served the organization in various capacities since 1977. He was named to his current post in 1981, and was interim executive director during the search process that resulted in Schiller's hiring.

Schiller, a recent finalist to become executive director of the NCAA, was widely heralded by the USOC when he was unanimously hired on Oct. 24. He was in the midst of adopting his new duties when he had his change of heart. On Monday he had received authorization from the USOC executive board to begin executing contracts on improvements to the Olympic training center in Lake Placid, N.Y. But on Tuesday he canceled a trip to Geneva, Switzerland, where he was to meet with USOC President Robert Helmick and various international Olympic officials, in order to make his decision.

In a statement from Switzerland released through the USOC, Helmick expressed surprise at Schiller's decision, and said he had yet to receive a formal resignation.

"The United States Olympic Committee has just received word that the SEC has reappointed Dr. Harvey Schiller as its commissioner, and we are very surprised," Helmick said. "This is totally unexpected, and personally, I am disappointed. The USOC has not yet received an official letter of resignation from Dr. Schiller . . . We conducted a long and serious negotiation with him, and thought we had arrived at a mutually satisfying commitment by both parties."

Although Schiller did not publicly indicate his doubts until Tuesday, many of the SEC presidents and chancellors learned of Schiller's interest in his old job at the NCAA convention in Nashville last week. There, Schiller spoke with a number of his former colleagues, who were in the process of interviewing candidates to replace him. Schiller also contacted school presidents over the last couple of days to discuss his status.

Schiller said one reason for his ambivalence about the prestigious, $150,000 a year USOC job was that he had trouble adjusting to his role as an employee, after serving as a creative Olympic volunteer and exercising considerable authority in the SEC. He also expressed regrets at leaving intercollegiate athletics, in which he has spent most of his career.

"It's a matter of level of participation and enjoyment," he said. "This way I can serve amateur athletics and intercollegiate athletics as well."

With the SEC, Schiller negotiated lucrative new basketball and football television packages, one reason for his hiring by the USOC, which he joined shortly after the New Year's Day college football bowl games. Also, Schiller displayed an insistence on compliance with NCAA regulations in a conference that has been plagued by violations in the past.

Schiller's departure may indicate much about the position of the executive director. It is said to include huge responsibilities in overseeing the largely volunteer organization, but sometimes little authority and frustrating bureaucracy. Pittenger also has a background in both the NCAA and college sports, but since 1977 has been employed exclusively by the USOC and has a solid working knowledge of the laborious administrative and fund-raising duties required.