After a week of closed-door interviews and the perusal of thousands of pages of documents, the Senate subcommittee investigating Billy Carter's Libyan ties will start public hearings Tuesday with testimony from two of Billy's business associates, Henry (Randy) Coleman and Charter Oil consultant Jack McGregor.

The president's brother, who was questioned under oath for seven hours by Senate investigators last Monday, is expected to testify Thursday.

According to Senate sources, Billy Carter said Monday that the original payment of $20,000 from the Libyan government was a loan. He originally had characterized the money -- part of $220,000 he eventually obtained, as a reimbursement for expenses.

Sources also said that Billy Carter told them he had only $11,000 of the Libyan money left. The rest was used to pay creditors, banks and the Internal Revenue Service, one source said.

In his first news conference since his appointment, chief counsel Philip W. Tone said yesterday the public hearings would not be finished by Labor Day as the subcommittee had hoped. No decision has been made on whether to call President Carter, Tone said, but should he testify, it would be after Labor Day.

Asked if the investigation would examine allegations that the Libyans planned to bribe a number of public officials, Tone said, "I can't say now the extent to which the investigation will go beyond Billy Carter's activities, but I don't anticipate we are going to limit ourselves to Billy Carter's relationship to the Libyans."

When Congress returns from recess Monday, the subcommittee is expected to meet in the afternoon and discuss the scope of the investigation, including a suggestion from Sen. Bob Dole (R-Kan.) that the inquiry be expanded to include other members of the Carter family.

"It would appear that dealings with foreign governments were a way of life for the Carter family," Dole said in a statement issued from Kansas, where he is campaigning for reelection. "The president's son Jeff was involved in business transactions with the aid of the Philippine government; Ruth Carter Stapleton, the president's sister, toured the Arab world courtesy of a pro-Arab businessman . . . Instead of calling this Billygate it might be more appropriately retitled 'a family affair.'"

Sen. Strom Thurmond (S.C.), ranking Republican on the subcommittee, said he will raise Dole's request with the subcommittee Monday. Sen. Birch Bayh (D-Ind.) said the Senate should examine legislation to restrict members as foreign agents -- a suggestion Dole endorsed in his statement.

Tone said yesterday the central issue of the inquiry is "the conduct of Billy Carter and other persons with respect to Libya and the effect on the foreign policy of the United States."

Other unanswered questions, he said, are "the settlement of the Billy Carter case in the Justice Department and the reasons for the payment of $220,000."

Tone, a former federal judge from Chicago, said he took the job because "it is desirable that the matter be aired, that the facts be determined . . . that the public understands that a thorough and fair investigation has been conducted."

He added, however, "It is too early to tell how important [the investigation] is but it is a subject of considerable interest to the public at an important time in the context of the election year."

Tone stressed that the subcommittee inquiry is a "legislative" investigation, as opposed to the Justice Department's investigation of possible misconduct of administration officials. The subcommittee does not have access to records of FBI interviews under that parallel inquiry, which is being conducted by the Justice Department's Office of Professional Responsibility.

Tone said the subcommittee's report, due Oct. 4, will examine "whether the Foreign Agents Registration Act is effective and properly administered and whether changes are necessary."

Louis Nasife, president of the Charter Oil Co., also will testify next week on Billy Carter's unsuccessful efforts to obtain Libyan crude oil for the company. Justice Department witnesses are expected to appear the following week.

So far, the subcommittee staff of 15 investigators has interviewed about 25 people. Atlanta attorney Charles Kirbo, an intimate of President Carter and a trustee of the Carter family business, was to be questioned yesterday on Billy Carter's finances.