Declaring that he was never cut out "to be a bookkeep er," Sen. Herman Talmadge (D-Ga.) maintained under oath yesterday that the charges of financial misconduct lodged against him are groundless.

In a combative appearance before the Senate Select Committee on Ethics, Talmadge said he never knowingly misappropriated Senate expense funds or campaign contributions. He insisted, instead, that the allegations were the result of mistakes by his aides and his accountant and lies by his chief accuser, Daniel Minchew.

"I'm not a book-keeper. I'm not an auditor. I'm not competent to keep those records if I tried," Talmadge protested. "I didn't figure the people of Georgia sent me to the Senate to be a book-keeper."

While he repeatedly assailed Minchew as a liar, Talmadge, somewhat more circumspectly, dismissed allegations by his wife, Betty, as untrue. She had testified that Talmadge used to keep cash hoards of as much as $4,500 hidden in an overcoat pocket in their Washington apartment.

"At no time did I maintain a cash hoard in the pocket of an overcoat or anywhere else," Talmadge testified.

The Georgia senator also ran through the charges leveled against him by Minchew, his former chief aide until 1974.

Minchew told the committee earlier that he set up a secret account in 1973 at Riggs National Bank here with Talmadge's approval. Both he and Talmadge got portions of the $39,000 in improper Senate expense payments and campaign contributions that were channeled through the account, Minchew testified.

"Any testimony by Daniel Minchew that I knew before June 1978 that my office was withdrawing excess funds from the Senate is a lie," said Talmadge, adding:

"When Minchew says I authorized or directed the opening of the secret Riggs account, he lies.

"When Minchew says that I ordered the diversion of Senate and campaign funds into the account, he lies.

"When Minchew says that he gaveme funds from that account or that I benefited from it in any way, he lies again."

Talmadge also said he did not know of any payments from the secret account to his former wife or to his late son, Robert Talmadge. Minchew has said the senator made such payments.

It was the first appearance under oath by Talmadge during the 26 days of testimony the committee has heard on his case.

At the opening of the hearing yesterday, Talmadge sat with a rigid smile on his face for television and still cameramen who crowded into the Senate hearing room.

But later, with the cameras gone he scowled and occasionally thumped the table in front of him when committee special counsel Carl Eardley began questioning him about the false expense accounts filed to the Senate by his office.

He did not, Talmadge said, pay much attention to the office expense accounts which he said were handled by staff members.

"Senators are having to deal with matters such as war and peace, inflation and the energycrisis," said Talmade "I doubt if very many, if any, senators, Mr. Eardley, pay very much attention to vouchers for Senate reimbursements.

After an audit ordered by Talmadge last year when the Senate overpayments were revealed in newspaper articles, Talmadge repaid the Senate $37,125 for the overpayments to his office account from 1971 to 1977.

Talmadge's performance yesterday was in sharp contrast to his demeanor throughout the hearings. He had spent most of the sessions sitting quietly at a table with his attorneys gazing through a blaze of smoke from a succession of long black cigars.

During yesterday's session, however, Talmadge challenged Eardley, interrupting his questions, glowering at him and, at one point, telling the special counsel to use the microphone to ask a question.

Talmadge did not make any new revelations about his complex and controversial financial affairs. Each time he was asked by Eardley to explain the false expense accounts and diverted campaign funds, he said he was not aware that they had taken place until recently.

Although the committee has heard testimony from some Talmadge staff members that the senator was meticulous in financial detail, he said yesterday his oversight of his office accounts was so lax that he did not know the office had for years billed the Senate for nonexistent expenses.

Talmadge has also maintained he knew nothing about the overpayments from the Senate until last year, but the issue was raised during a divorce deposition he gave in 1977 to W. Stell Huie, he ex-wife's attorney.

Talmadge said he dismissed Huie's warning to check possible Senate overbillings by his office. "He was a hostile lawyer," Talmadge said. "His job was to destroy me if he could." CAPTION: Picture 1, Sen. Herman Talmadge confers with his attorney, James Hamilton, before beginning testimony to committee. By James