CIA Director William J. Casey defended himself in an upbeat speech to agency employes yesterday, but acknowledged that the Senate investigation of his activities is going to take longer than he had hoped.

The Senate Intelligence Committee made clear earlier in the day that there would be no quick wrap-up of its investigation of Casey by announcing the appointment of a special counsel to carry on the inquiry it began July 17.

Hundreds of CIA employes reportedly responded with a standing ovation for their director at the windup of a combination status report and pep talk that Casey gave them in the agency autitorium.

"He feels he's been doing a good job at CIA," said agency spokesman Kathy Pherson. "He was upbeat. He sounded good."

Casey, however, had hoped to have the issues that have made him the first of President Reagan's Cabinet-level appointees to come under congressional investigation cleared up this week.

The assignment of Nashville lawyer Fred Thompson, 38, who once served as Republican counsel for the Senate's investigation of the Watergate scandal, indicated that the committee intends to dig deeper.

Senate Majority Leader Howard H. Baker Jr. (R-Tenn.), however, expressed doubt that the investigation would last much longer, despite the appointment of Thompson.

"What hiring Fred Thompson signals to me is that the committee wants to have someone do a capable job without haveing to learn on the job -- to do it as soon as circumstances permit and not make a lifetime career of it," Baker said.

Thompson joined Sens. Barry Goldwater (R-Ariz.), the committee chairman, and Daniel P. Moynihan (D-N.Y.), the vice chairman, yesterday afternoon for a special briefing from committee staff director Jack Blake and his aides ontheir preliminary findings.

The full committee of eight Republicans and seven Democrats is scheduled to meet this morning to review the same material.

The inquiry this far has dealt primarily with Casey's business practices before he joined the agency and has included a review of FBI files compiled during the bureau's background investigations of Casey for the CIA post and for an earlier appointment in 1973 as head of the Export-Import Bank.

The investigation was triggered by the resignation of Casey's handpicked chief of covert operations, Max Hugel, on July 14 following disclosure in The Washington Post of alleged improprieties concerning the stock of a firm Hugel once headed.

Goldwater said last week that he thought the poor judgment Casey showed in insisting on the inexperienced Hugel as CIA's deputy director for operations was in itself sufficient cause for his resignation. Two other Republicans voiced their agreement the next day, but most other committee members have since adopted a wait-and-see attitude, suggesting that the Hugel episode is not enough for them.

One, Sen. John H. Chafee (R-R.I.) said in a statement yesterday that he intends to await the outcome of the committee's investigation, which is expected to include testimony from Casey.

"I do not believe in hanging a man -- and then giving him a fair trial," Chagee, a committee member, said. ". . . I anticipate it will be a while before we can come to any conclusions."

The inquiry thus far has covered litigation concerning Casey's role in a bankrupt agribusiness firm, Multiponics Inc., and has started to touch on his prviously undisclosed legal work for a New Jersey waste disposal firm with alleged ties to organized crime, SCA Services Inc.

Casey represented the firm several years ago when a number of its top officials were accused by the Securities and Exchange Commission of diverting about $4 million in corporate assets to personal use.

In his talk at the CIA, Casey said he was surprised so many allegations could corp up in a week.

"He said it might take a few more weeks, but he asked the employes to bear with him and said he was confident he would still be with them as head of the CIA," Pherson reported.