The Senate investigation of fraudulent military recruiting got off to a rocky start yesterday as chairman Sam Nunn (D-Ga.) complained that the Army is investigating itself rather than letting an independent office take a look.

"I'm not satisfied," Nunn said after hearing Army witnesses assure his Armed Services manpower subcommittee that the Army Recruiting Command was conducting a full investigation of recruiter malpractice.

Rather than settle for an Army command investigating itself, Nunn suggested that the secretary of defense commission some independent office to take a look.

Brig. Gen. James F. Hamlet, Army deputy inspector general, told the subcommittee that he recommended the Army Recruiting Command be allowed to clean its house.

The inspector general, whose office could have conducted the inquiry said the recruiting command had the resources availiable, including computers, to do a thorough job; that any fraudulent recruiting was the responsibilty of that one command, rather than spread across the Army, and that he had full confidence in Brig. Gen. Donald W. Connelly.

Connelly, deputy commander of the Army Recruiting Command, is heading the Army's investigation of fraudulent enlistment practices.

Nunn said he had asked Sen. Robert Morgan (D-N.C.) to act as cochairman of the subcommittee probe into military recruiting. The next public hearing is not expected until after Jan. 1.

Morgan, in summing up the first day of hearings, said "somebody is not willing to face up to the fact that they have a problem" in fraudulent recruiting. "It doesn't seem to bother anybody in the Army's high command" that so many involved in recruiting have had to be relieved for malpractice.

Morgan said the high command in this case included Army Secretary Clifford Alexander Jr., who led off yesterday's hearing.

Alexander told the subcommittee "overall results have been very good" in signing up an all-volunteer Army. He said "the Army has been within 2 percent of its authorized end strength in every year since fiscal 1974."

The "principal reason" the Army fell 16,400 people short of its goal for volunteers in fiscal 1979 was the failure to put enough manpower and money into recruiting, Alexander said.

Connelly, in giving the subcommittee a rundown on findings to date from the Army's recruiter fraud investigation, said 324 recruiters and supervisors in 41 of the Army's 57 recruiting districts had been relieved. Five of those relieved from recruiting duty were officers.

"Possibly 12,700 soldiers were fraudulently or erroneously enlisted since October 1977," Connelly reported.

"I categorically reject the idea" that establishes quotas for volunteers is forcing recruiters into malpractices, Connelly said. He added that "the vast majority" of the field recruiting force, well over 90 percent, has been exonerated by the Army's investigation to date.

Nunn said the larger question is whether the recruiting frauds which have been documented in the Army, Navy, Marine Corps -- and to a limited extent the Air Force -- signal that the all-volunteer force experiment is failing. He noted that no military service reached its goal for recruits in fiscal 1979.

Nunn said the Pentagon must face up to the fact that conscription may have to be resumed. The senator said that he has stopped short of calling for the draft but feels young men and women should be required to register.

Sen. James Exon (D-Neb.) said during yesterday's hearing that after being a firm supporter of the all-volunteer force and an opponent of returning to the draft, "I'm having second thoughts about my position."