Sen. John L. McClellan, a conservative Southerner whose investigations of crime and abuses in organized labor made him a national figure, died in his sleep yesterday at his apartment in Little Rock, Ark. He was 81.

Aides said that Sen. McClellan, the second-ranking member of the Senate in terms of seniority and the chairman of its powerful Appropriations Committee was found dead in bed about 6:30 a.m. by his wife, Norma. The senator had a pacemaker implanted last summer to regulate his heartbeat.

Death occured a week after the Arkansas Democrat announced that he would not seek re-election when his term expired on Jan. 3, 1979. Had he completed his term, he would have served 36 years in the Senate. Including four years in the House of Representatives, his career in local and national politics spanned a full 50 years.

It was a career in which Sen. McClellan knew obscurity and power, poverty and affluence. In manner he often appeared aloof and stern. His voice could be gruff and prosecutorial. Yet he was open-minded as well as skeptical, and a suggestion that he had used his position to advance his own interests could move him a spontancous outburst of tears.

In private life he knew much tragedy. His three sons died in early manhood. His first marriage ended in divorce. His second wife died of a sudden illness. His mother died when he was only three weeks old. Throughout his wife he sought solace in work.

His work first came to national attention in 1954 during the televised Army-McCarthy hearings. These were conducted by the late Sen. Joseph R. McCarthy (R-Wis.) as chairman of the Permanent Subcommittee on Investigations. In the end, McCarthy's vilification of witnesses accused of supporting communism led to his censure by the Senate and his political downfall.

Sen. McClellan was the ranking Democrat on the committee. He was among the first to walk out on McCarthy. When the Democrats gained control of Congress in 1955, he became the subcommittee chairman. One of his first acts was to devise a set of rules for the conduct of congressional inquiries that both served the purposes of Congress and gave witnessess a measure of protection from harassment.

In the next 18 years, he used the committee to investigate organized crime and abuses within the labor movement. He hired the late Sen. Robert F. Kennedy, then a young lawyer, as a committee staff member. President John F. Kennedy, then a senator, was a subcommittee member.

The fruits of these hearings included the ultimate imprisonment of David Beck and James R. Hoffa, successive presidents of the Teamsters Union. Even more spectacular were the hearings Sen. McClellan conducted on organized crime. The star witness was Joseph Valachi, who described himself as a member of the Mafia. His account of that organization's machinations kept a national audience riveted to its television sets.

Other investigations over the years dealt with securities thefts, the operation of military service clubs overseas, the TFX aircraft contract and the cost of the plane, the riots that erupted in the cities in the late 1960s and unrest among blacks and students.

In 1973, Sen. McClellan relinquished chairmanship of the investigating subcommittee when he became chairman of the Senate Appropriations Committee. In that role he pursued his interest in such matters as a strong national defense and fiscal conservatism in most things.

Last spring he and Sen. Edward M. Kennedy (D-Mass.) introduced a bill entitled the Criminal Code Reform Act of 1977. Sens. McClellan and Kennedy worked together to eliminate provisions in earlier proposals that had drawn sustained criticism from liberal groups. If enacted, the bill will be the first comprehensive revision of the federal criminal code in the nation's history.

"I've always been called a conservative and I've never resented it." Sen. McClellan once said. He added. "I did not became a senator to transfer the United States into a socialistic, paternalistic state."

Sen. McClellan's passing yesterday drew tributes from President Carter and many of the Senator's colleagues.

The President said in a statement that Sen. McClellan was "a resolute and gifted lawnmaker. During his 39 years of service on Capitol Hill . . . he persistently spoke out for a strong national defense and upheld integrity in the operations of government. The economic development of the Arkansas River is an achievement for which he will be long remembered, and in which he took a deep personal pride.

"I am greatful for his wise and generous counsel during the early months of my administration. In the distinguished history of the Senate, only eight members have ever served longer than John McClellan. His passing is a loss to the Congress and to the nation."

Sen. James O. Eastland (D-Miss.) said that "Arkansas has lost her most distinguished son. I have lost one of my dearest friends."

Eastland was the only senator senior to Sen. McClellan. Both took their seats on the same day, Jan. 3, 1943. But by a rule of the Senate, Eastland has seniority because Mississippi became a member of the Union before Arkansas.

Senate Democratic leader Robert Byrd (W.Va.) called Sen. McClellan "a man of vision who authored and guided legislation to make our country a better place to live."

Vice President Mondale said: "As one who was privileged to know and work with Sen. McClellan in the U.S. Senate. I considered him a friend as well as a highly respected colleague whose courage and integrity were beyond question."

Former Sen. Roman Hruska, a Nebraska Republican, called him "a man of compossion."

Sen. Kennedy stated that Sen. McClellan stood as an example "of the finest American tradition of service to the public.

Sen. John C. Stennis (D-Miss.) praised him for "his high degree of honor and (the fact) that everyone knew that under all circumstances his word was his bond." Sen. Edmund Muskie (D-Maine) said Sen. McClellan's death was "a loss to the country," said Sen. Jennings Randolph (D-W. Va.) called him "a diligent and effective legislator."

Sen. Hubert H. Humphrey (D-Minn.) said: "Sen. McClellan was a hardworking, dedicated public servant who did what he believed was right for our country.

Attorney General Griffin Bell said that if the revision of the federal criminal code become law, "it will be a monument to the efforts of Sen. McClellan to modernize federal criminal laws."

Gov. David Prytor of Arkansas, whom Sen. McClellan defeated in a primary runoff in 1972, when he had one of his most difficult battles for re-election to the Senate, said he would not seek to have himself appointed to the vacant Senate seat. He issued a statement that said: "The people of Arkansas have lost a friend and defender in Sen. McClellan. He was a strong advocate and a tireless worker for those principles he espoused. It is unlikely that we shall see a man of his stature again."

John Little McClellan was born on Feb. 25, 1896, on a farm near Sheridan, Grant County, Arkansas. His father, Isaac Scott McClellan, a sharecropper and school teacher, named him after a local congressman.

"I felt sorry for that little motherless boy of mine," the senior McClellan told an interviewer years later. "When I was teaching I took him to school with me. He would listen to the older classes recite - they were all in one room - and when he was 4 he could read, write and do some arithmetic. When he was 6 he was in the fourth grade. And then I taught him at home at night. I pushed him."

The McClellan, father and son, both became self-taught lawyers. Young John was admitted to the bar at the age of 17, and became the youngest lawyer in the history of his state.

By Sen. McClellan's own account, he had become interested in politics long before he became a member of the bar. He used to recall watching the people on the streets of the little towns where he grew up and wondering what they were thinking. It is said that he gave his first political speech at the age of 8, an oration written by his father to embarrass a Republican teacher who had disciplined his son in class.

There is another story dating from that time bears on Sen. McClellan's character. He was playing baseball for the Sheridan team and the Saline County club was at bat. Sheridan was leading 4-3 with two out in the ninth inning. Saline had a man on second base and he attempted to steal third. The umpire ruled that McClellan had tagged the player out. McClellan denied it, and Saline went on to win.

Throughout his political career Sen. McClellan was known as a man of his word. He was a scrupulous in remembering promises made to him and broken he was in keeping promises of his own, according to numerous associates.

In World War I he gave up his law practice and joined the Army. He was a lieutenant when discharged in 1919.

By that time he had married the former Eula Hicks. The marriage ended in divorce. The couple had one child, Max Eldon McClellan. In 1943, Max McClellan died of an illness while serving in the Army in North Africa.

In 1922 Sen. McClellan married the former Lucille Smith. The couple had three children, John L. Jr., James and Mary Alice. The second Mrs. McClellan died of spinal meningitis in 1936.

The following year the senator married the former Norma Myers Cheatham, a widow.

In 1949, while driving to funeral services for Max McClellan, whose body had been returned to this country from North Africa, John L. Jr. was fatally injured in an automobile crash. Son James died in an airplane crash in 1958.

Sen. McClellan began his political career as a prosecutor, serving his hometown of Sheridan and the surrounding judicial district of Arkansas from 1927 to 1930. He served two terms in the House of Representatives from 1935 to 1939.

It was during this period that he made his first try for the Senate. In 1938, he challenged Hattie Caraway, who had taken her late husband's seat in the Senate, and he lost. He tried for the state's other Senate seat in 1942, won, and held the seat there-after until his death.

One of his interests in his long Senate career was waterways. The Arkansas River Navigation Project was one for which he worked in particular and his office once issued a statement terming it "a monumental example of his leadership in opening the gates to unlimited opportunity and progress in Arkansas and the Southwest."

On national affairs, Sen. McClellan helped pass numerous laws reorganizing the federal government as chairman of the Senate Operations Committee of which he was chairman for 22 years.

He also was chairman of the Senate Judiciary Committee subcommittee on criminal laws and procedures. As such, he was the chief sponsor of the Omnibus Crime Control Act of 1960.

As a crime fighter, he defended the use of wiretaps by government investigators and advocated the death penalty for certain crimes.

Other legislation in which Sen. McClellan played a major role was the passage last year of the complete revision of U.S. copyright laws since 1909. He also succeeded in persuading the Senate to pass the first major restructuring of U.S. patent law since 1836. The House failed to act on the measure.