The Supreme Court agreed yesterday to decide whether state and local governments may withdraw their employes from the Social Security system.

The case stems from an attempt by California, acting under a 1950 federal law, to pull some 34,000 of its local government employes out of the system. While California's effort was pending, Congress amended the federal law in 1983 to bar such withdrawals.

Last May, a federal judge in California invalidated the ban on withdrawals, saying it amounted to a "taking" of California employes' property and was unconstitutional under the Fifth Amendment.

The Reagan administration appealed, arguing that the 1983 amendment was necessary because accelerating withdrawals in the early 1980s were threatening the solvency of the system and costing it up to $1 billion a year.

When the amendment passed, about 9.4 million of an estimated 13.2 million state and local government employes were enrolled. Only five states -- Alaska, Maine, Massachusetts, Nevada and Ohio -- do not have their employes enrolled.

A decision in Heckler v. Public Agencies Opposed to Social Security Entrapment is expected this summer.

The court also said yesterday it would decide whether a University of Nebraska medical school professor can be sued for libel in Colorado by a former student who says his reputation was damaged by an unfavorable letter of reference the professor gave to a Colorado hospital.

The case, Connolly v. Burt, could resolve important questions about the scope of recent court decisions that gave plaintiffs greater leeway in selecting the most convenient state for them to sue for libel.

The professor, who lives in Nebraska, argues that it violates his right to due process to force him into court in Colorado, where he has never worked or lived.

In another case, the court agreed to decide whether Mississippi's school financing system discriminates against students in the northern part of the state and whether federal courts have the power to intervene in school financing.

The state financing system, based on income from land trusts set aside in each township, has created major disparities between funds available in 23 northern counties and 59 southern counties.

Lawyers for the northern counties say the disparity, though offset by funding from other sources, denies the northern counties' 180,000 pupils a "minimally adequate" education.

A federal appeals court, ruling in Papasan v. Mississippi, dismissed the complaint last June, saying the courts could not overturn the financial system.

The court also ended yesterday the 18-year-old legal odyssey of a couple who claimed their constitutional rights had been violated by congressional investigators.

The couple, Alan and Margaret McSurely, were antipoverty workers in rural Kentucky in the late 1960s when their home was raided by local police.

Their books and papers -- including a diary that disclosed Margaret's relationship with nationally syndicated columnist Drew Pearson -- were seized and shipped to Washington as part of a Senate investigation into "civil disorder," conducted by the late Sen. John McClellan (D-Ark.).

A jury awarded the McSurelys, who were later divorced, $1.6 million in damages in a suit against McClellan and two of his aides, but an appeals panel here reduced the award to $21,000 for invasion of privacy by one of the investigators, John Brick. The court turned down both their appeal and an appeal by Brick's estate.

The court also declined to hear an appeal by Marine Pvt. Robert R. Garwood, who was court-martialed in 1981 for collaborating with the enemy while a prisoner of war in North Vietnam.

In other action, the court:

*Agreed to study the legitimacy of a gambling casino accounting technique the federal government says could cost millions of dollars in tax revenues. The case is U.S. v. Hughes Properties, Inc.

*Agreed to decide whether income earned by the American Bar Association's group insurance plan is taxable and whether a bar member's assignment of his policy dividends to the ABA are tax-deductible charitable contributions.

The Reagan administration argues that the questions raised in U.S. v. American Bar Endowment potentially involve hundreds of millions of dollars in revenue, including $6 million over a three-year period in taxes by the endowment.

*Acting in a case involving the anti-nausea drug Bendectin, agreed to decide whether federal judges had the power under federal law to hear claims for damages brought by non-U.S. residents. The case, Merrell Dow Pharmaceuticals v. Thompson, involves lawsuits by two women from Scotland and Canada who claimed the drug caused birth defects.

*Said it would decide if a state university employe can file a civil rights suit against his employer in federal court after the same charges were dismissed during a state administrative proceeding. The case is University of Tennessee v. Elliott.