The Supreme Court drew a firm line yesterday on how far government can go in accommodating religious freedom without violating the Constitution's ban on government establishment of religion.

The court, in an 8-to-1 decision written by Chief Justice Warren E. Burger, struck down a Connecticut law that said companies could not force employes to work on their sabbath. The court ruled that the law was too absolute and granted too much preference to religious concerns at the possible expense of nonreligious employes and the employer.

The court overwhelmingly rejected, for the second time in less than a month, the Reagan administration's arguments that the justices should alter the court's approach to testing the validity of statutes which involve religion and give greater leeway for religious needs.

Numerous states and the federal government have laws requiring employers to try to accommodate religious needs whenever practical. Connecticut's law was unusual because it was more absolute.

"Government must take pains not to compel people to act in the name of any religion," Burger wrote.

The Connecticut law was unconstitutional, he said, because it "decreed that those who observe a sabbath any day of the week as a matter of religious conviction must be relieved of the duty to work that day, no matter what burden or inconvenience this imposes on the employer or fellow workers."

Burger said the law thus "provides sabbath observers with an absolute and unqualified right not to work on their sabbath," but also violated the constitutional ban on establishment of religion by saying "religious concerns automatically control over all secular interests at the workplace."

The case involved Donald Thornton, a devout Presbyterian, who began working for Caldor Inc., the operator of a chain of retail stores in Connecticut, in 1975.

In 1977, Caldor began operating on Sundays. Thornton said he could not work on Sundays because that was his sabbath. The company offered to transfer him to another store in another state, but Thornton objected. Caldor then demoted him to a job not requiring Sunday work but paying almost 50 percent less. Thornton resigned and appealed the way in which Caldor treated him. Thornton died in 1982 but his estate continued the appeal.

The Connecticut Supreme Court ruled that the state law violated the U.S. high court's traditional test that a statute must have a secular purpose, not promote or restrict religion and not excessively entangle government and religion.

Groups asking that the law be upheld, in addition to Connecticut and the Reagan administration, included various religious bodies and state and county government groups. They argued it was a law against private discrimination and simply accommodated the needs of those with religious beliefs.

Opponents, including the AFL-CIO and employer groups, said it forced employers to adhere to employes' religious needs regardless of cost to the companies or to nonreligious employes.

American Civil Liberties Union attorney Charles Sims said the ruling "struck the right balance in the case" and appeared to be a "sensitive weighing" of those concerned with accommodating religious needs and those concerned about state involvement in religion.

The American Jewish Congress, which sided with the administration and the state in this case, also seemed satisfied with the ruling.

Lois Waldman, an official with the American Jewish Congress, said the ruling would not affect many state and federal laws that "protect religiously observant employes from discrimination on the job and require employers to accomodate within unreasonable limits the religious observances.

"Although we disagree with the specific result, we take comfort in the court's willingness to reject efforts to weaken the principle of church-state separation in the guise of religious accommodation."

Joseph Conn, a spokesman for Americans United for Separation of Church and State, a group which also lost, nevertheless praised the ruling. "This court seems to be looking for a balancing of interests between the religious community and those outside the religious community. This law tipped the balance too far in favor of religion."

Justice William H. Rehnquist, who delivered a lengthy dissent earlier this month when the court struck down an Alabama law requiring a moment of silence for meditation or prayer, dissented from yesterday's ruling but did not write an opinion.

In a concurring opinion, Justice Sandra Day O'Connor, joined by Justice Thurgood Marshall, said the "message conveyed" by the Connecticut law "is one of endorsement of a particular religious belief to the detriment of those who do not share it." As such, she said, the Connecticut statute "has the effect of advancing religion and cannot withstand constitutional scrutiny."

O'Connor added that she did not see any suggestion in the opinion that the provisions of the more flexible federal law on accommodating religious employes' beliefs were invalid.