The lights were dimmed and the drapes were drawn for an unusual 70-minute matinee yesterday in Alexandria. But the eight women and four men watching "Girls of the A-Team" were not there by choice.

The 12 jurors who sat impassively through the sexually explicit videotape will be asked to decide whether that film and three others are obscene and therefore illegal.

The screening came on the fifth day of the federal trial of three Fairfax County residents charged with racketeering for allegedly distributing obscene videotapes and magazines at outlets in suburban Virginia and Maryland.

The case against Dennis E. and Barbara A. Pryba of Lorton and Jennifer G. Williams of Woodbridge, as well as a Pryba-owned firm called Educational Books Inc. of Upper Marlboro, is being watched around the country as a test case for expanded efforts by the Justice Department to prosecute distributors of pornography.

The first-of-its-kind racketeering prosecution has generated widespread concern even among mainstream publishers and booksellers because a racketeering conviction permits the goverment to seize all of a defendants' assets. Such a confiscation could include materials that are not obscene and therefore protected by the First Amendement, raising serious constitutional issues, according to some critics.

In pretrial hearings, U.S. District Judge T.S. Ellis III turned aside defense counsels' arguments that the federal racketeering charges against the Prybas and Williams were unconstitutional, allowing their trial to begin Oct. 27.

In addition to three racketeering counts, the defendants have been charged with seven counts of interstate distribution of obscene materials. The Prybas also are charged with two counts of tax evasion for allegedly failing to report $300 a week in income from coin-operated "peep shows" at their adult book stores during a two-year period.

The defendants have denied the accusations.

According to testimony presented by Assistant U.S. Attorneys Lawrence Leiser and Cynthia Christfield, Dennis Pryba purchased tapes and magazines from a New York distributor of sexually explicit materials, Model Magazine Inc., and transported the merchandise by van to a warehouse in Upper Marlboro. From there, they said, it was distributed to the Prybas' adult bookstores and Video Rental Center outlets.

Williams, who is Barbara Pryba's sister, was a bookkeeper and payroll clerk at Educational Books, but also was listed as the firm's president in official documents, according to evidence.

Former Pryba employe Bun-Siv Ngor testified he collected about $500 every two days from coin-operated peep shows in the adult book stores. Ngor testified that he delivered $300 in cash from the machines each week to Barbara Pryba, sometimes at her Lorton home.

According to Internal Revenue Service auditor Susan McCarthy, the $300 weekly amounts did not appear on the Prybas' joint tax return in 1984, when they reported income of $148,000, or in 1985, when they reported income of $37,075.

Defense testimony is expected to begin tomorrow.