Arlington police, investigating allegations of corruption and financial irregularities, have seized many of the records of a Virginia agency which last year collected more than $5 million in cigarette taxes in Northern Virginia.

"This is a sensitive case," said a source familiar with the investigation who did not want to be named. The investigation, the source said, has focused on the operations of the Northern Virginia Cigarette Tax Board.

On Monday night detectives assigned to the Arlington white-collar crimes unit went into the tax board's offices in the county courthouse and seized boxes of records. The board, created in 1970 to help eight localities collect local taxes on cigarettes sold in the Virginia suburbs, operates out of a small, unmarked office and is run by a board composed of the chief administrative officers of its member governments.

Law enforcement authorities were vague yesterday about the specifics of what they are probing. But officials close to the investigation said that police are looking into possible financial mismanagement and charges that agency personnel obstructed a probe of cigarette smuggling in North Virginia.

Police said that none of the allegations has been substantiated and that all the records were seized without search warrants.

"This thing has been kicking around here for at least a month," said one source, who added that complaints from at leas two former tax board employes and several others familiar with the operation of the board are being investigated.

"We've heard charges on and off about the board for years, and we decided that now was the time for a full and impartial investigation," the source said.

The determination to proceed with the investigation was made by Arlington prosecutor Henry E. Hudson, police chief William K. (Smokey) Stover and detectives in Arlington's white collar crime unit.

Federal officials said yesterday that they have discussed the case with Arlington officials but have not begun a separate probe.

Tax Board administrator James L. Taylor Jr., who has directed operations of the agency since 1978, could not be reached for comment yesterday.

Former board administrator Paul H. Beeson rejected all the allegations as untrue. In a telephone interview, Beeson, who served as Arlington's tax collector from 1945 until his retirement in 1977 and was the Cigarette Tax Board's first administrator, said the charges "have absolutely no basis in fact."

Beeson said that the tax board had pursued its own investigations of the smuggling of untaxed cigarettes before its investigative unit was disbanded as the result of a budget cut in the late 1970s.

"We never found any large degree of smuggling in Northern Virginia," said Beeson.

When asked about allegations of involvement with organized crime figures, Beeson laughed heartily. "I'm a good guy," he said. "I wasn't about to go bad at the end."

Beeson said that the board was formed in 1970 to collect and disburse local cigarette tax revenue, which ranges from 5 to 10 cents a pack. Virginia's state tax is 2.5 cents a pack and is collected by a separate state agency.

Board employes are supposed to inspect cigarettes sold in Northern Virginia stores and businesses and determine whether they have the proper tax stamps. If not, those cigarettes are supposed to be confiscated.

While daily operations of the board are left to the administrator, an eight-member board of directors composed of the chief executive officers of eight jurisdictions meets four times a year to review the agency's accounts and set the agency's overall policy. It currently has only five employes and an annual payroll of almost $80,000.

Sources said that neither the board's current chairman, Arlington County Manager W. Vernon Ford, nor any board members were under investigation.

Acting Fairfax County Executive J. Hamilton Lambert, a member of the board, said he was surprised by the investigation. "We really do very little at the meetings. They're very perfunctory," Lambert said.

According to an audit of the tax board for the year ending June 30, 1979, the agency collected about $5.3 million in revenue, and distributed about $5.2 million to the localities with the remainder being used for operating expenses.

Fairfax County received $2.4 million in cigarette revenue; Alexandria received about $1 million; Arling, $992,000, Fairfax City, $328,624; Falls Church, $183,000; Vienna, $167,000; Herndon received $66,000 and the town of Clifton, $690.