A federal judge has refused to dismiss tax charges brought against a former CIA agent, saying Iran-contra independent counsel Lawrence E. Walsh has the jurisdiction to bring the case, officials said yesterday.

As a result, Thomas G. Clines's trial on the four-count indictment remains scheduled to begin Sept. 4 in Baltimore, said Mary Belcher, spokeswoman for Walsh's office.

Clines assisted the Nicaraguan contra arms resupply network run by Richard V. Secord and Albert Hakim and overseen by then-White House aide Oliver L. North. He helped locate a European arms dealer as a source of weapons, helped arrange for shipping and lined up crews to transport the supplies to the rebels.

Accused of hiding arms sales profits from the government, Clines was indicted Feb. 22 on charges of underreporting his income to the Internal Revenue Service and failing to report foreign financial accounts to the Treasury Department. If convicted on all counts, he could face a maximum of 21 years in prison and $950,000 in fines.

U.S. District Judge Norman P. Ramsey in Baltimore rejected Clines's bids to dismiss the charges and to suppress financial records obtained through a U.S.-Swiss treaty.

Clines argued in his motion to dismiss that the alleged tax crimes were "without any relationship whatsoever to the events of the {Iran-contra} affair" and outside Walsh's jurisdiction because Clines has not been charged with committing any crime by virtue of his participation in the purchase, sale or transfer of arms. He also has not been charged with any crime relating to supporting the contras.