Dana B. Hamel, who has headed Virginia's sprawling community college system since it was created 13 years ago, has agreed to leave office amid widening complaints of financial mismanagement of they system.

Much of the controversy has stemmed from recent findings by the system's own internal auditors, who reported a series of irregularities including possible enrollment padding. The enrollment figures are used to determine how much state financing the colleges should receive. The State Council for Higher Education currently is auditing the system's enrollment statistics.

Under an agreement reached with the State Board of Community Colleges, Hamel, 55, will step down from his $51,000-a-year post as chancellor by July 15. He will continue as a consultant to the community college board until June 30, 1980, and is expected to retain his current salary.

The arrangement was proposed by Hamel at the start of a board meeting in Richmond yesterday morning and was accepted during a 1 1/2-hour close-door session. The board's resolution said the Hamel would be "temporarily relieved of his day-to-day duties." The board left uncertain who would head the colleges system after June 30, 1980. It plans to name an interim chancellor for the next year.

In his statement, Hamel referred obliquely to the current controversy, saying that "unfortunately recent events" had "disrupted the continuity" of his efforts to reorganize the college's management Some other officials said, however, that Hamel clearly had stepped down under pressure and might have been ousted had he not made the offer.

The community college system - a collection of 23 two-year colleges on 33 campuses that includes about 100,000 full-time and part-time students - has encountered increasing crititicism in recent years. The questioning has occurred at time when the system's enrollment has started leveling off after years of rapid growth, according to state officials. The system now spends about $120 million a year.

Auditors for the Virginia General Assembly questioned the accuracy of the system's enrollment statistics in 1975.

Last November, Charles B. Walker, the state secretary of administration and finance, urged the community college board to assume the community college board to assume broader responsibility for college audits. And after the recent series of critical audits was made public, Walker gave the board additional proposals for tightening the colleges' management.

A spokesman for Gov. John N. Dalton said yesterday that Dalton has asked board members to act on Walker's recommendations.

None of the recent criticism has focused on Northern Virginia Community College, officials said yesterday. NVCC, with five campuses, is the largest in the community college system and the only one in the Washington area.

Robert D. Stevenson, the community college system's chief auditing official, said in a telephone interview yesterday that enrollment reporting was one of three key issues raised in recent audits. Auditors found that some college enrollment figures improperly included students who had withdrawn after classes started, he said.

The other major findings, Stevenson said, were that some colleges kept inadequate inventories of their equipment and that some officials engaged in improper procedures that circumvented requirements for obtaining approval for purchases.

Despite the recent controversy, Hamel was praised by officals for developing the college system since he took on the task in 1966. He previously had headed the former Virginia Department of Technical Education. "He's done a hell of a job building the system and now you need some fine tuning," one state official remarked yesterday.