Freshman classes began this month for 46 black students at George Mason University, an early start that is symbolic of the school's increasingly aggressive but not always successful effort to attract minorities to its overwhelmingly white campus in Fairfax County.

The students, all regarded as talented but having weak academic backgrounds, are attending a six-week summer institute designed to bolster their reading, mathematics and study skills. Satisfactory completion of the course is a condition of their admission to the state-supported school south of Fairfax City.

The students begin their day at 7:30 a.m. and take 9 1/2 hours of classes. "It's hard work," said Sydnee Reese, an 18-year-old from Newport News, Va., who plans to major in broadcasting. "You really have to budget your time."

As the proportion of college-bound blacks declines in Virginia and across the country, George Mason is stepping up its search for qualified minority applicants. Among its techniques are a persistent recruiting campaign, mass mailings, special brochures, a faculty mentor program, student counseling and the summer institute.

It has paid off, but not enough, school officials say.

George Mason, located in a state that is 19 percent black, had less than 5 percent black enrollment in 1985-86. Blacks are not much better represented at most of the other state-supported schools; the Virginia average for majority-white institutions was 7.5 percent black in 1985-86.

Virginia is one of 17 southern and border states that are required to meet desegregation goals in their public colleges under a 1973 federal court order stemming from a lawsuit by the NAACP Legal Defense and Education Fund. Despite a desegregation plan adopted in 1978, Virginia officials recently reported that the gap between the percentages of black and white high school graduates entering college has nearly doubled in eight years.

Virginia has asked the federal government to relax its monitoring of state desegregation efforts, arguing that some gloomy numbers should not overshadow some major gains.

Although its minority enrollment is growing every year, George Mason has not met its federal desegregation goals for admitting black students from Virginia since the 1984-85 school year. In 1986-87, for example, it enrolled 195 black freshmen or transfer students; its goal was 300.

"Comparatively, I think we look pretty good," university President George W. Johnson said in an interview. "In terms of an ideal, we have a long way to go."

GMU officials blame societal factors outside the school's control for the shortfall, and some say that Reagan administration cuts in student aid have made it more difficult for many blacks to attend college.

George Mason has tried to expand the pool of college-bound blacks by admitting some students with motivation and talent but below-average grades or standardized test scores, on the condition that they sign up for the summer institute.

Similar programs have generated complaints that schools are lowering standards, but university officials argue that the risks are worth taking because schools benefit by identifying and developing students with potential.

Rod Young, a George Mason senior who grew up in Richmond, was in the first group of students to go through the summer program in 1984 and is a volunteer counselor to this year's enrollees.

"It's one of the reasons I'm here," said Young, who hopes to attend law school after graduation. "It really helped my writing skills. I've gotten nearly all As in English here."

Also, the university employs an energetic recruiting program that includes at least one visit a year to every public and private high school in Virginia. "We made seven visits to T.C. Williams" in Alexandria, said Patricia Riordan, George Mason's admissions director. "We nearly wore out our welcome. It paid off . . . . We had a 30 percent increase in applications."

The university sponsors several events each year designed to attract prospective minority applicants, such as a seminar on being a minority student at a predominantly white institution.

The school publishes a special brochure for minority applicants that is sprinkled with photographs and profiles of black students and faculty members Only 3.4 percent of George Mason's faculty is black, but that is above average for the state universities.

The school mails enthusiastic letters to promising black high school students encouraging them to apply, and it follows up with postacceptance letters or calls. Some applicants who do not get in attend Northern Virginia Community College, and George Mason officials regularly check on their progress in hopes that they will apply to GMU again in two years.

George Mason has worked with local civic groups to sponsor tours of the campus for students when they are still young enough to sign up for the right preparatory courses.

In April, the entire seventh grade of Hine Junior High School on Capitol Hill went for the day. D.C. students do not qualify to help GMU meet its federal civil rights goals, but George Mason has begun recruiting them in hopes of increasing its overall black population.

Mindful of the above-average dropout rates for black students, George Mason officials say they are trying harder to hold on to those who do enroll. Faculty members are alerted to warn school officials by the first exams in October whether black students are faltering, so counseling can be offered.

University officials say they have no current reliable statistics on whether the retention effort is working, but they say they are compiling them.