At George Washington University, where tuition has reached $6,710 a year, the number of new freshmen climbed 35 percent this fall to a record 1,276. Their average score on the Scholastic Aptitude Test also was higher, climbing 10 points to 1,100 out of a possible score of 1,600.

Meanwhile, at the University of the District of Columbia, tuition remains low at $462 a year and the college remains open to any high school graduate regardless of grades or test scores. But UDC's freshman class has shrunk by almost 10 percent.

To the surprise of many college officials, the paradoxical pattern taking place here also appears to be occurring around the country. Healthy enrollment gains have been reported at many relatively selective colleges -- both high-cost private schools and major state universities. Declines are taking place at many less selective schools, most of them low-cost, open-admissions community colleges, and at nonselective private colleges, too.

Although complete national figures on this fall's college enrollments have not yet been compiled, experts on higher education suggested that the changes probably reflect the upswing in the national economy and some improvement in the quality of high school graduates, as well as growing sophistication among both college applicants and recruiters.

"I think we're seeing a lot more stratification between the have and the have-not institutions," said Alexander Astin, director of the Higher Education Research Institute at the University of California at Los Angeles.

"The students are becoming more sensitive to the economic benefits of college, and they think they get more benefit from a degree from a more prestigious, selective school," Astin continued. "And the selective colleges are recruiting the stronger students and giving them financial aid . . . . The nonselective colleges may be getting it from both ends. Their best students are looking elsewhere and many of their marginal students may not be going to college at all because there are more jobs for them than there were a few years ago."

Astin said the open-admissions colleges may be hurt further by aggressive recruiting by the military services.

In the Washington area, freshmen enrollment and average SAT scores are up this fall at American University, the University of Maryland, George Mason, and Georgetown. But the number of students has dropped by the largest percentages ever at Northern Virginia Community College and Prince George's Community College. Enrollment is also down at Montgomery College, another open-admissions school.

Despite a shrinking pool of potential students in their late teens, enrollments and apparent quality have risen at a long list of selective colleges nationwide, from Boston University to Kalamazoo College in Michigan. At the most selective schools, such as UCLA, Princeton, and Cornell, competition for admissions has become fiercer than ever.

Meanwhile, enrollment has fallen at open-door schools in many states, including an estimated 6 percent drop in California's large community college system.

"The competition among colleges is tremendous," said David W. Breneman, the president of Kalamazoo and a former senior fellow at the Brookings Institution who has written a major study of college enrollment trends. "It may well be that the students are trading up. There seems to be more concern about quality and standards, and the quality colleges stand to benefit from that trend."

At UDC and several of the local community colleges, spokesmen said the enrollment decline stems largely from the flourishing economy that has increased employment and caused a labor shortage in some of the suburbs.

"The economy is up and unemployment is down," said John Britton, director of public information at the University of the District of Columbia. "That tends to mean that people are less likely to come to our type of college to upgrade their skills, particularly the older students who are here part time."

The part-time students, who comprise almost two-thirds of UDC's enrollment, declined this fall by 6.5 percent compared to a 3.6 percent drop in those carrying a full load of courses. The university's full-time equivalent enrollment, calculated by dividing the number of course credits taken by the 12 credits needed for a full load, declined by 6.3 percent to 7,827.

At Northern Virginia Community College, the full-time equivalent enrollment is down 8 percent to about 15,400. At Prince George's Community College it is down 6.5 percent, at Montgomery College, 5 percent.

Nationally, the community colleges are expecting their steepest enrollment decline ever after more than 20 years of major, almost uninterrupted growth, said Dale Parnell, president of the American Association of Community and Junior Colleges. Their only slight decline in the last decade was a drop of less than 1 percent in 1978 -- also a prosperous year for the economy.

"I think it's mainly a matter of economics, sheer economics," Parnell said. "When times are good, people get a job or they have a few more bucks in their pockets and they will go away from home to a four-year college . . . . We've continued to grow in part just because the baby boomers were there. Now the baby boomers are gone because of the sharp decline in births that began in the mid-1960s . We notice it." But Parnell added: "The drop this year is bigger than any of us anticipated."

The situation at George Washington University is the opposite. After two years in which freshman enrollment dropped by a total of 19 percent, the incoming class is the school's largest ever. Overall enrollment has stabilized.

George Stoner, George Washington admissions director, said the school has stepped up efforts to attract students: improving its literature, sending representatives to high schools in the Midwest and South where it had not recruited before, and offering more "merit" scholarships, based on academic performance rather than financial need.

Astin at UCLA said this is part of a nationwide pattern of "much more sophisticated recruitment" by the more selective colleges -- both private and public. He said community colleges and the weaker private schools "just don't have the resources to compete." Also, despite Reagan administration efforts to make major cuts several years ago, federal aid is widely available, totalling almost $15 billion. About half of it is in subsidized low-interest loans that have grown rapidly.

Among area colleges: The University of Maryland has 363 more new freshmen than a year ago and a three-point rise in its average SAT score. All of the increased enrollment is accounted for by out-of-state students. Over the past two years they have risen from 18 percent to 29 percent of the incoming class after the Board of Regents lifted the limits on out-of-state enrollment. American University continued a rebound that started last year after a sharp drop in freshman enrollment in 1982. Average SAT scores went up 29 points to 1,070 and are now almost 130 points higher than they were in 1978 when the university started to raise admission standards. Total enrollment at AU rose this fall for the first time in five years, while the number of freshmen is almost back up to the 1978 level but with much higher quality. The size of the new freshman class at Howard University declined by 199 students, while average SAT scores rose by 9 points. William H. Sherrill, dean of admissions and records, said this reflects an effort to upgrade admission standards, which are planned to go up again next fall. Since 1981, the average SAT score of new freshmen at Howard has risen by 45 points to a total of 841 this year. George Mason University in Fairfax raised both the size and quality of its freshman class after a slight dip in size last year. Total enrollment, which had dropped last year because of a state budget cut, spurted by almost 1,000. The area's most selective college, Georgetown University, had its largest increase in applications ever (up 1,200 to 9,727), and took the smallest ever proportion of those who applied (27.9 percent).