JOEL AGEE needed help. In 1978 Agee, writer and son of novelist James Agee, was desperate for "time to write and do nothing else." Unable to find that time in his noisy Brooklyn apartment, he applied to the Virginia Center for the Creative Arts (VCCA) at his editor's suggestion, with the following deeply felt epistle:

"In addition to the growing pressure of a deadline, I'm being plagued by an increasingly disruptive atmosphere. The building I'm living in is being renovated. Plumbers, painters, and electricians come trooping in and out of my apartment. A large joyful family has moved into the apartment below mine, and they have been treating me and my wife and daughter to daily concerts from their powerful Hi-Fi set. Needless to say, I am finding it very difficult to work under these circumstances. It would certainly be a boon to be able to concentrate on my work without interruption for a few weeks in a quiet setting . . ."

He was accepted at VCCA, where he spent a quiet six weeks working on his book, "Twelve Years: An American Boyhood in East Germany" which was published in 1981. Agee found the atmosphere "very pleasant" and he attributed this to fact that people were "doing what they wanted," working on things that mattered to them. Agee found the center so helpful to him he returned this year.

The VCCA, which is patterned after the more established MacDowell Colony in New Hampshire and Yaddo in Saratoga Springs, N.Y., is in Amherst, Va., 160 miles southwest of D.C. It was founded in 1970 by Virginia author Nancy Hale ("The Life in the Studio," "Secrets" and "The Prodigal Women"), whose idea was, according to director William Smart, to "give a writer a motel room and meals."

The center had a few homes before its present one, first in Wavertree, then Charlottesville. In 1975 it was transplanted to the more spacious Mount San Angelo estate, where a more artists could be in residence at one time. The new, more modern facility houses 24 fellows: 13 writers, nine visual artists and two composers. Each fellow gets three meals a day, a bedroom and a studio in a converted cow barn. The "Golden Rule," as the brochure puts it, "is that no one interrupts a Fellow in a studio."

Artists from all over the country apply; about 200 of the 800 to 1,000 applicants are accepted yearly. Their work, whether it is a musical score, a painting or a short story, is judged by a panel of past fellows. The fee is $15 a day but if an applicant is unable to pay the fee, it is waived or adjusted--or artwork is offered as payment.

Once an artist arrives at the center for a stay that typically lasts from one to three months, the only commitment he has is to himself. Many fellows report that it takes a few days to readjust to a new life, without phones, families or clocks.

No phones.

No commercials.

Just a desk with a lamp, a waste basket and a bed.

You've arrived at your studio at the VCCA. You shut the door behind you, set the typewriter down, look around. There are two windows that look out on a field with the obligatory grazing cow, endless pastures and shady old trees.

You plug in the typewriter. But after a few sentences, the pounding halts, while the insistent hum of its motor paralyzes you. You wonder about the artists who had the studio before you. You picture them in the studio, buried under stacks of prose. You decide it's all been a farce, calling yourself a writer.

No phones.

You panic, you sweat. Then you hear it. Buzzing. It's an enormous fly with a crazy flight pattern designed to drive you crazy. So you use the rough draft of your Great American Novel to chase that insect around the room, waving the tattered prose around till you're dizzy. Defeated, you sit down, ignore the buzzing and resume writing.

Author William Styron, who is on the advisory board of the VCCA, sees a definite need for the retreat. "I have not felt the need only because I've been very fortunate--my writing career has taken another turn, I've found my solitude in my own way. But I plainly recognize that other people have the need and the desire . . . John Cheever and Philip Roth were always impassioned for their devotion for those places, specifically Yaddo. Both Philip, who is my good friend, and John, who was also my good friend, were devoted to Yaddo, as a place where they could work in a spontaneous and fruitful way, and now that VCCA is there I think people like John and Philip will be able to work there in the same way."

Styron also said he "doesn't think the country at large has a great concern for artists and writers in general" and the existence of places like the VCCA is an "excellent way to encourage" artists.

Richard Wargo, who has been to the center five times in four years, works best at night. His nocturnal composing sessions have become legendary among the other fellows. "There's no concept of the time of day here, I just like the total quiet of the night." During his recent stay at the center, the 26-year-old composer usually worked all through the night, with only Jerome, a gray cat, for company. He kept a saucer of milk in the middle of the floor for Jerome.

Wargo speaks softly. His eyes glance away from the person he speaks with, especially if forced to talk about himself. Wargo, who attended the Eastman School of Music, now lives with his parents in Scranton, Pa., where his father is a maintenance man at a monastery and where his grandfather was a coal miner. He says his parents are very supportive of his life style, of his music. "They understand," he says.

He has had several works commissioned by the Northeastern Pennsylvania Philharmonic, one of which was televised. He is now working on a comic opera, based on Neil Simon's play, "The Good Doctor." After seeing a production of the Simon play, Wargo wrote the playwright for permission to put the work to music.

"VCCA gives me concentration I take home with me. It also gives me a lot of ideas . . . it gives me a chance to recharge my battery." He stops, hesitant, twirling a blade of grass, "It isn't the real world here."

The day he left, Wargo bought a Virginia ham and gave it to the kitchen staff. He also gave bouquets of flowers to two other staff members. One of the fellows noted that Wargo's fees had been waived, and that he probably spent what little money he had on the gifts.

Jerome was restless for days, wandering from studio to studio, searching for a new home.

James Dodson says a writer shouldn't get caught up in the idea of Writing Fiction, that he should just tell a story. Dodson, a 30-year-old writer with the build of a college football player, recently gave up his job as a staff writer for the Atlanta Weekly. A resident of Greensboro, N.C., he is most comfortable living in the South, but is reluctant to be labeled a "southern writer." Dodson is afraid of "terminal lyricism"--a tendency he finds in many southern writers.

He's working on his first novel, "Union Grove," which is scheduled to be published by Harper and Row next year. He says the novel, which takes place in a small town about an hour from the colony, addresses an issue "the south is just beginning to deal with . . . farmers, rural people coping with 'progress' . . . it's not very autobiographical," he pauses, smiling, "but it is some."

Five naked people stare down at Judy Jashinsky. They are part of the in-progress series she calls "Coy Nudes." Jashinsky, a Washington artist, stands in the colony's gallery, having just hung her work. She explains that the series got started when a gallery asked for a self-portrait. She created a cardboard cutout of herself, and later began to add to the collection. In addition to nudes, she also does still lifes. Jashinsky's studio is filled with her subjects--moth wings, driftwood, seashells, smooth stones and pebbles. She says the other artists "see I paint natural things and they bring me stuff. They put things in my mailbox."

Jashinsky teaches art at Prince George's Community College and paints at her Capitol Hill home. She came to VCCA for many reasons; one is that she enjoys the support of other artists. "Every time I talk to somebody here I get artistically generated." She's so stimulated she's having trouble sleeping: "At home I fall asleep with the cat in my lap, watching 'Hill Street Blues.' Here I'm too hyped to sleep."

Kathleen Fox wants to paint sounds. Sitting with brush in hand by the window in her studio, the converted cow barn, she says, "The cows come up from the valley at dusk. There's this beautiful mist rolling in." She makes rolling motions in the air with her red-tipped brush. "The only way you know the cows are there is that they're chewing. You can hear them . . . the sound of cows breathing."

Her table is cluttered with shells, butterfly wings, curling magnolia leaves. Paint tubes and paper towels surround the painting she works on now, a large, careful magnolia flower. She paints while she talks about why she is at the VCCA: "Nobody bugs you. You've got everything you need--a studio and hot water. It's going to be hard to go back to work." When she's not painting she works as an administrator at Harvard.

"One of my beefs is that people think if you don't earn all your income from your art you're not a real artist--coming here legitimizes that you're an artist, everyone here has other kinds of jobs." Fox finishes a leaf, then puts her brush aside. "It's a relief to be here because you don't have to explain."

In the late evenings, after most of the fellows have quit for the night, several may pile into the car and head for McDonald's or Howard Johnson's, nine miles down the road. Others gather to hear a reading of excerpts from someone's novella or take a walk around the nearby Sweet Briar campus, with its gravel lanes, wooded back roads and stables. The colony's relationship with Sweet Briar College, a private liberal arts college for women that leases the land to VCCA for $1 a year, is mutually beneficial. The fellows can use the college library and recreational facilities and the students are sometimes treated to poetry readings and lectures.

Major funding for the center comes from the National Endowment for the Arts, the Virginia Commission for the Arts and fund-raising events. Smart worries that because the 445-acre grounds are so impressive, and because the facilities are so nice, the center appears to need no help. Not true; it is constantly in need of money.

Smart looks at the "artist colony movement as a sort of Utopian community," a temporary way of life that provides the artist with a guilt-free existence. He recognizes the artist's obstacles because he's been there.

"I was very depressed because I wasn't Hemingway yet," says Smart, who in addition to running the VCCA teaches creative writing at Sweet Briar. "I was trying to write, but up against so many physical frustrations. I had no time to write--there's a sustained voice in fiction you need time for." Smart pauses for a moment, his usual warm expression turning worried. "If you only have rare, occasional successes it's hard to justify the selfishness . . . artists are selfish, but they have to be."