A FLOOD OF light from the low winter sun streams through his second story window. From here Andrew Oliver's view falls on the dormant gardens, sloping prettily down the hill. Beyond this protected, almost cloistered serenity rises a clutter of chimneys and television antennas from the Kalorama embassy quarter near Dupont Circle. And farther off, the downtown rooftops, out of focus in the bright haze of cold weather exhaust.

Oliver stands, hands in his pockets, his towsled brown hair giving him a boyish appearance, and stares out into an empty corner of the yard.

"One of our elm trees died," he says. "Yep. Sure was Dutch Elm's disease. They got it all cut up and took it away in a day."

The elm tree is surely dead, but at the Textile Museum where Oliver is director, certain life signs are pulsing with promise.

In the neo-Georgian building facing S Street, the building that museum originator George Hewitt Myers bought more than 50 years ago to house his extensive textile collection, carpenters are busily hammering into place a $670,000 facelift. When work is finished (in July, Oliver expects), visitors will find it has shed its grandmother's-attic image for one of a professional gallery.

Plans for the renovation date back to 1975, shortly after Oliver, a former curator with the Metropolitan Art Museum, took over the Textile Museum directorship. When completed, they will add greatly to the wall space available for mounting the museum's more than 8,000 rugs and tapestries. They also will provide a modern space for the museum's library, climate-control facilities to protect old textiles, recessed track lighting throughout and a brand new coat of white paint.

Designing the renovation is the New York architectural firm of Conklin and Rossant. William Conlkin, a museum trustee, has engineered changes that are not necessarily dramatic -- no skylights or pedestrian causeways here -- but ones that will make of the 70-year-old house an exhibition area befitting one of the world's two museums (and the only one in this country) devoted entirely to textiles.

"We looked at rising construction costs and decided we'd dilly-dallied long enough," said Oliver. "We decided to plow ahead and get it done. I know we're going to make some mistakes, But I'm glad we went ahead."

Museum officials had considered closing down the museum for a year until work was completed, but decided instead to keep at least two small areas in operation. The construction, limited to the building's interior, has completely shut off the main exhibition area, forcing the body of the museum's collection into temporary storage. The former dining room and living room on the ground floor of the Myers house next door, however, remain open to the public with textile displays. the Textile Museum consists of two buildings. The smaller of the two, where Myers lived, was designed in the early 1900s by John Russell Pope, who, among other things, also designed the Jefferson Memorial. The second and larger building, where the renovation is taking place, was designed a few years earlier by Waddy Wood. Myers bought it for his collection and until recently it was the only one of the two where exhibits were mounted.

The Myers house is now the center of museum activities. The other building is considered an annex.

To enter the annex, you pass through a long hallway into a great room on the north side. Here false beams have been removed to make room for climate-control ducts and track lighting without sacrificing ceiling height.

The ducts will supply air to an enclosed display area designed for smaller and more sensitive textiles. This is being built on the north wall, where the windows have been covered over to provide additional and uninterrupted wall space.

Another great room rises two stories from the ground floor on the south side. As a library or reception room, it must have been a fine place for milling guests to await the appearance of debutantes on the balcony above. At a reception to open an exhibit of indonesian textiles last year, a small orchestra played indonesian music from the balcony while visitors browsed below.

Now the balcony is being partially closed off for more wall area. In the library, the false beams are being removed and the ceiling lowered approximately two feet, lessening the cathedral effect.

To save a few areas where displays can be enjoyed in more intimate surroundings, the museum is keeping two smaller rooms on the eastern side of the ground floor.

Much of the second floor, however, has been torn apart and redone. Walls that once divided the east side into many tiny closets, servants quarters, bathrooms and stairways have been removed to create one long gallery. In doing so, the museum has added more than 100 linear feet of display area. The third floor has been remodeled as well to make room for new curators' offices, storage spaces and a library bathed in light by several south-facing dormer windows.

The basement has been modernized to provide a badly needed central storage area with climate control and a large examination room.

Some of the money for the work came from the National Endowment for the Arts, which granted $60,000 to the project on a three-to-one basis (three dollars from the museum for every dollar of grant money.) The lion's share was pooled by the trustees themselves, who donated $400,000 among them (a sum unheard of around the museum in years gone by) and organized the museum's first major public fund drive.

In all they hope to raise $1.5 million, half for construction, the other half to increase the museum's endowment, presently at $2.3 million. They have established a $150,000 restricted fund for purchases of old-world textiles. They will hire a full-time librarian, plan to increase the number of traveling exhibitions and hope to improve conservation activities, now housed largely in the former chauffeur's carriage house.

Many of those close to the museum agree with trustee W. Russell Pickering, head of the fund drive, that the "explosive interest" in Oriental carpets that began in the late '60s spilled over to the museum. Ten years ago, membership in the museum was less than 500. Today more than 1,600 persons here and abroad belong.

An exhibit of rare indonesian textiles last year, one of the largest of its kind anywhere, brought in visitors from around the world. A major world conference on Oriental rugs has chosen for the first time to meet in the United States, this coming October at the Textile Museum.

With all this interest, Oliver says, museum officials "finally realized the Textile Museum is no longer a private club." (He likes to chuckle over mail he receives from fans around the country. "I really enjoyed your programs this year," he quotes one woman writing from California. "Too bad I couldn't make any of them.")

Pickering attributes the increased activities also to "a gradual infusion of people who are a lot more active in the rug world."

Trustees such as Arthur D. Jenkins, an 82-year-old former newspaper publisher and entrepreneur in Mascoutah, Ill., have contributed both textiles and moral support. But much of the new blood revitalizing the museum is younger, and more of it from the Washington area.

"When I was 35 years old in 1965," said Pickering, "I was one of the few members under 65. Before there was only one local turstee. Now there are six."

And Oliver remembers one of the old timers siding up to him at a reception and remarking: "Who are all these people? I don't remember seeing any of them before."