Officials of the Department of Housing and Urban Development are weighing a legislative proposal to offer a limited number of variable-rate mortgages through the Federal Housing Administration, as part of an effort to make FHA mortgages more like those now being offered in the private market.

Also under consideration is a demonstration program of the shared-appreciation mortgage for first-time buyers.

While the proposals are still in the talking stages, they do show that the administration is considering whether the popular federal housing insurance program should continue on a fixed-rate basis.

One legislative proposal would have HUD and the Government National Mortgage Association seek authority to insure variable-rate mortgages.

The number of VRMs would be limited to 10 percent of the total single-family-home lending activity for a year, with interest-rate adjustments tied to the Treasury's costs of funds.

Limits on the frequency and size of interest-rate adjustments would be imposed, possibly at one a year and with a maximum of no more than 5 percent over the life of the mortgage.

A drawback to this approach is that most of the inflation risk is transferred from the mortgage investor to the buyer. The higher monthly payments resulting from possible rising interest rates could increase default ands foreclosure rates on FHA mortgages. If the variable-rate mortgage becomes a permanent part of FHA, controls on participating lenders might be necessary to ensure fair pricing, officials say.

Some private insurers believe HUD would do better to end FHA entirely.

Ralph Waldo, president of the Mortgage Insurance Companies of America, argues for a gradual phrase-out of the unsubsidized portion of the program. Any prospective buyer who can qualify for a privately insured loan at a lower premium, he contends.

Earlier this year, Waldo told the Senate housing subcommittee that in 1980, "35 percent of our insurance involved 95 percent loans [loans requiring down payments of only 5 percent]. The comparable FHA program in that year involved 15 percent of oans with a 5 percent down payment and 22 percent of loans with 4 and 3 percent down. The average down payment for our industry was 12 percent; for FHA 11 percent."

The proposal for a federal experiment in shared-appreciation mortgages draws from the private market, in which borrowers are granted a lower-than-market rate of interest in exchange for agreeing to share with the lender a certain portion of the appreciated value of their properties. The sharing would take place when the property is sold, transferred or refinanced, or when the loan is terminated.

Under a proposal being circulated at HUD, the agency would insure mortgages with an eye toward eventual purchase by the Governmen National Mortgage Association and secondary market investors.

To ensure that investors who buy the loans receive a market-rate yield, HUD would guarantee payment of deferred interest.