Federal savings and loan associations were authorized to begin offering variable rate mortgages today amid concern by some consumer groups that homeowners could be "wiped out" by rising interest costs under the new system.

The new mortgage system, which was approved last week by the Federal Home Loan Bank Board, allows the lender to tie the mortgage rate to a variety of interest rate indexes published by the federal government. These include the monthly interest average of Treasury bills, the cost of funds to savings and loan institutions and average mortgage rates.

The most volatile of these during the past year has been that for three-month Treasury bills, which increased 124 percent between June and December last year, from 6.99 to 15.66 percent. (It has since fallen by 14 percent.) The least volatile of these is the cost of funds index, which moved up just 3.8 percent in the June-to-December 1980 period. (No 1981 figures are available.)

Assessments of what the impact of the new mortgage system will be vary widely. Consumer organizations generally oppose the variable rate system, while industry officials claim it will make issuing mortgages desirable for lenders again.

Ralph Nader's Public Interest Research Group calculated that a 9.12 percent mortgage issued in December 1977 and pegged to the three-month T-bill rate would have risen to 18.54 percent by December 1980. Were the entire increase passed on to the borrower, monthly payments would have gone up almost 90 percent during the period, the group estimates.

Another feature of adjustable mortgage loans is negative amortization, so the loan amount is increased rather than the monthly payments rising. Robert Gnaizda, a spokesman for Public Advocates Inc. in San Francisco, claims that negative amortization can virtually wipe out a buyer's initial investment in just six years.

His example is that of a $20,000 down payment on a $120,000 house. The initial mortgage rate is 12 percent. In the first year, the index, to which the adjustable mortgage loan is tied, goes up by half to 18 percent and remains there for the next five years. Therefore the yearly payment required for full amortization becomes $18,000.

The buyer agrees to increase annual mortgage payments by $1,000 each year. The second year, he or she pays $13,000 and adds $5,000 to the principal; the third year, the payment is $14,000, and $4,000 is added to the principal. This continues through the sixth year until the total amount added comes to $15,000 plus interest which, Gnaizda theorizes, might raise the negative amortization to $18,000. Thus in six years the buyer has lost almost his or her original equity.

In another subcommittee, a $50,000 loan is made at 15 percent. A 1.5 percentage point increase is permitted yearly. After four years the maximum interest rate is 21 percent. The lender has put a 5 percent increase on the amount the mortgage payments can go up annually. This means that at the end of 12 years, the original monthly payment of $632.22 has risen to $1,100.07. But even this increase is not sufficient to cover the interest rate rise, so negative amortization results. At the end of 12 years the original $50,000 due has jumped to $62,500 due.

These examples present the worst case, however. They do not take into consideration appreciation or downward interest rate fluctuations.

The following cases were worked out by Jim Christian, chief economist of the U.S. League of Savings Associations. They show how much equity a homeowner would have foregone had he or she been able to get an adjustable mortgage loan before now. The calculations are based on a 20 percent down payment and a 30-year mortgage. Home prices and appreciation are based on National Association of Realtors national averages.

Interest rate changes are made every six months as necessary according to the Federal Home Loan Bank Board's composite index of mortgages on new and existing homes. If rates fall, the buyer has the option of paying off the mortgage early or reducing monthly payments if rates fall. If rates rise, the buyer continues to pay the same monthly dollar amount, while the extra amount due is added to the principal.

In the first case, the buyer purchases a $48,250 home in May 1978 when the mortgage rate is 9.37 percent. The down payment is $9,560, and the monthly payment $320.91. Three years later interest rates have risen to 15.37 percent. To maintain the same monthly payment of $320.91, the owner must add $1,783 to the principal.

Meanwhile, however, the market value of the house has risen to $65,600. So the owner's equity amounts to $25,217. But, if the buyer had had a mortgage at the fixed rate of 9.37 percent during those three years, his or her equity would amount to $27,807, or $2,590 more.

In the second case the loan is made on the same house in May 1979, now worth $56,125. The loan rate is 10.45 percent. Two years later the owner's equity would amount to $44,421 with an adjustable rate mortgage, or $1,689 less than with a fixed mortgage. In the third case, the house is worth $65,000; the loan rate is 13.92 percent. A year later, there is no negative amortization. But the amount of mortgage principal paid off is just $51 instead of $123 under a fixed-rate loan.

Although the Federal Home Loan Bank Board allows rate and payments to change as often as monthly, Christian believes that lenders will not change rates more than once a quarter because of the 30-day notification rule and the sheer paperwork involved. Moreover, he feels that lenders will be more inclined to pick for an index either the average mortgage interest rate or the average cost of funds for S&Ls rather than the more volatile Treasury securities rates.