Federal banking officials raised the interest ceiling on small savings accounts yesterday and authorized a new type of mortgage that would allow homeowners to gamble against future inflation.

The actions came in separate votes by the Federal Home Loan Bank Board and the Federal Reserve Board. The bank board raised the ceiling on passbook accounts in savings institutions from 5.25 percent to 5.5 percent while the Federal Reserve lifted the ceiling on bank savings accounts from 5 percent to 5.25 percent.

Both boards also voted to allow banks and savings institions to issue a new type of four-year certificate in any amount, no matter how small, with interest pegged to the interest paid on certain Treasury securities. Based on current rates, the interest on the new certificates would be 8.1 percent.

The actions take effect July 1.

The more controversial action by the bank regulators, however, gave banks and savings institutions permission to issue variable-rate mortgages. These mortgages, which would be optional for the borrower, represent something of a gamble for both the bank and the homeowner.

Under the terms of these lending agreements, the mortgage interest rate would be allowed to rise or fall one half of one percent a year with inflation. During the life of the loan, the interest rate could not rise more than 2.5 percent, but there would be no limit on the amount it could drop.

Changes in the rates would be dictated by changes in the average cost-of-funds index issued by the bank board.

The actions by the two regulatory boards came just eight days after President Carter asked Congress to make essentially the same changes.

Congressional reaction yesterday was swift and mostly negative. Rep. Benjamin S. Rosenthal (D-N.Y.) accused both agencies of "skirting around the discrimination issue" by failing to permit savings institutions to pay small savers the same rate large investors earn to $10,000 money market certificates.

Rep. Fernand St Germain (D-R.I.), chairman of the House Banking Subcommittee, objected to what he called a decision by "administrative fiat." And Rep. Frank Annuzio (D-Ill.) said he would introduce legislation to block the new type of mortgage.

Robert H. McKinney, chairman of the bank board, said that because the new regulations are certain "not to please anyone, that makes them fairly acceptable."

The banking industry praised the variable-rate mortgages, but expressed dissatisfaction about the higher rates for small savers. Calling higher rates "a major change in the federal government's housing policy," the National Saving and Loan League warned, "these actions could lower the volume of housing, restrict home ownership and produce a more severe earnings crunch for the thrift industry."

The National Association of Mutual Saving Banks said the cost-of-funds index to be used to determine VRM rate changes move too slowly to permit thrifts an adequate spread. And the U.S. League of Savings Associations predicted "still higher monthly payments for borrowers and trouble for the housing industry."

In remarks before yesterday's vote, McKinney said he had "agonized" over the "symbolic" one quarter of one percent increase for small savers, but felt that thrift institutions could do no more at this time. Last month their withdrawals exceeded deposits by a record $1.50 billion. Because 30 percent of all savings at thrifts are in passbook accounts, McKinney calculated that these savers would reap an extra $140 million in interest payments this year.

He expressed the view that borrowers - who pay back loans in inflation-cheapened dollars - have been subsidized too long by savers.

Savings institutions issuing variable rate mortgages will be required to paint a "worst case" for the borrower, indicating how much the payments would be if the mortgage interest rate increased by 2.5 percent. They must also provide a 10-year history of the national cost-of-funds index.

The borrowers must be offered a choice between a fixed-rate and a variable-rate mortgage, and no more than half of a saving institution's loans may be VRMs. The agency also soon will be considering authorizing rollover mortgages, a form of multi-year VRMs.

In other actions, the agencies voted to reduce penalties for early withdrawals from savings accounts and to permit individuals ororganizations to pool their funds to take advantage of higher interest rates.