Kickbacks and referral fees in connection with home mortgages have generated a policy brouhaha inside the Reagan administration. They also raise new questions about the federal government's role in protecting borrowers seeking home loans.

The controversy focuses on this issue: Should real estate agents, lawyers and others be allowed to receive compensation in any form when they refer home buyers seeking loans to a particular lender? Should lenders be permitted to offer real estate agents cash, free trips to tropical islands or other inducements when they pass along a specified quota of new mortgage transactions? Should the largest mortgage originator in the country -- Citicorp -- be permitted to continue a fast-growing mortgage-referral program that sometimes involves fees to lawyers and loan brokers, paid directly by borrowers?

Questions such as these have split the Reagan Administration into rival camps, with the Department of Housing and Urban Development (HUD) on one side and the Office of Management and Budget (OMB) on the other. Sources close to the controversy say OMB has delayed for nearly nine months HUD regulations that OMB officials believe intrude too much into the operation of the free-market mortgage system.

HUD, by contrast, contends that its draft regulations merely fulfill the objectives of the Real Estate Settlement Procedures Act (RESPA). That's the federal law governing new and resale home-mortgage closings.

Though most American home buyers may not recognize the name, RESPA provides critical protections for them. The uniform settlement disclosure sheet used in residential closings, for instance, was mandated by RESPA. Prior to RESPA's passage in 1974, under-the-table kickbacks among title and escrow companies, lawyers, insurers and others were common, according to congressional testimony. RESPA prohibited anyone from giving or taking a fee, kickback or anything of value under an agreement that settlement-related business would be referred to a particular person or organization.

The central theory behind RESPA, according to its authors and supporters, is that referral fees and kickbacks on home transactions are particularly injurious to the consumer.

Warren Lasko, executive vice president of the Mortgage Bankers Association of America, says the spirit and intent of the law is to "allow the consumer to make a full, fair and free choice" among competing providers of services in the home purchase, financing and settlement process.

During the past two years, referral fees have made major comebacks, particularly in the increasingly tough market for home-mortgage originations. Lenders have begun offering real estate brokers and lawyers Hawaiian and Caribbean vacations, TV sets or cash payments for new loan customers steered in their direction. Other lenders have offered as much as half a percentage point in cash to brokers as part of aggressive efforts to maintain or increase their share of new mortgages.

Complicating the referral-fee issue have been two court decisions plus informal "opinion" letters from HUD. The two federal court rulings, one involving a Michigan mortgage banker, another involving ComFed Mortgage Co. of Massachusetts, held that mortgage lending is not a "real estate settlement service" under the terms of RESPA. The specific applicability of the federal antikickback provisions of the law to mortgage referrals has been thrown into doubt by the decisions, according to Jeremiah S. Buckley, a Washington lawyer.

One of Buckley's clients, Citicorp, plays a central role in the wider controversy over RESPA. In 1986, Buckley received an opinion letter from HUD's then-general counsel John J. Knapp, clearing a new Citicorp program called MortgagePower of any RESPA entanglements.

The program provides participating real estate brokers, lawyers and private mortgage brokers "fast-track" application processing, cut-rate "points" and other benefits for home-buying clients. Participating firms must pay a flat fee of $2,500 per year to Citicorp. In exchange for the financing privileges, the so-called "power brokers" may charge consumers for each loan.

Next: New kickback regulations or new legislation?