The Tax Equity and Fiscal Responsibility Act--makes investments in questionable tax shelters far more dangerous gambles. In addition, breaks to be gained from shelters will be reduced or eliminated in some cases.

There are three basic elements in the law that will function to restrict the use of shelters:

* The most direct attack results from a significant expansion of the alternative minimum tax, a levy that falls upon high-income persons making extensive use of deductions and credits to reduce their tax liability. This is the type of taxpayer who tends to invest in tax shelters.

* The second involves the creation of severe penalties for those who substantially underestimate their tax liabilities without a "reasonable belief" that the claimed deductions are correct and for developers and promoters of abusive tax shelters. At the same time, the scope of Internal Revenue Service audits of shelters is expanded so that the examination can cover entire shelters, instead of individual taxpayers.

* The third involves a reduction in the value of the breaks resulting from investments in equipment, a common element in a tax shelter. Under old law, a taxpayer making an investment could, in most cases, get a 10 percent investment tax credit and depreciate 100 percent of the cost over, in most cases, five years. The new law, known as basis adjustment, makes the taxpayer choose between taking an 8 percent credit and full depreciation, or a 10 percent credit and depreciation on only 95 percent of the cost of the purchase.

The danger created by the expanded alternative minimum tax is that a person investing in a tax shelter has an increased chance of discovering that apparent tax reductions would be matched by tax liabilities under the new law.

In addition, credits that could be used in the past to reduce the old minimum tax will no longer be allowed against liabilities, except the foreign tax credit. "The taxpayer (who had invested in a shelter) may be very rudely surprised," said James L. George, manager of Ernst and Whinney's national tax office here in Washington. "These credits do not reduce your alternative minimum tax."

Along with eliminating the use of most credits to reduce alternative tax liability, the expanded provisions also turn a number of tax preferences used in tax shelters into liabilities for those who are subject to the minimum tax. The new preferences covered by the law include expensing--writing off in one year--mining development and exploration costs, research and development costs and magazine circulation expenditures.

The toughened penalties do not alter the economic attractiveness of legal tax shelters, but they do make investments in questionable propositions increasingly chancy.

Under old law, a taxpayer who substantially understated tax liability by claiming tax shelter losses that were subsequently disallowed was not subject to penalty as long as the claims were made on a "reasonable basis opinion."

The rough definition of such an opinion, according to Herbert J. Lerner, of Ernst and Whinney, was that when told to a room full of 10 tax professionals, "not everyone would chuckle."

The new law requires the taxpayer to have a reasonable belief that the claimed losses are "more than likely correct." In effect, this means that a tax shelter must pass a higher threshold test. This threshold is generally set when an outside professional assessment of a tax shelter is provided prospective investors.

Lerner suggested that one way to describe this higher requirement would be that in the hypothetical room of 10 tax professionals, six would have to agree that the claimed deductions are correct, while the other four would have to say the deductions are arguable.

The penalty for a taxpayer who substantially underestimates tax liability without such a reasonable belief in its legitimacy is 10 percent of the tax owed.

"For those who insist on playing variations of 'stick the IRS,' the law now creates much greater downside risks," IRS Commissioner Roscoe L. Egger Jr. said in a recent speech.

George, in a similar comment, said "the stakes in the audit lottery have gone up." The "audit lottery" is the practice of some taxpayers of purposefully underestimating taxes on the calculation that (1) they may not be audited, and, (2) if audited, the penalties are not severe. The new 10 percent penalty changes this equation.

In addition to the new penalty on taxpayers, the law creates a fine for persons who knowingly set up abusive tax shelters. In this case, the organizer of a shelter who, for example, makes a gross overvaluation of the cost of investments on which credits and depreciation are to be claimed is subject to a $1,000 penalty or 10 percent of the fees made. In the case of a tax shelter with 50 investors, this could amount to a $50,000 fine.

Eggers' IRS will also be able to conduct far broader audits of tax shelter cases through new procedures allowed under the law. These permit the consolidation of examinations of individuals using the same shelters into single audits of the shelter itself.

Some of the other provisions of the 1982 legislation likely to alter investment practices include:

* A 5 percent penalty on certain distributions made from insurance company annuities within 10 years, unless the policyholder is at least 59 1/2 years old, the distribution is made to a beneficiary after the death of the policyholder, the policyholder is disabled, or the distribution is allocable to an investment made before Aug. 14, 1982, the date of committee action.

* Dividends, along with interest payments, will be subject to 10 percent withholding, unless the recipient had, in the case of a joint return, tax liabilities in the prior year of $1,000 or less, or, in the case of the elderly, tax liabilities in the prior year of $2,500 or less for joint returns.