At a time when the administration and Congress are trying to close a budget gap that grows with every new estimate, the president is simultaneously moving in the opposite direction and proposing a series of new tax expenditures that would increase the projected deficit.

These expenditures reflect the fundamental philosophical premise of a conservative president that it is better to use the tax system than direct-spending programs for social goals such as aid to education, urban development, Third World growth and job training.

A second conservative theory behind these tax programs is that they are designed to use the market system to encourage the hiring of the disadvantaged and increase investment in beleaguered urban areas at a time when direct spending in those areas is being cut severely by the Reagan administration.

But the proposals are irritating friends and foes on Capitol Hill who see the measures as creating election-year focal points for special-interest lobbies seeking additional federal money.

Referring to the most recent Reagan proposal, tuition tax credits, Sen. Ernest F. Hollings (D-S.C.), ranking Democrat on the Senate Budget Committee, said the credits for parents of students in private schools is "not just unconstitutional, uneconomical and unfair, but at this point unconscionable."

A Republican with a large Catholic constituency who has been sympathetic to the concept of tuition tax credits, Barber B. Conable Jr. (R-N.Y.) commented with characteristic understatement that "now is not the time" to propose to Congress programs that will cost tax dollars.

The proposed tax reduction for tuition tax credits, targeted jobs credits, the Caribbean initiative and urban enterprise zones follow passage last year of the largest tax cut in history, $749 billion through 1986.

That tax cut, in addition to the basic individual rate reductions and stepped-up depreciation schedules for business, created eight new tax expenditures ranging from the "All Savers" certificates to breaks for truckers hurt by deregulation.

On top of that, the tax bill expanded 11 other existing tax expenditures, including a major reduction in the tax of foreign income, expansion of Individual Retirement Accounts and broadened employe stock ownership plans.

Tax expenditures are becoming an increasingly controversial element of federal budgeting. In contrast to direct-spending programs such as those for housing, food stamps or defense, there is no annual congressional review required by the budget and appropriation process.

Disregarding the new administration proposals, tax expenditures are expected to grow by leaps and bounds over the coming years. From 1982 to 1987, expenditures benefiting corporations are expected to increase by 121 percent, from $55.1 billion to $122 billion. For individual taxpayers they will grow by 60 percent from $198 billion to $317 billion.

Depending on the viewpoint, tax expenditures either are loopholes that give benefits to special interests and distort investment decisions, or they are essential for the encouragement of investment in key sectors of the economy.

Many tax reformers contend that the favorable treatment of capital gains income--profits from the sale of assets held one year or more--gives the wealthy an unjustifiable break on income, while proponents contend that the reduced rate--a 20 percent maximum compared with 50 percent top rate on ordinary income--is necesssary to encourage investment and saving.

Recently, a number of conservatives, including Sens. Pete V. Domenici (R-N.M.), chairman of the Budget Committee, and Nancy Kassebaum (R-Kan.), have joined reformers in calling for tighter control of tax expenditures, prompted by concerns that this section of federal "spending" has grown faster than any other.

This concern has not been voiced by the administration, which, if anything, questions the basic notion of such a thing as tax expenditures. The Treasury recently objected to the concept of tax expenditure on the grounds "that it seems to imply that the government has control over all resources. If revenues that are not collected due to 'special' tax provisions represent 'expenditures,' why not consider all tax rates below 100 percent 'special'?"

In this context, Norman Ture, undersecretary of the Treasury, supported the tax breaks contained in the administration's urban enterprise zone plan on the grounds that they will create "a significant free-market opportunity to revitalize economically depressed areas by employing federal tax incentives while minimizing federal expenditures."

That statement contains at least two key, highly controversial assumptions: that tax breaks are not federal expenditures and that creation of special tax breaks restricted to specific areas--a fundamental element of the legislation--does not distort the market but, instead, serves as a "free-market opportunity."

The four new administration programs involving tax expenditures are:

* Tuition tax credits. Parents of students in parochial and private schools would be eligible for phased-in tax credits, starting at $100 next year, growing to $300 in 1984 and to $500 in 1985.

The credit would be based on 50 percent of the cost of tuition, so that in 1985, tuition would have to be $1,000 or more to qualify for the full $500 credit. The full credit would be available to families with adjusted gross incomes up to $50,000 and would be slowly reduced at incomes above that. Once income reached $75,000, no credit would be available.

The administration estimates that the tuition tax program would cost $100 million in 1983 but that as it is phased in, the cost would grow to $1.5 billion by 1987, for a total five-year cost of $4.6 billion.

* The creation of urban enterprise zones, where employes and employers would be the beneficiaries of a wide range of tax reductions, credits and other benefits. The administration estimates the cost at $100 million in 1983, rising to about $1.3 billion by 1987, or a five-year cost of $3.6 billion.

The Joint Committee on Taxation has warned, however, that the way the legislation is written, there is no restriction on the size of each enterprise zone, and the program could cost five times the administration's estimates. If so, that could push the cost to $18 billion over five years.

Supporting this view, Robert Lighthizer, chief counsel on the Senate Finance Committee, warned that the pressure to create enterprise zones in every conceivable city will be tremendous if they are enacted, and that it will be compounded by pressure from other sections of the country to create "suburban" and "rural" enterprise zones.

The administration bill provides an extraordinary host of tax breaks for businesses and, to a lesser extent, their employes, within a designated enterprise zone.

These breaks include a 10 percent employer credit for wages paid qualified employes, up to a ceiling of $1,500 for the credit; a 50 percent tax credit for wages paid to disadvantaged employes; a 5 percent credit on income up to a ceiling of $450 a year for employes living in the zone; and elimination of capital gains taxes on sale of property.

* Caribbean initiative. As part of the economic program for friendly Caribbean countries, the administration would create new tax benefits for U.S. firms and investors active in the area.

American business could qualify for a 10 percent credit on investments in countries willing to sign agreements with the United States for the exchange of tax information. In addition, shareholders holding 5 percent or more of firms incorporated in Caribbean countries would receive a pro-rated shared of investment tax credits.

The legislation also would extend the investment tax credit and the new depreciation tax breaks in the 1981 Economic Recovery Tax Act to subsidiaries of U.S. companies in Puerto Rico and the Virgin Islands. The total cost of the package would be $105 million in 1983.

* Targeted Jobs Tax Credit. The administration has not given details of its proposal, but Labor Secretary Raymond Donovan has told Congress that he intends to press for extension and alteration of the existing jobs credit program, designed to encourage businesses to hire persons on welfare, those thrown off federal jobs programs, disadvantaged Vietnam War veterans and others.

As currently designed, the program gives employers a 50 percent credit on the first $6,000 paid a qualified worker and, in the second year, a 25 percent credit, or a $3,000 credit the first year and $1,500 the second.

This program currently costs between $250 million to $300 million a year, although congressional critics have argued that there is no solid evidence that it has acted as an incentive to companies to hire the poor. Instead, these critics have charged that employers continue their normal practices, and just collect the credit for those employes hired who happen to meet the requirements for qualification.