As agencies begin to count the cost of complying with President Reagan's latest budget-cutting proposals, there are growing complaints that they will be unable to perform their required duties if they are forced to reduce outlays as much as the president wants.

One in five employes in the Securities and Exchange Commission would have to leave, the SEC told Budget Director David A. Stockman this week. This would reduce by 15 percent the number of enforcement cases that it can bring against illegal corporate activity, the watchdog agency says. It would also be forced to close some pending investigations and to stop regular reviews of investment companies and other financial offices, it says.

Coast Guard Commandant John B. Hayes told a House subcommittee that if his agency were cut a further 12 percent below Reagan's March budget request it would have to halt its drive against drug smugglers, although he added that he expected the Coast Guard to be spared the full cut after negotiations between Transportation Secretary Drew Lewis and Stockman.

Agencies are due to submit proposals for meeting the new cuts to the Office of Management and Budget this week. The government will then issue deferral notices to legalize immediate implementation of the cuts, even before Congress approves them.

The administration is also searching for cuts in entitlement programs, which it originally hoped to send to Congress this week but has delayed to next week.

One of the cuts in basic benefits being considered, sources said yesterday, is a 5 percent cap on growth of the long-term nursing home care component of Medicaid. More than 40 percent of the fast-growing federal and state Medicaid outlay now goes for long-term nursing care.

Reagan may also propose shifting power to the states to determine eligibility standards for food stamps for welfare recipients. These account for about half of the 22 million people on food stamps. G. William Hoagland, administrator of the Agriculture Department's Food and Nutrition Service, said that "a minimum federal standard" should remain, but the present national guideline which is fully funded by the federal government would end if the administration and Congress go ahead with this proposal.

Many details of how agencies might implement the new cuts remain sketchy, but some were revealed yesterday:

The Federal Communications Commission has concluded it would have to fire 375 employes out of more than 1,900 to comply with the president's request.

The Federal Trade Commission has agreed to accept the cuts without appeal but has not yet worked out where it will make them.

Many other agencies, such as the Consumer Product Safety Commission, that say they have to get rid of some employes are considering ordering furloughs of two to three weeks for almost all employes to reduce the overall number of Reductions In Force.

The Labor Department has told employes that a total of 2,460 people may have to be fired this year, many by Dec. 31, a source in the department said. Of these, 700 would come from the Mine Safety and Training Administration and 250 from the Occupational Safety and Health Administration. Part of the total stems from the original March budget cuts and part from the latest round, an official said.

The Drug Enforcement Administration, which is appealing the order for further cuts, has concluded that it may have to eliminate preventive drug abuse information and significantly curtail programs that teach state and local police officers how to investigate narcotics.

The Equal Employment Opportunity Commission says the latest round of cuts will prevent it from processing discrimination complaints without unreasonable delays and will greatly increase its already heavy backlog, which has been criticized by Congress and business.