New York City was rebuffed yesterday in its first attempt to borrow money on its own again in the major credit markets - meaning it will remain dependent on federal bailout efforts for at least several more months.

Plans to offer $200 million in short-term city notes were canceled abruptly late yesterday after the city received an investment rating that underwriters said was too low to attract any buyers. The offering was to have been held next Thursday.

The action marked a major psychological setback to the city, which had re-entered the market earlier than expected in part to satisfy demands from Congress that it show its good faith in trying to get back on its feet. Analysts speculated the city will not be able to try again to borrow until sometime next spring.

Immediately after the underwriters made their decision, the city's comptroller, Harrison J. Goldin, said New York would "continue to rely on our current sources" of credit, and would seek help from the state legislature to strengthen its financial position.

Goldin said that despite the cancellation yesterday, "There is no emergency, because our credit needs for the fiscal year are provided for adequately." The city still may borrow up to $825 million from the federal government between now and June 30.

The cancellation was sparked by a decision by Moody's Investor's Service, one of the nation's two largest investment-rating firms, to assign the city's notes a rating of "Municipal Investment Grade 4" - the lowest of four ratings available.

Although the MIG - 4 rating technically was adequate to allow the sale to proceed, advisers at Merrill Lynch, the city's investment underwriter, told Goldin there would be "no possibility for successful offering" at this time.

The sale would have marked the first time in 2 1/2 years that the city entered the credit markets. Moody's suspended its ratings on New York City notes in 1975, after it appeared that bankruptcy was imminent. Before that, New York enjoyed ratings of MIG-1 and MIG-2.

Jack Phillips, executive vice president of Moody's, offered two reasons yesterday for the low rating the firm assigned the city:

The city has been too slow in paying back the notes it issued previously. Many of its obligations have remained unpaid since July, 1975.

The city's budget still is not in balance - a situation Phillips said provides "enough uncertainty that this is not a very good security for an investor to buy."

A spokesman for Goldin said the underwriters also were concerned that there was not enough "oversight" by the state in managing the city's financial affairs.

During its turnaround in recent months, the city's financial affairs have been supervised by the state-run Municipal Assistance Corp. However, Goldin has been pressing for a series of management changes that he says would provide long-run stability for New York.

The rating by Moody's was a surprise to city officials. A spokesman for Goldin's office said the comptroller had been expecting that the city would receive a rating of MIG-3, which he said would have been good enough to go to market without any reservations.

Felix G. Rohatyn, the city's top fiscal adviser during the 1975-76 crisis, said the rebuff yesterday means the federal bailout credit line "has to be renewed by Congress." The $825 million still available is the last of a credit line that originally totaled $2.3 billion.

But the city had not borrowed as much as it had thought it would need from that total, and the new offering would have been an attempt to reduce what is already owed.

Rohatyn and other New York City fiscal experts said the failure to issue the new notes will have no effect on the budget or cash flow of the city.

"After labor negotiations in the spring and after the budget is balanced the city will be able to go back to Moody's and get to better rating," Rohatyn said.

He said the estimated budget gap for New York next year is between $400 million and $500 million.