An Annapolis classmate of Jimmy Carter's was sworn in yesterday as chief of the Federal Railroad Adminstration, and said that if the administration fails to progress while he's in charge, the fault will be nobody's but his.

"There are some outstanding people in this agency," John Sullivan said, in a brief statement after taking the oath of office. "It will be nobody's fault but mine if we don't make any progress with the FRA."

Since 1967, the FRA has had primary responsibility for keeping the railirads safe, much as the Federal Aviation Administration oversees air safety. More recently, however, Congress has been funneling money for the improvement of the nation's railroads through the agency, making it even more of a government power to reckon with.

Sullivan has no railroad experience, a point he glossed over in stressing instead what he called the "diversith of my background." That background covers eight years with the Navy, including stints as a radar officer, aircraft carrier pilot and instructor, and 15 years as a businessman.

But aside from hard and a bit of luck. Sullivan hinted that what may have been most responsible for his appointment was a decision in January 1975 to campaign for Carter. Eventually. Sullivan became the campaign coordinator for Pennsylvania.

In welcoming Sullivan aboard yesterday. Brooks Adams the Secretary of Transportation, said Sullivan had been his personal choice as well as the President's Pointing to Sullivan's business experience. Adams said the new administrator would bring to the railroads the "perspective of a shipper," in addition to "particular organizational skills which we need at this time."

Listing briefly the number of railroad bills enacted recently to improve railroad transportation - the 3R Act of 1973 (Regional Railroad Reorganization), the 4R Act of 1976 (Railroad Revitalization and Regulatory Reform). Amtrak ConRail - Adams remarked. "HERE'S A lot of legislation that will tax the skills of the department."

Said Sullivan, in a prepared statement. "I will bring an independent business and user view to the department and feel I have enough knowledge of the important railroad legislation that you so carefully guided through Congress to work very aggressively on its implementation."

Attending the ceremony at the Department of Transportation building were a smattering of Navy officers in uniform - a rather unusual sight for the swearing in of a Transportation official. The wife of one former Navy man, also in attendance, said the reason for the royal welcome was to honor a classmate of that increasingly famous Annapolis Class of 1946.

"The last time this many of us got together was just last year, for the 38th reunion," she said.