Two of the nation's major retail chains, two credit card firms and a petroleum company yesterday were ordered by the federal government to refund more than $3 million to their customers.

Following an investigation into complaints that the five companies were using for their own purpose funds of consumers who overpaid bills, the Federal Trade Commission obtained consent agreements designed to eliminate alleged unfair charge account procedures and to require refunds of existing over-payments exceeding $1 during the past three years.

Virtually identical consent agreements with the FTC, to settle complaints without admission that any laws had been violated were reached with the following companies:

Federated Department Stores, Inc., of Cincinnati, the nation's largest department store firm, for alleged overpayments of consumers at such divisions as Bloomingdale's, the fashion-oriented New York retailer; Abraham & Straus of Brooklyn; Filene's of Boston; and I. Magnin on the West Coast. About a dozen other Federated divisions, in various cities, were cited in the FTC investigation.

City Stores Co., of New York, which owns the W&J Sloane furniture stores, Franklin Simon specialty stores and eight other regional retail firms.

Diners' Club, Inc., of New York, and Carte Blunche Corp., of Los Angeles.

Atlantic Richfield Co., of Los Angeles, whose retail outlets are operated under the Arco name.

Typically, according to the FTC investigation, the companies cited have had credit operations that occasionally showed money owed to consumers because of inadvertent overpayments of bills or credits given for the purchase price of returned merchandise.

Often, consumers were informed of the credit balance on their next bill. But the second such notice of a credit typically was not mailed until six months later and no other notices were sent, unless additional purchases were made.

"At no time," said the FTC of Federated's divisions, were cash refunds sent without request. In the case of Arco, the agency said consumers never were informed that they were entitled to request and receive such a refund.

Through such alleged practices, the five companies retained in their posession "substantial dollar amounts" belonging to their customers, according to the FTC. An exact amount of such funds could not be determined but refunds ordered yesterday should total at least $3.5 million, the FTC said. The number of individuals involved is not known.

Four other department store companies were cited by the FTC for similar credit balance practices in 1975.

Under yesterday's consent agreements, the five companies agreed to provide consumers with periodic statements of credit balances, notify each consumer of the right to request a refund in cash and automatically send refunds after a prescribed period of inactivity in the consumer's account.