The Rouse Co., a Columbia real estate management and development firm, yesterday reported a substantial gain in operating earnings for the quarter and nine months ended Sept. 30.

Not counting one-time gains from sales of interests in shopping centers, Rouse posted third-quarter earnings of $2.1 million compared with $1.4 million last year. The figures do not reflect non-cash charges (depreciation and amortization of property, deferred income taxes or gains and losses from translation of foreign debts into American money).

Rouse provides per-share earnings data only after deducting dividend payments on preferred stock. On this basis, Rouse earned 10 cents a share in the recent quarter compared with a 2-cent loss in the 1976 period.

Including non-cash charges and one-time gains, Rouse profits totaled $1.43 million vs. a loss of $247,000 for third quarter. Revenues rose to $21.4 million from $19.3 million.

For the first nine months, Rouse operating earnings were $6.2 million vs. $2.2 million. Net income declined to $4.3 million from $6.7 million, reflecting a one-time gain from property sales of $8.9 million in the year-earlier period. Sales rose to $62 million from $57 million.

Rouse spokesmen said all operating divisions posted improved results in the 1977 period, with "particularly strong" gains from its 28 retail shopping centers and its mortgage banking subsidiary, James W. Rouse & Co.

Three new developments opened in the third quarter - Hulen Mall in Ft. Worth, a joint venture with Federated Stores Realty; a center in downtown Philadelphia; and the second phase of Faneuil Hall Marketplace in Boston. Six retail centers are scheduled to open next year in Ohio, Georgia, New Jersey and Pennsylvania.