An investigation to determine whether Sen. Edward W. Brooke (R-Mass.) committed perjury during his divorce case was begun here yesterday by Middlesex County District Attorney John J. Droney.

He assigned Richard Kelley, assistant district attorney in charge of priority prosecutions for grand jury, to review all transcripts and depositions taken during the nearly 2 year old bitter divorce proceedings to see if Brooke made any "material misrepresentations."

Brooke has admitted that he has made "misstatements" in sworn testimony during the divorce proceedings.

Assistant District Attorney John Kerry said the investigation is meant to determine "whether or not the statute for perjury should or can be prosecuted."

A perjury convicted here carries maximum penalties of 20 years in prison and $1,000 in fines.

Brooke, who is running for reelection for a third term, was unavailable for comment yesterday on the investigation.

The possible perjury charges stem from Brooke's false testimony on a court deposition that he received a $49,000 personal loan from a friend, liquor distributor A. Raymond Tye.

The truth, the Senator later admitted, was that only $2,000 came from Tye and the remainder was money he was holding for his late mother-in-law - money he said he listed as a loan because he felt he had an obligation to pay back to her estate.

Meanwhile, Brooke's wife, Remigia, said yesterday, after a two-hour meeting with her attorney, George Ford, and her daughter, Remi, that she is still unsure about whether to proceed with a new divorce trial.

Under a court order issued by Middlesex County Probate Court Judge Lawrence T. Perera, Mrs. Brooke has until June 26 to make her decision. He stated, however, that the current settlement is "fair and reasonable," warned her that a new trial might not be better for her.

She is receiving $18,000 a year in alimony, and has been awarded homes in Newton, Mass., and the Caribbean island of St. Martin. Mrs. Brooke is seeking full payment of her legal fees and payment of about $80,000 in transfer taxes on St. Martin house, estimated to be worth about $300,000.

Brooke's attorney claims a larger settlement would "wipe him out" financially.

"The next move is up to daddy," said Remi Brooke, 28, the daughter, after the conference with lawyer Ford.

Brooke said at a news conference - called to explain inconsistencies in his financial statement - several weeks ago and that hsi daughter peddled information about him to reporters in an attempt to blackmail him into agreeing to a larger settlement.

Judge Perero had reopened the divorce case in light of Brooke's admission of misstatements to determine whether Brooke had misrepresented his financial condition to his wife, her attorney and the court.

In his decision Thursday, the judge said Brooke's testimony in the deposition "was false and could reasonably have misled" Mrs. Brooke.

However, Perera ruled against initiating a civil prosecution against Brooke's Democratic challenger in the ring the matter instead to District Attorney Droney, who had been Brooke's Democratic challenger in the 1972 Senate race here.

The state tax department is also auditing the senator's returns. And in Washington the Senate Ethics Committee is reviewing Brooke's financial disclosure statement for his failing to list the personal loans, an apparent violation of Senate rules which require the listing of all loans over $2,500!

The state perjury statute reads: "Whoever, being lawfully required to depose the truth in a judicial peoceeding or in a course of justice, willfully swears or affirms falsely in a matter material to the issue or point in question, or whoever, being required by law to take an oath, or affirmation, willfully swears or affirms falsely in a matter relative to which such oath or affirmation is required, shall be guilty of perjury.