Republican state attorneys general on Tuesday weighed in strongly against remarks from U.S. Attorney General Eric H. Holder Jr., in which Holder suggested the states' top cops could decline to defend gay marriage bans.

Holder said Monday that state attorneys general are not obligated to defend the gay marriage bans if the bans discriminate in an unconstitutional way.

The Republican Attorneys General Association said that's the wrong message from the Obama administration. RAGA Chairman Alan Wilson of South Carolina compared the administration's picking and choosing of which laws to enforce to its last-minute changes to Obamacare.

"Our freedom depends on upholding the rule of law and obtaining the consent of the governed," Wilson said. "Republican attorneys general will continue to fight every single day to protect our Constitution and defend states’ rights.”

Added Alabama Attorney General Luther Strange: "A state attorney general has a solemn duty to the state and its people to defend state laws and constitutional provisions against challenge under federal law. To refuse to do so because of personal policy preferences or political pressure erodes the rule of law on which all of our freedoms are founded."

Montana Attorney General said Holder's comments amount to "a gratuitously offered non-binding legal opinion on an issue that has not been decided by a national court of competent jurisdiction at this time."

The Justice Department, under Holder, previously declined to defend the federal Defense of Marriage Act.

Holder said Monday that such decisions "must be exceedingly rare" but that they are warranted in certain circumstances.