Conservatives are calling on the Richmond Federal Reserve Bank to remove the rainbow flag flying outside its building representing gay rights in conjunction with Gay Pride month.

Del. Robert G. Marshall (R-Prince William), one of the most conservative members of the General Assembly, wrote a letter to Jeffrey M. Lacker, president of the Richmond Federal Reserve, urging him to take down the flag.

He wrote that homosexuality “adds significantly to illness, increases health costs, promotes venereal diseases, and worsens the population imbalance relating to the number of workers supporting the beneficiaries of America’s Social Security and Medicare programs.”

“I do not believe that a celebration of ‘gay pride’ has anything to do with the mission of the Federal Reserve under the Federal Reserve Act passed by Congress,” Marshall said. “This is a celebration of a behavior that is still a class six felony in Virginia. How can the American people trust the judgement of the Federal Reserve as an institution when its spokesperson celebrates an attack on public morals?”

The flag, which is next to an American flag, is being flown at the request of PRISM, one of the Federal Reserve’s employee groups representing gay, lesbian, bisexual and transgender people.

“The flag is an example of our bank’s commitment to diversity,’’ Federal Reserve spokesman Jim Strader said.

Strader said the bank has received comments from people both for and against the flag, and from those who mistakenly think the Federal Reserve is a government entity. Del. Riley E. Ingram (R-Hopewell) also contacted the bank in opposition, Strader said.

Victoria Cobb, president of the conservative Family Foundation, mentioned the flag in an an e-mail earlier this week.

“At the Family Foundation, we will simply choose to use this flag, like the view of Mr. Jefferson’s Capitol, as motivation for the work that lies ahead.”

James Parrish, executive director of the gay rights group Equality Virginia, released a statement Friday congratulating the bank.

“The Federal Reserve Bank of Richmond should receive accolades for its decision to recognize and celebrate its GLBT employees, customers and vendors during pride month,’’ he said. “It’s a private business and should be able to make its own personnel and corporate policy decisions without Bob Marshall’s guidance or the Family Foundation’s approval.”