The Naval Investigative Service has begun an investigation of Marine Corps procedures in checking out charges of an alleged security lapse among President Reagan's Marine helicopter guards during an October visit to Mexico.

A White House spokesman, David Prosper, said the investigation was begun Sunday after members of the elite Marine helicopter-security unit accused their superiors of failure to fully investigate the incident.

Marine guards told The Washington Post that they observed two of their unit's sergeants, including the noncommissioned officer in charge of security on the Cancun, Mexico, trip, displaying the president's itinerary and talking openly about the schedule while drinking with two women in a Mexican disco.

The witnesses said they reported the incident to their superiors but believed that it was "shoved under the rug" before the reports got past the sergeants, and that unit morale had suffered. Later, an anonymous letter outlining the incident was written to Lt. Col. Paul Johnston, commanding officer of the unit, known as HMX-1.

Johnston said that both Marine and White House military-office probes concluded that no serious security violations were committed.

Sources said the naval investigative unit began taking formal statements from about a dozen members of the unit, and warning the Marines of their legal rights, Sunday morning shortly after the article appeared in The Post. The Navy investigation is examining only Marine procedures, not White House military-office procedures, Prosper said.

A White House aide said Sunday that no further action was contemplated against the marines involved.