Eighteen Western European countries pledged today to expand cooperation in advanced technologies in order to compete more effectively with the United States and Japan, but they failed to reach a consensus on how to manage and finance the program.

Foreign and research ministers from the participating nations approved a charter of principles and began selecting projects to be encouraged in the civilian research program, known as Eureka, that was initiated six months ago by France.

The French concept was spurred largely by President Reagan's multibillion-dollar "Star Wars" research plan into space based antimissile defenses. European nations, fearing that such a massive U.S. project would lure away their best scientists and overwhelm their technological capacities, decided in Paris last July to press ahead with their collaborative research program in nonmilitary applications of technologies.

Its main purpose is to enable European industry to stay abreast of developments in many of the sophisticated economic sectors that are to benefit from the infusion of scientific talent and government money under Reagan's Strategic Defense Initiative.

West German Chancellor Helmut Kohl, who opened the two-day conference here, warned that "only consistent and purposeful cooperation will enable Europe to hold its own in the triangular relationship with the United States and Japan."

While the political commitment remains strong among European leaders for more dynamic cooperation and new technologies, Eureka has made little progress beyond the conceptual stage because of divergent views about the nature of the program, how it should be funded and who will run it.

The Eureka schemes are intended to stimulate cross-border collaboration among European companies in such fields as telecommunications, robotics, biotechnology and lasers. By pooling their resources, the Europeans then would be able to match the strengths of American and Japanese multinationals. Governments have promised to earmark funds for Eureka and to lower national barriers to trans-European projects.

West Germany has called for long-term research projects, such as seeking new means to measure environmental pollution, to be the central focus of the Eureka program. But Britain and France want greater emphasis on pragmatic, money-making projects that could compete in the world market with American and Japanese giants such as IBM and Sony.

Government funding also poses a problem. Only France has put up any money. The Socialist government of Francois Mitterrand has set aside $125 million from its current research budget to subsidize the Eureka program.

Britain has resisted offering any public funds because it wants private companies to invest most of the seed money in Eureka projects. Other countries have pleaded that they have no money available.

Another conflict has arisen over how Eureka should be administered. Britain, France and West Germany want a lean and flexible operation that will not evolve into a sprawling international bureaucracy. But Italy and the smaller countries insist on a Eureka secretariat to represent all member states, so they would not be crowded out.