Herein, we shall define evaluation as the opinion that one or more experts provide to whoever has the power to make a decision so that this decision-maker can choose a course of action with full knowledge of the facts. Quite clearly, evaluation has a decision-making finality. An example thereof is the judge who adopts a position after reading a report by experts. We might point out that such expertise in the courts is the only instance, apart from the pharmaceutical industry, of evaluation regulated by law.

Evaluation is spreading in all fields. As we shall see, all forms of evaluation, whether for judicial, administrative or political decision-making, have several points in common. Common principles govern these procedures. To cite but one for now, evaluation always takes the form of a recommendation. In no case do the expert's conclusions bind the decision-maker: they are nothing more than an opinion, i.e., advice. By studying various types of expertise, we can highlight the major common principles. Once this is done, we shall take a look at the legal questions related to evaluation procedures.

Part One: Experiences with evaluation

I. Institut National de la Santé et de la Recherche Médicale

"to be capable of setting up groups of experts for analyzing the state of science in given fields at the request of its natural partners or on its own initiative"[JE98].

This assignment has led INSERM to adopt its own procedures for making evaluations.

During the preliminary phase, a group of experts is assigned to evaluate a specific field. The objective during this phase is to clearly define the field and specify all underlying questions. A dialogue should take place; and information, be exchanged between the experts and the "partner" requesting the expertise. Between these two parties, a "coordinator" is assigned to "translate the question(s) being asked by the partner into a scientific problem that will be acceptable for a group of experts". [JE98].

This phase ends with the signature of an agreement specifying the terms and conditions for the evaluation.

The evaluation phase itself then takes place. The head of "collective expertise" in INSERM appoints a group of experts, usually from ten to twelve persons. Clearly defined criteria guide the selection of these experts: each member is to be independent (i.e., have no interests in the evaluation) and qualified (as indicated mainly by the number of publications about the subject to be investigated). One of these experts is appointed president. INSERM's CEO [directeur général] must also approve the group's composition.

The group of experts can then go to work. It starts by collecting documents on the question to be evaluated. In scientific matters, this phase is especially important. After five or six meetings on the average, a synopsis is made. All this leads up to writing a final report.

Some French scholars have suggested that, in general, work group members should, in the final report, be able to note their own conclusions when these differ from the majority's. At present, this is not the case, but the idea seems worth considering. It would have repercussions in terms of liability. For example: a group member may, by standing back from the majority opinion, clear himself from any liability if the experts' opinion causes a tort.

The last phase of evaluation is the circulation of the findings.

The agreement sets the rules for doing this. In particular, it specifies whether or not the report is to be published. Confidentiality usually prevails.

At INSERM, there is a boom in evaluation. Parties, public as well as private, regularly submit questions to this institute for evaluation.

II. Institut National de la Consommation

To provide consumers with objective information for making their purchases, comparative tests are run more on the qualities than the defects of products; and the results are published in INC's magazine, 60 Millions de Consommateurs. A right of reply is now recognized to the manufacturers whose products are tested.

Test are performed at INC's testing center. Personnel at the center buy the products anonymously in ordinary stores. After the tests but before publication of the results, manufacturers are notified.

There are no specific regulations, at either the national or EU level, for comparatively testing goods and services. This situation prevails in other European lands. However, INC does follow evident methodological and deontological rules throughout the evaluation process.

First of all, there are standards. In France, AFNOR [Association Franáaise de Normalisation: the French Standards Association] has set standards. And at the international level, ISO [International Standards Organization] has drawn up a manual for comparatively testing consumer products and services. The whole profession represented by Consumers International [CI], which federates consumer associations at the international level accepts and applies these standards. CI published a guidebook on comparative testing in July 1985.

This code of conduct, or set of "deontological rules", guarantees comparative testing's objectivity. They are useful for our study, since they indicate ways to offer parties the best possible guarantees. In 1987, INC set up, along the lines of the German model, an authority under its board of directors to play an advisory role about applying the principles of comparative testing.

A methodology for the whole testing procedure turned out to be indispensable. These tests, it should be pointed out, are not performed under contracts signed between INC and companies. Common law provisions about liability apply if conflict arises among these parties. The grounds of a suit do lie in contractual liability.

The full procedure lasts from six to nine months. It starts with a marketing study in order to obtain samples. Once the samples are bought, the testing center runs the tests. INC then notifies each manufacturer of the results for his products, and for them alone. Obviously, INC does not communicate information on rival products during this phase. The testing center and manufacturer then exchange information about the product. This exchange is quite formal, via registered letters with acknowledgment of delivery, official documents, etc. The testing center then makes a definitive decision. This decision usually stays the same as the one following the tests. Unless technicians committed errors, the exchange of information with the manufacturer does not reverse INC's opinion. The testing center is alone responsible for its decisions.

The use made of tests results (e.g., by manufacturers for advertising purposes) is regulated by an AFNOR standard. Such a use is possible in France under a standard procedure authorized by INC, but it seldom happens. The use of test results raises problems for all forms of evaluation. Manufacturers are tempted to advertize the classification of their products and test results. This causes problems when the classification should remain confidential or when the persons performing the evaluation do not want the results of their work to be used for commercial purposes. Furthermore, the manufacturer should not overlook the question of authors' rights to published tests. These rights are vested in the evaluating party. The solution is simple: the evaluated party whose products or services are tested must obtain (even more so if the results have already been published) permission from the evaluating party. In case of publication, the latter has a copyright.

INC may, of course, be held liable. The manufacturer may ask for a provisional decision forbidding publication. For obvious reasons, this has hardly ever happened. A manufacturer may also challenge the results of tests run on his product. This sets off a battle between experts. However, INC has seldom been sued. For that to happen, it would have to commit a breach of obligation (for example, by favoring one brand over another) or a technical mistake during the testing.

III. Agence du Médicament

Unlike the other experiences, evaluation at the Agence du Médicament is strictly regulated at the national, EU and even international levels. This holds for all European lands. These regulations ensue from obvious reasons related to public health.

The French system tends to resort to experts from outside the Agency. This does not always happen in other countries, where in-house experts may be used. The future will undoubtedly be a mixed system.

According to Professor Alexander, the director of evaluation at the Agency, the selection of experts follows traditional principles: qualifications, independence and availability (The best experts are seldom available). The Public Health Code Article n¯R.5141-4 imposes an obligation of confidentiality on committee members, experts and rapporteurs. This secrecy extends to the committee hearings and deliberations.

Furthermore, Article n^oR.5141-2 stipulates that the members of the committee that authorizes marketing "may neither be wage-earners nor have a direct or indirect financial interest" relating them to the evaluated party. After appointment, each expert has to issue a statement of his interests. Independence is absolutely indispensable for an evaluation procedure that recruits all experts from outside the evaluating organization. To cite an example, Article 4 of the 25 January 1995 decision creating a work group on drugs against cancer at the Agence du Médicament stipulates: "work group members, when appointed, have to send to the Agency's CEO a declaration mentioning the direct or indirect bonds they may have with the firms whose products might be examined by the work group.

They pledge to notify any modification in these bonds. Work group members may not take part in the group's work if they have a direct or indirect bond with the affair under examination." And Article 3 states that "the work group's proceedings are confidential." A 13 April 1981 decree provides for committee experts and rapporteurs to be paid for any report made for the Agence du Médicament.

Several phases can be pointed out in the procedure followed at the Agency. The indispensable first one is to compile data. Otherwise, any claim to objectivity would surely be lost. Next is the technical and scientific evaluation itself, performed by an independent committee of experts. Information is exchanged. The manufacturer must, in particular, provide information on the side-effects of the drug that he wants to place on the market. The manufacturer's representative must answer questions during hearings. However, the committee makes its decision without the manufacturer being present.

The criteria for evaluating the drug are set ahead of time in regulations. Scientific advances may eventually modify them. Manufacturers have full knowledge of these criteria, and thus of the requirements for obtaining permission to market a drug.

Under this system of evaluation, independent experts work in an objective way. But persons from outside the Agency verify that the system works right this suggests a parallel with the advisory authority set up at INC. An Agency task force deals with deontological matters. The magistrate at its head sees to it that the general principles for any system of evaluation are respected.

Evaluation reports are published. According to Professor Alexander, committee hearings will, in the future, very likely be held in public. The committee's decision is not unanimous. A relative solidarity exists among committee members in the quest for the best solution, but in no case is the decision a compromise. Since a consensus does not exist, it is impossible to sue one or more committee members. Furthermore, the committee has a merely advisory role.

The final decision is made by the Agency's CEO, who is not bound by the advisory opinion in the committee's report. According to Prof.

Alexandre, the CEO normally follows this opinion. Not following it expresses wariness of the quality of the expertise, the determination to take other (for example, economic) factors into account, the unwillingness to take risks, etc. Besides, other criteria than those (mainly scientific ones) established for the evaluation may weigh in the final decision and thus explain this wariness. The CEO is personally responsible for the evaluation procedure. His decision may be appealed. When permission to market a drug is refused, a list of questions about possible risks related to the product is sent to the manufacturer. After receiving the answers, the CEO issues a final decision.

A similar evaluation procedure exists at the EU level. The pharmaceutical specialty committee at the European Agency for the Evaluation of Medical Products [EAEMP] appoints two of its members as rapporteurs. For the evaluation, these rapporteurs rely on a team of experts chosen from a list containing the names of the experts on whom authorities in each member state rely.

Here too, the experts sign a "statement of interests" and are bound by confidentiality. On 4 December 1996, EAEMP's board of directors adopted a code of conduct for its partnership with national authorities. Under Article 7, committee members and experts will act for and in the name of the Agency. They will have no financial or other interests in the pharmaceutical industry that may impair their impartiality. To this end, they sign a public statement of interests. And under Article 8, the rapporteur, corapporteur and experts pledge to not use or disclose to third parties (other than competent national authorities, the Agency's executive or his representatives), any fact, information or document brought to their knowledge while accomplishing their assignment. They will be bound by this obligation after their assignment ends and until the forementioned information is made public.

The EU and French procedures are quite similar: tightly regulated and very strict. This makes them different from other evaluation procedures.

IV. Legal expertise

The function of legal expertise is to provide a judge with an opinion. The judge is free to not take it into account. Under Article 246 of the new Code of Civil Procedure [NCPC], "the judge is not bound by the technician's findings or conclusions". For evaluation in the field of linguistic engineering, this question is left hanging. Is the party requesting expert testimony bound by the findings of a group of experts? This very point is a stumbling block, since the question of liability depends on it.

Under Article 232 of the new Code of Civil Procedure [NCPC], the selection of experts is up to the judge:

"the judge may swear in the person of his choosing in order to shed light [ ] on a factual question requiring a technician's advice".

According to Article 233 however, the expert is "empowered by the judge on the basis of his qualifications"; and he "has to personally fulfill the assignment with which he has been entrusted". This assignment must be carefully defined beforehand.

The expert, once he accepts an assignment from a judge, must accomplish it. He "has to give his advice on the points for which he was sworn in to examine" [Article 238]. Above all, he "has to accomplish his assignment with conscientiousness, objectivity and impartiality" [Article 237]. This highly important requirement crops up in all forms of evaluation. As several examples from the jurisprudence prove, experts can be held liable for not holding up to this obligation. Furthermore, experts have to perform their duties "within the time allowed". Once again, they may be held liable if they fail to do so. Finally, expert must, in accomplishing their assignment, hold to the adversary principle.

Each of the other EU member states has an equivalent institution.

Since traditions and differences in the judicial system may lead to variations, let us look at a second example. In the United Kingdom, experts have an especially important role [MAY90]. The rule is that no other witness can be called in scientific matters than scientific experts (This principle dates back to 1782 when it was expressed for the first time in the Lord Mansfield case in Folkes v. Chadd). This rule's objective is for the court to be provided with information that normally lies outside the scope of the judge's and jury's qualifications. Hence, the decision whether expert testimony is necessary or not lies with the court. Likewise, the court decides whether or not a witness is qualified to give scientific advice. The witness's education, experience, etc., enter into account. An expert will usually be asked to deliver a report on a specific question to the judge. Under no circumstances may the expert comment the final sentence. He merely helps the judge and jury. This is a constant in all forms of expertise: the expert helps to make a decision but does not himself decide. It is up to the jury to decide how much weight to lend to his report. The rule is the following: if nothing contradicts the expert's advice, the jury has to accept it.

V. Synopsis of evaluation experiences

Part two: Major legal aspects of evaluation

I. The legal framework of evaluation

I.A. Defining evaluation in legal terms

I.A.1. Legal notions distinct from the evaluation to be performed in linguistic engineering

I.A.1.a. Certification[PONT96]

"any marking, distinctive sign or attached document attesting, for commercial purposes, that industrial products or capital goods present certain specific characteristics controlled by an organization distinct from the manufacturer or seller."

Certification involves a control by an organization that, at the time of the Scrivener Act, had to be approved by the Ministry of Industry. The certificate is delivered only after control by an independent organization of this sort. A 3 June 1994 act [JORF94] modified this procedure, in particular by abolishing the ministerial approval of certifying organizations. The Consumer Code [CC] article L.115-28 states:

"the only organizations that may certify products or services are those that have filed with the administrative authority a declaration concerning their activity and containing, in particular, all the information necessary about measures for guaranteeing their impartiality and competence."

It is now easier for any organization to perform controls for delivering certificates. This act engendered, in turn, the need for an independent authority to oversee certifying organizations.

COFRAC [Comité Franáais d'Accréditation, French Accreditation Committee] was thus born. Accreditation is a procedure whereby an organization with authority formally recognizes that an organization or person is qualified for performing certain tasks [COUR95]. COFRAC thus oversees certifying organizations. In particular, it checks whether they are competent and impartial as required by the 1994 act. A ministerial order of 30 March 1995 officially recognized COFRAC as the accreditation authority.

The 3 January 1994 act also implemented certification procedures for services. Since no enabling decree had been drawn up about this, only products were, at that time, subject to controls and certified. Now, services can be treated like products in this regard. The Consumer Code [CC] once again provides the definition. Under Article L.115-27, "certification of products or services" refers to:

"the activity whereby an organization distinct from the manufacturer, importer, seller or service-provider, attests, at the request of one of these parties and for commercial purposes, a product's or service's compliance with characteristics that are described in a list of specifications and are controlled."

France has been long familiar with AFNOR's NF [normalisation franáaise] stamp of approval. A stamp "NF-Services" has been created for service-providers. Firms value the NF stamp, since it tends to be a guarantee to consumers. The purpose is to inform and, thus, protect consumers rather than to gauge a product or service on a scale of quality.

All goods and services on the market are concerned by certification except as specified under Article L.115-29 of the Consumer Code [CC]:

"the certification of foodstuffs and agricultural products that are not for food and are not processed, mentioned in Article L.115-21; the permission to market drugs for human or veterinary use that fall under Book V of the Public Health Code; the delivery of hallmarks, stamps, visas, court authorizations, or attestations of compliance with EEC provisions made by public authorities or the organizations appointed to do so and subject to a technical or administrative control by public authorities under the terms of legislative or regulatory arrangements; the delivery of labels or marks as foreseen by Article L.413-1 of the Labor Code, and of craftsmen's and master craftsmen's marks insofar as these marks tend to certify only a product's or service's origins and the implementation of the rules of the trade and of usages specific thereto."

According to this article's second paragraph:

"the list of specifications [référentiel] is a technical document defining the characteristics that a product or service must present and the means for controlling the product's or service's compliance with these characteristics."

This list is drawn up through the concerted efforts of all interested parties. It lays down the "rules of the game", so that each manufacturer knows precisely what he must do to meet up to the specifications and thus obtain certification. Paragraph 4 of Article L.115-28 holds that:

"the existence of lists of specifications is mentioned in the Official Journal of the French Republic. They may be consulted either free of charge at the certifying organization; or copies may be delivered at the expense of the party requesting them."

Clearly, the intent is to both improve consumer information and enable firms to know precisely the requirements made of them as regards the quality of their goods or services.

Paragraph 3 of the Consumer Code [CC] Article L.115-28 states:

"any reference to certification in advertising, labelling or presenting any good or service, or on the sales documents going with the good or service, must contain clear information about the nature and extent of the certified characteristics."

Several violations have been foreseen. For instance, Article L.115-30 forbids any such reference to a certification that has not been performed or to the use of any means for making consumers falsely believe that a good or service has been certified.

The EU is developing a certification policy [MORTU90]. A CE stamp of approval is intended to endow goods with the advantages of certification at the EU level. It is worthwhile to take a closer look at EU certification procedures [JOCE93].

For information to be effective and for controls and expertise to be accepted, a list of binding standards must, from the start, be drawn up in each field. For new products, drawing up norms is the starting point. To target the EU market, the manufacturer has to ask the EU standards center for certification. We imagine that such might be the case for products to be evaluated in linguistic engineering.

Once the standards are drawn up, manufacturers must use them and make them a full part of the manufacturing process if they want to have their products certified. It is then necessary to check this: an organization will evaluate whether products comply with standards.

Different systems may come into play: certifying products, which calls for regular controls; running tests on a product's major characteristics; granting quality guarantees by the manufacturer himself .

Controls by a third-party organization imply a system of accreditation, i.e., a tightly defined procedure with the participation of public authorities for checking whether an organization is qualified to perform controls. In principle, each member state has one such organization; in France, it is COFRAC.

The EU has a well-defined policy in these matters. The quite classical criteria are: the availability of personnel and equipment; the independence and impartiality of the personnel, who must have neither direct nor indirect ties with the product (thus excluding the inventor, manufacturer, etc.); qualifications and "professional integrity" (so as not to give in under pressure and change opinion); the obligation of professional secrecy; etc. Furthermore, emoluments should not depend on either the number of tests run or the results thereof.

Under this system, the fundamental principle, as laid down by a 1985 European directive, is the manufacturer's liability, especially for dangerous products. But this does not mean and this is always the case that the organization that evaluated the product cannot be sued if it committed a mistake while running tests.

A final point: since all this is a matter not of contracts but of regulations, we must not forget the possibility of sanctions if, for example, the manufacturer does not meet up to safety standards.

Many points in the evaluation procedure that the European Commission wants to establish in linguistic engineering clearly resemble certification. In both cases, an independent organization must make a decision about a product's characteristics. There is a legal obligation for this evaluation to be performed in a fully objective way; there must be no doubt about the certifying organization's independence. But we must not see these two procedures as being identical.

The procedure for evaluating systems for processing written and spoken languages is of a different nature than certification, a well-established part of consumer law. They must be differentiated so that linguistic engineering is not bound by domestic and European laws pertaining to certification.

Both these procedures are "open". They do not make a selection, which is the characteristic of closed procedures, where evaluation classifies products by comparing them. Open procedures do not aim at establishing a classification. We must bear this in mind, since it has obvious repercussions on the legal status of evaluation procedures.

Instead, the aim is to define, upon request from the evaluated party, certain characteristics of his product. In no case, should we, at the start, close the door to candidates for an expertise. This is an important point, since it reduces the possibility for suing. As we saw in the case of INC, any activity for comparing products inevitably leads to the possibility of legal recourse on the grounds of the results of comparisons.

Major differences, nonetheless, exist between open and closed systems. First of all, the procedure for evaluating linguistic engineering will not be performed for the purpose of delivering a certificate attesting a certain degree of quality. Instead, the aim is to objectively measure the evaluated products. In no case does the party performing the evaluation judge the product's quality as in the case of certification. It is up to the user to classify the products. He will retain only the ones he deems worthwhile given his needs. To this end, he may resort to the objective measures of the products made during evaluation. This system is not at all the one that holds in certification, where, for obvious reasons of safety, there is no question of letting consumers themselves make this "selection".

As seen, certification is a regulated part of law, since the necessities of public health and safety must be taken into account. The evaluation procedure we are proposing is a matter of contract, based on an agreement between at least two parties. There are barely any grounds for sanctions apart from the case of fraud committed by the party whose product is to be evaluated.

Another difference between certification and the evaluation we are proposing has repercussions throughout the whole procedure. Under certification, predefined elements are indispensable for establishing the specifications list. In the case of plans or of products for which there are no specifications at the start, there will be problems with implementing the procedure it is hard to talk about a specifications list, even if, in most of these cases, certain characteristics of the product are already defined whereas others are yet to be defined.

Certification has its source in pre-established standards.

An analogy can, to a certain extent, be made with innovative products up for certification. In this case, it is, by definition, hard to proceed on the basis of a prior set of standards. Only manufacturers can make warranties about innovative products. At the EU level, this entails a "manufacturer's declaration", which makes him responsible for his own analysis of the product. Controls are performed not by a certifying organization (since there are no standards) but by an independent third party. Certain characteristics of a product or the product as a whole may be controlled. In fact, the manufacturer provides, along with his declaration, a thorough technical record covering the product from design to production. An evaluation is then performed on this record.

Laws about certification cannot, therefore, apply to the evaluation procedure for linguistic engineering. The latter is much more "neutral".

There is no intention to judge tested products, an operation left up to users. In certification, this way of doing things cannot be imagined: the certifier, after having measured a product's major characteristics, is the one who decides whether the product is adapted to the market on the basis of whether or not it meets up to certain predefined standards.

The procedure for evaluating linguistic engineering must be less complicated, simpler than certification, even if certain principles of the latter remain valid. In any case, certain aspects of certification are relevant to our field, in particular those concerning innovative products.

I.A.1.b. Approval procedures

"the agreement that has to be obtained from the administration so that certain plans to be carried out by private persons can be executed or can benefit from a preferential financial or fiscal system."[DAL]

Approval procedures exist in many fields. In telecommunications, they condition the access of service-suppliers to the market. This obviously tends to favor government control. Approval is always indispensable: it must be obtained prior to the marketing a product or service. It is, therefore, necessary, depending on whether or not specific laws apply, to differentiate it from the evaluation proposed herein.

The argument for not applying this legal procedure to evaluation in our field is the same as the one for not applying certification. For this reason, we shall be brief. Approval procedures serve to "regulate" specific markets: severely selected products or services are forced upon consumers.

The "approver" judges the product submitted for testing. For linguistic engineering, the evaluating party would not "regulate" the market. On the contrary, those who use the tested products would judge them and reject the unsatisfactory ones. Even though, in any democratic system, it is ultimately up to the consumer to make his own choice, we can find cases and approval is one where certain products are kept out of the market without the consumer having any say.

Once again, this practice has obvious repercussions on the evaluating party's responsibility. Since the latter has the assignment not to judge a product but, instead, to make certain objective measurements, the risks of being held liable decrease considerably. On the contrary, since the "approver" can keep certain products or services off the market, just like the certifier who can refuse to warrant certain qualities, he can clearly be held liable.This is no surprise: these procedures lose objectivity, and the evaluated party thus has legal recourse.

This is not the case with our proposed evaluation. Value judgments about products are made directly by users: no product is rejected from the start, apart from a few exceptions to be discussed shortly.

I.A.1.c. Comparative testing

"a set of tests to which products of like nature or filling similar functions but coming from competing firms are submitted"[LAMY98];

and we might add:

"test results provide consumers with impartial information about the qualities or defects of each of the products tested, and enable them to choose the one best suiting their needs, tastes or budget."[LAMY98]

Ultimately, any evaluation procedure has a purpose like the one pursued through comparative testing, namely: to enable definite persons to make a choice out of a list of products of like nature. The results of tests run on these products facilitate this choice.

Comparative testing's finality is to classify the evaluated products on the basis of standard characteristics. Unlike the other procedures, comparative testing classifies; this special characteristic sets it apart from the proposed evaluation for linguistic engineering.

How does comparative testing differ from comparative advertising? The tests run by consumer organizations in France, such as UFO (Union Fédérale des Consommateurs) or the previously mentioned INC, are not done for advertizing purposes. Clearly, there is no doubt about the lawfulness of comparative testing, given the aim of informing consumers. In France, this ensues from article 11 of the Declaration of the Rights of Man and of Citizens, which stipulates that:

"any citizen may freely speak, write or print but must answer for the abuse of this freedom in cases determined by law.

Likewise, the publication of comparative test results in magazines entails a widely recognized right of reply [GMD75][LES75]. In contrast, comparative advertizing is tightly regulated, even though we observe a general tendency to slacken restrictions [CALA96]. Whereas the United States has allowed comparative advertising for a long time now, France forbade it till 1992. The EU has adopted a directive allowing comparative advertising under certain conditions.

In France, there is full freedom in matters of comparative testing, to such an extent that to talk about regulations seems excessive. Since the law has nearly nothing to say about this, interested parties must manage their relations through contracts. Nonetheless, two points must be made.

First of all, although no law explicitly regulates comparative testing, AFNOR has set several standards. A standard is:

"a document drawn up by consensus and approved by a recognized organization that, for common and repeated use, provides regulations, guidelines or characteristics for activities or their results, guaranteeing an optimal level of order in a given context."

ISO's standard X-50-005 concerns comparative testing; and X-50-006, the use of such test results. From the start, it must be pointed out that these standards are not binding. They merely propose standards to the various parties for managing their contractual relationship as best possible. With regard to comparative testing, these standards are a sort of code of conduct for the consumer associations that run tests. These standards of conduct could serve as a model for evaluation in linguistic engineering.

Several sectors, such as software, have standards. These various standards (The major ones are Z.67.133, Z.67.134, Z.67.135 and Z.67.143) might be valuable tools for helping the European Evaluation Agency improve its practices. Furthermore, several countries have set their own standards, e.g.: the German Deutsche Institut für Normung [DIN], the Standards Council of Canada [SCC], the British Standards Institution [BSI], the Italian Ente Nazionale Italiano di Unificazione [UNI], the Japanese Industrial Standards Committee [JISC], and the American National Standards Institute [ANSI]. The EU has, of course, tried to harmonize standards. To this end, the Europe Standards Committee was set up. For various fields, it proposes standards valid throughout the Union not to mention those of the Czech Republic and Switzerland. And at the international level is the previously mentioned ISO.

Secondly, since comparative tests are intended for publication, the freedom-of-the-press act of 29 July 1881, which provides for a right of reply, applies. For broadcasts, the act of 29 July 1982 on radio and television applies.

Comparative testing has, therefore, a different legal status than the evaluation procedure for linguistic engineering. The two are, however, alike as regards one point: for want of legislation, contractual law manages relations between parties.

I.A.2. The legal status of evaluation in linguistic engineering

However, we must bear in mind that this way of determining the law to be applied is not very sure. Depending on the methods or even the terms used, the conclusion can turn out to be amazingly different. For instance, adding the idea of measuring a "quality" to the evaluation procedure makes it more like certification. Yet another example: when, deliberately or even undeliberately, products are even implicitly compared, evaluation resembles comparative testing.

France holds to the principle of the "autonomy of the will": parties have full contractual freedom. This principle is, however, conditioned, in particular by laws about certain types of contractual relations; but this is not the case in our field.

Since the existing models are far from satisfactory, it is possible to create in full the evaluation procedure for linguistic engineering in lines with the EU's recommendations. We can now formulate a few proposals based on important principles. To eliminate the possibility of the evaluated party suing, the evaluation procedure must contain rules of conduct and of methodology for performing the evaluation successfully.

Mainly, this means adopting the principles of neutrality and objectivity, and requiring professional qualifications for conducting tests. The procedure would then be satisfactory on this point. In brief, this means finding the best solutions for the difficulties inherent in any evaluation procedure. These difficulties, even in the simplest system, cannot be fully forgotten.

I.B. Organizing evaluation

Whether the evaluation procedure is followed or not will obviously have repercussions on liability.

What phases will the evaluation procedure have? What arrangements are called for? The evaluation party is made up of three actors. First of all, a committee must be specially set up to "pilot" the procedure. Secondly, the "evaluators", i.e., the persons actually performing the evaluation, should have a degree of autonomy to carry out tests on the types of products or techniques selected by the forementioned steering committee. Thirdly, a sort of scientific council of experts should be assigned to work out a methodology and oversee the procedure's implementation.

The "steering committee" will be the driving force. It will determine the fields to be investigated and, in parallel, the criteria for evaluating them. In our opinion, this committee must, from the start, be able to strike a balance between research and development. The aim is to inventory the research conducted in linguistic engineering so as to set priorities and thus measure needs to be satisfied as best possible.

This objective can probably not be achieved without a dual structure. It would be preferable to divide steering committee members in two. Each group would be assigned to draft a report about linguistic engineering: the one, on basic, and the other, on operational research. A work group, specially appointed by the committee, would use these two reports to write the final one. It would propose the fields to give priority and the major selection criteria. This report would be debated during a special meeting of members of both groups and, later, by the full committee. The intention is to avoid favoring one work group over the other and to be sure that the priorities established by both groups are carried over into the final report. Products and techniques would thus be preselected for evaluation. The final report would be transmitted to the persons performing the evaluation. The latter will retain a field mentioned in the report and the basic criteria for the evaluation. Manufacturers and service-providers will be notified that they may submit their products for evaluation.

From this point on, a detailed procedure would be followed with the aim of seeing how a person, natural or juristic, will show his interest in this "invitation to tender" and how his application will be accepted. There are two steps: receiving and recording the projects filed by the interested parties; and then the steering committee's examination of these projects as it makes a preliminary selection. The first step presents no special problems. The second step is trickier. This examination must be superficial; in other words, the steering committee must simply set aside the projects that are far out of line with the stated objectives. The real selection must be made by the users when they use the evaluation's results to compare products on their own. There is no doubt that recourse would be open to someone whose application is unjustly set aside during this phase for reasons unclear to him.

Nonetheless, a "minimal preselection" could be made, and should even be encouraged, so that users are presented with a relatively coherent panel of products. This preselection should preferably involve preliminary tests following precise criteria so as to make it as objective as possible.

This would decrease the risks of law suits. Thanks to these criteria, derived from a method, each type of product or technology can be measured.

The scientific council should fix these criteria beforehand. This would strengthen the guarantees of neutrality and objectivity. The evaluators would be charged with applying this methodology.

The comments of the evaluated firms should be taken into account.

Information and ideas could be exchanged with the evaluating party, as in hearings with cross-examination.This would offer the best guarantees.

The selection of experts for the scientific council must follow principles, which will be stated hereafter. This council will examine the steering committee's final report and adopt, for each product and technology, a method of evaluation. This method will be used by the persons running the tests to obtain objective results under the conditions (of procedure and time) set by an agreement.

This contractual agreement should leave room for having test results verified by a sort of "certifying third party", who would have certain powers (of access, limited or not, to the site, to information about the product, etc.) for retesting products and verifying that the first evaluation was satisfactory. He would, ultimately, prove that the results are coherent.This provides the evaluated party with an additional guarantee. It should be supported.

Test results will then be transmitted to the users of the products or technology under conditions to be set by the contractual agreement

II. The contractual aspects of evaluation

II.A. Preliminary questions

II.A.1. Selecting experts

However a scientific council made up of experts could turn out to be quite useful for overseeing the procedure, enforcing the code of conduct and establishing methods for testing products and technology. How to select these experts? In our opinion, two major principles should guide the selection of all experts: first of all, professional qualifications in the scientific field and, secondly, objectivity. Experts must reason objectively, scientifically; they must not give subjective opinions. This neutrality entails the absence of direct or even indirect relations with what is being evaluated; otherwise, an unlawful conflict of interest arises.The evaluated party would have recourse if the experts are not qualified or lack objectivity. Courts have often granted recourse under these conditions.

II.A.2. The procedure's objectivity

II.B. Contractual obligations

II.B.1. Contractual relations between the steering committee and evaluators

The contract must precisely define the conditions under which product-users may use test results.

II.B.2. Contractual relations between the evaluators and evaluated parties

II.B.2.a. The evaluator's contractual obligations

For example, evaluators could be forbidden to communicate preliminary test results. If the communication of results is to be paid for, each evaluator must hold to the obligation of confidentiality as long as the specified conditions (about the payment or signature of an agreement, for example) are not fulfilled.

Evaluators must do their work sedulously by performing the procedure properly and testing the products within the allotted time. As already stated, this work must be done in full objectivity. This obligation binds evaluators to both the steering committee and the evaluated party.

Evaluators must work autonomously from the steering committee.

There is no question of orders or even recommendations, either direct or indirect. Once again, whether or not the evaluated party can sue will depend on whether or not this principle of independence has been respected.

Thought must be given to the tricky question (much discussed in common law) of liability. Liability will be quite limited if all the fundamental principles are respected. In effect, the evaluated party would probably have the possibility of legal recourse only: 1) if the evaluators were not professionally qualified or lacked objectivity and this would be hard to prove; or 2) if there were doubts about the methods drawn up and adopted by the scientific council. Only a suit brought for this last reason would have any chances of succeeding apart from the situation, already pointed out, of an evaluator with a conflict of interest; but these chances should nonetheless be rather small. The same principles of liability will, a priori, govern both the first evaluation and the evaluation performed by the "certifying third party". The evaluated party will have the same grounds for holding the evaluators responsible. However, we suppose that the burden of proof will be hard to bear if the previously mentioned warranties are respected.

II.B.2.b. The evaluated party's contractual obligations

A tricky point has to do with the obligation on the evaluated party to deliver information about how his product is made, information that may include secrets about the manufacturing process. Following the first phase of evaluation, a meeting of all evaluated parties will be held for this exchange of information. This raises more than one problem. The evaluated party must accept this obligation from the start or else be eliminated form the evaluation, if the steering committee considers this condition to be indispensable and applicable to all applicants. He must, therefore, be precisely informed about this well before the procedure starts. This is a first, highly important, point.

This exchange of information gives rises to another problem. It lets the possibility open that one evaluated party may recreate a product by using the data provided by another evaluated party. Are these parties ready to accept such a possibility? There is a real problem about using information. The contract must deal with it.

A firm accepts the contractual conditions for the evaluation, and thus the procedure. Of course, legal recourse is possible. In the case of legal expertise, there is talk about a "right to fair expertise" under Article 6 of the European Human Rights Convention. This quite interesting notion could be broadened to cover all forms of evaluation. It would mean that an evaluated party has the right to have his product evaluated with the best possible warranties, thanks to the quality work performed in full objectivity and confidentiality by a group of independent experts.

Otherwise, he may seek legal redress.

Finally, the evaluated party must clearly accept that his product's test results will be circulated. As tends to be the case in comparative testing, we consider that, given authors' rights, the evaluated party does not have the right to use these results himself without the evaluators' consent.

II.C. Conclusion

We can summarize the evaluation procedure, as described in the foregoing pages, in five points.

II.C.1. The source of evaluation

This final report will be transmitted to both the evaluators and the scientific council. The former will be asked to select as broadly as possible the products to evaluate (= a sort of invitation to tender). The scientific council, eventually with the help of the concerned firms, will then propose a method of evaluation for each field.

The steering committee is the party who asks for the evaluation; but in reality, any interested firm could have its product evaluated only if the latter has certain minimal characteristics. There is a sort of double impulse underlying evaluation; both impulses are indispensable for the procedure's operation.

II.C.2. The legal framework of evaluation

For the evaluated party: under what conditions does he make his product available for evaluation? what may he forbid or allow, especially with regard to circulating test results? The communication of information about tested products or techniques including the communication to other evaluated parties must be tightly defined in line with what the parties want.

For evaluators: which obligations (of discretion, etc.) bind them? How to settle the question of author's rights to test results? For users: under what conditions may they acquire and use test results?

II.C.3. The phase of preliminary investigation

II.C.4. The experts: Selection and status

II.C.5. The circulation of test results

However, these rights may be made over, in particular to the steering committee. The party having the rights could then allow or not allow test results to be circulated. The code of intellectual property applies; it provides for exceptions to the general rule.

From the start, the contractual agreement must meticulously state the conditions for using test results.

There is no dearth of legal questions. But they have one point in common: they must be settled within the framework of a contract.

[CC] Code de la Consommation. Available at URL: http://www.legifrance.gouv.fr/citoyen/uncode.ow?code=CCONSOML.rcv

[COUR95] A. Couet, J. Igalens and H. Penan in "La certification", PUF, Collection QSJ, 1995, pp 19-24.

[CALA96] J. Calais-AUloy et F. Steinmetz, Droit de la consommation, Précis Dalloz, 4^ème édition, 1996, n^o99 et suivants.

[DAL] Lexique de Termes Juridiques, Dalloz.

[GMD75] D. Ganancia-Mitz et P.-F. Divier, La presse et le droit de critique des produits, G.P. 1975, 2, p.514.

[JE98] Jeanne Etiemble, L'expertise collective à l'INSERM, dans "Prévenir les risques: de quoi les experts sont-ils responsables?", dirigé bar G. Decrop et J.-P. Galland, L'aube, 1998.

[JOCE93] Journal Officiel de la Communauté Européenne, JOCE L 220, 30 août 1993, p.23.

[LAMY98] Droit Économique, sous la direction de R. Bout, Lamy, 1998, n^o5646.

[LES75] G. Lesourd, Liberté du commerce et défense des consommateurs, G.P. 1975, 1, p.380.

[MAY90] R. May, "Criminal Evidence, Sweet & Maxwell, 1990.

[MORTU90] M. Mortureux, L'Europe des normes et de la certification, Enjeux, Avril 1990, p55.

[NCPC] Nouveau Code de Procédure Civile. Available at URL: http://www.legifrance.gouv.fr/citoyen/uncode.ow?code=CPROCIV0.rcv

[PONT96] J.-M. Pontier, "La certification, outil de la modernité normative", p. 335., au D. 1996.

[JORF78] Loi du 10 janvier 1978 n^o78-23, Journal Officiel de la République Française du 12 janvier 1978, JCP 1978, III, 46698.

[JORF94] Loi du 3 juin 1994 n^o94-442 (JORF du 4 juin 1994, p.8072, D. 1994, législation, p.300) complétée par le décret n^o95-534 du 30 mars 1995 (JORF du 5 avril 1995, p.5437) et par un arrêté de la même date.