Joakim Lavér and Olof Larsberger of Hannes Snellman Attorneys Ltd discuss the advantages and disadvantages of relative evaluation models in public and utilities procurement.
In procurement proceedings when a contracting authority or entity elects to award a contract based on the principle “the most economically advantageous tender” it must consider certain criteria linked to the subject matter of the contract in question, such as price and quality. The contracting authority or entity must specify, in the contract notice or documentation, the criteria on which it will base its evaluation and the relative weighting of each of those criteria. Apart from these obligations, neither the EU directives in the field of public and utilities procurement (directives 2004/18/EC and 2004/17/EC respectively) nor the Swedish acts on public and utilities procurements (mirroring in the essential parts the EU directives) stipulate how the evaluation procedure should be conducted. Thus, a contracting authority or entity is at liberty to set the method for such procedure.
However, such liberty is not unlimited. In accordance with the fundamental principles of public and utilities procurement, a contracting authority or entity must, in a transparent and predictable manner, present the method for the evaluation procedure. Thus, the contracting authority or entity must, beforehand, present an evaluation model, indicating how “the most economically advantageous tender” will be identified. This obligation raises a central question in the field of procurement. How should an evaluation model be constructed and applied in order to comply with the fundamental principles of transparency, predictability and equal treatment? Looking one step further, how should an evaluation model be constructed in order to certify the award of “the most economically advantageous tender”?
The most commonly-used evaluation model when awarding contracts based on the principle “the most economically advantageous tender” is probably – at least in Sweden – the so-called relative evaluation model. This article will, in the light of a recent court case in Sweden, consider the use of such a model. In our view, the legality in applying relative evaluation models in public and utilities procurement is always open to challenge.
ABSOLUTE AND RELATIVE EVALUATION MODELS
The common purpose of all evaluation models is to establish a method for weighting different criteria and comparing tenders. This procedure may be conducted in different ways. A so-called absolute evaluation model usually follows a three-step procedure: First, each tender is awarded a certain score, based on evaluation of the non-price criteria. Second, each score is transformed into a monetary figure, which is added to or deducted from the offered price. Third, the tenders are compared with each other. The characteristic of an absolute evaluation model is the non-dependency of other tenders when evaluating a certain tender. However, the relative weight between price and other criteria will vary, depending on the outcome of the evaluation of the non-price criteria.
As stated above, the most commonly-used evaluation model in Sweden is probably the relative evaluation model. The key characteristic of such a model is the dependency of other tenders when evaluating a certain tender. Thus, the final score awarded to a certain tender will depend in part on the score awarded to other tenders. Consequently, the evaluation of a given tender will not, in practice, be based exclusively on “the own merits” of such tender.
In the Swedish legal and economic debate, relative evaluation models have been criticised. The critics have referred to the fact that the ranking between two tenders, when applying a relative evaluation model, might be dependent on a third irrelevant tender. Further, it has been argued that the relative weight between the criteria, regardless of the specifications in this regard in the contract notice or the documentation, de facto cannot be set beforehand but will depend on the lowest price.
PROBLEMS RELATED TO THE USE OF RELATIVE EVALUATION MODELS - A SWEDISH EXAMPLE
The problems emanating from the use of relative evaluation models were well illustrated in a recent Swedish court ruling by the Gothenburg Administrative Court of Appeal (case 5293-10). The case concerned judicial review of a procurement which was subject to the Swedish Act (2007:1091) of Public Procurement (SAPP), carried out by the City of Malmö. In the case, the Court of Appeal requested a statement from the national supervisory body for public procurement, the Swedish Competition Authority (the SCA), as to whether the relative evaluation model applied by the City of Malmö was compatible with the stated intention to award a contract based on the principle “the most economically advantageous tender”.
The procurement in question concerned the purchase of a GPS-based map and guidance system for road maintenance vehicles. The assignment was split up in two stages, and the tenderers were asked to include both stages in their respective tender. According to the contract documentation, the award of the contract followed the principle “the most economically advantageous tender” based on the following criteria and relative weights:
Price 60 per cent
Operation costs 20 per cent
Organisation/references 20 per cent
Price 50 per cent
Operation costs 20 per cent
Development costs 10 per cent
Organisation/references 20 per cent
The final score awarded to each tender was to be the average of the scores earned in stages one and two.
In the evaluation of “price” in stage one, the tender with the lowest price was awarded five points. The other tenders were awarded scores with a value relative to the lowest tender. For example, if a tender’s price exceeded the lowest tender’s price by 50 per cent, it was awarded two-and-a-half points and so on. If a tender’s price exceeded the lowest tender’s price with more than 100 per cent, it was given a score in the form of a negative figure. The evaluation of “operation costs” followed the same principle. “Organisation/references” was evaluated on a scale from one to 10. The evaluation in stage two followed the same principles (where “development costs” was evaluated in the same manner as “price” and “operation costs”).
According to the SCA, where two or more tenders are equal in terms of quality standards (ie, are awarded the same score on non-price criteria) the tenderer with the lowest price should be awarded the contract in question. In the procurement in question all tenders were awarded the same score for “organisation/references”. Consequently, following the logic of the SCA, the contract should have been awarded to the tender with the lowest price and costs. However, this was not the case. The application of the evaluation model resulted in the award of the contract to a tender with a total price (ie, “price, “organisation costs” and “development costs” combined) that exceeded two of its three competing tenders.
Now, how was this possible? According to the SCA, the model was clearly and satisfactorily described in the contract documentation and was also applied in accordance with the description. Thus, the problem was not the description or the application of the model but its construction. According to the SCA, the flaw in the model could be well illustrated in the scenario in which a hypothetic tenderer, who submits a tender with the highest total price (ie, “price”, “operation costs”, and “development costs” combined) and which is given the lowest quality score, nevertheless may be awarded the contract. Such a result - as preposterous as one might characterise it - could occur if the hypothetic tenderer practises “strategic bidding”, by, inter alia, offering a very low price on the heavily weighed “price” criterion - resulting in very low, or even negative, scores to the other tenders - and at the same time “compensating” for such low up-front price through excessive operation costs.
Consequently, the SCA concluded that the evaluation model in question was not compatible with either the intention to award a contract based on the principle “the most economically advantageous tender”, the principle of equal treatment or SAPP in general.
The Administrative Court of Appeal, ruling in accordance with the findings of the SCA, stated the following:
“When two or more tenderers have exactly the same total number of points on non-price criteria, an evaluation model which intends to select the most economically advantageous tender should lead to the success of the tenderer with the lowest price.”
Referring to the findings of the SCA, the court noted that the contract had been awarded to a tenderer with a higher total price than two of its three competitors, despite the fact that the other two tenderers had been awarded the same score as to the non-price criteria. According to the findings of the court, such an evaluation model does not result in the award of a contract to the most economically advantageous tender and, therefore, violates the SAPP.
As stated above, relative evaluation models have been criticised in Swedish legal and economic debate. In our view, the problem connected with this type of model is obvious. It should be clear to everyone that the application of any evaluation model, in which the ranking between two tenders might be dependent on a third irrelevant tender, runs the risk of leading to unfair and unacceptable results. Thus, our view is that, in order for an evaluation model to be compatible with the fundamental principles of transparency, equal treatment etc, it is not sufficient simply that the model and its application be satisfactorily described in the contract documentation and that the model be applied in accordance with the description. In our opinion, the model itself must be constructed in a way that satisfies the fundamental principles of public and utilities procurement. That can never be the case if the application of a model - however well described in the contract notice or documentation - may lead to the award of a tender which, based upon the given criteria, is not the most attractive one.
Despite the criticism in Sweden of relative evaluation models, Swedish courts have been ambivalent (to say the least) as to the legality of the use of such models in public and utilities procurement. From older Swedish court cases, it could possibly be inferred that relative evaluation models do not violate the procurement rules per se, but that the legality of such model must be assessed in each individual case. However, case law has been contradictory as Swedish courts have come to opposite conclusions in relation to almost identically-constructed relative evaluation models. Further, the Swedish Administrative Supreme Court (the highest court in procurement legal review cases in Sweden) has consistently refused to hear appeals in cases concerning this matter.
In the light of this previous legal uncertainty in relation to relative evaluation models, the above-mentioned ruling of the Gothenburg Administrative Court of Appeal is welcome. According to our interpretation of the ruling, models that may not result in the award of the tender with the lowest price, where two of several tenders are equivalent in relation to non-price criteria, will breach the procurement rules per se and must not be used. Thus, in our view it is irrelevant if the application of such model, in an individual case, de facto leads to the award of the “the most economically advantageous tender”. The simple risk of the opposite result is sufficient to disqualify the model, and its application, from being compatible with the procurement rules. The rationale in this conclusion follows from the fact that it will always remain unknown how the tenders would have been drafted had an acceptable model (without risks for manipulation etc,) been applied.
Obviously, not all relative evaluation models are as poorly constructed as the one applied by the City of Malmö in the above-mentioned case. In this case, the defects connected with the application of the model were self-evident. However, regardless of the construction of the individual relative evaluation model, the fundamental problem connected with these models, ie, the risk that the ranking between two tenders might be dependent on a third tender still remains. In the light of this, one must ask whether relative evaluation models will ever be compatible with the fundamental principles of procurement and the “most economically advantageous tender” principle. In our opinion, the answer must be “no”. The application of such a model – regardless of the “degree of defects” in its construction – is, in principle, always in breach of the procurement rules.
The identification of “the most economically advantageous tender” might be a difficult task. What criteria should be used? How should the evaluation process be performed? The list of questions might be extensive. In any event, we suggest that a contracting authority or entity who wishes to award a contract pursuant to this principle ought to refrain from applying a relative evaluation model. Instead, another method, such as the application of absolute evaluation models, should be considered. In this way, the contracting authority or entity may not only accomplish the procurement in a transparent, predictable and non-discriminating matter, but may also get the best value for the taxpayers (or stakeholders).
The case law of the Court of Justice of the European Union on the use of different criteria in connection with the fundamental principles of public and utilities procurement is extensive. However, to our knowledge the Court has never ruled on relative evaluation models and their compatibility with the EU public and utilities procurement rules. In our view, such a ruling would be most welcome.