In part 1 of this article I sketched out some issues to consider when taking into account evidence of past performance in decision-making about funding. In part 2, I attempt to provide some answers.
It would be easier if there was a simple right way to approach the problem - a set of rules about what to do - but commissioning of services is usually fairly messy. It is an attempt to reconcile and balance competing sets of views and priorities using available evidence in a critical and sensitive way, within a complex political, economic and social context. There are no simple answers and what I outline here is a set of principles and an approach rather than a blueprint.
For me, the starting point is the fundamental aim of funding, which is to provide high quality services for the service beneficiaries. This is the bottom line, nothing else. All other considerations are or should be secondary. In most cases, in most situations, all stakeholders agree on this. Where other considerations predominate, for example, a particular organisation has to be funded, evidence of past performance is probably irrelevant anyway, as it is unlikely to outweigh other factors. There is a long history of public authorities continuing to fund poor performing organisations as the fallout of decommissioning the project would be too great.
Perhaps controversially I think that being fair to potential applicants is a secondary consideration and that it is unlikely that you can equally get the best deal for the service beneficiaries and be “fair” to all applicants. Fairness, of course, is a tricky concept, but here I am using it as it is often used in procurement, to mean that all applicants are treated in the same way, there is an attempt to create a “level playing field”- new organisations, who have no links, track record or reputation in an area have the same opportunities as those that do.
The idea of a level playing field in procurement (grants or contracts) seems to me to come principally from the ideology of “free markets” and “equal access to the market”. Organisations in the market have a right to equal treatment. I’m unsure that these ideas are a useful starting point when applied to the provision of health and welfare services in the same way that would work for buying easily countable things like paper clips.
People who use services don’t appear to be calling for “a free and open market in service provision”. This is an ideology that has been adopted by some commissioning authorities in the belief that it will lead to better services, but it is unclear what evidence they have for this in the purchase of health and social services, especially at a local rather than a national level. In certain limited conditions, national and EU legislation require equal treatment in procurement; however, this does not mean we should apply legislation about “free access to the market” in all procurement exercises.
It seems to me sensible, reasonable and in the best interest of users of services that organisations that are providing high quality services should continue with minimal disruption, whilst those providing poor services should be stopped. Despite the title of this article the important issue in procurement is not reward/punishment for service providers, it is good quality services.
A reason given by those that believe in “free market” and “open competitive” procurement is that it encourages innovation, but does that claim stand up to scrutiny? Whilst new organisations might come along that offer different models, this has to be balanced against the damage to innovation that comes from competitive procurement, in terms of destabilisation, destruction of existing networks, management and other transaction costs.
I suggest that the first principle is that commissioning and procurement processes should seek to continue to fund existing services that have a good record of past performance (whilst at the same time decommission poorly performing services). To be explicit, I am suggesting that decision-makers should show a preference for existing well-performing services over new entrants and “adjust” their application processes to reflect this. This doesn’t mean that there won’t be any new entrants as it is unlikely that all existing service providers will be performing at a high standard, it would just be more difficult for new entrants to succeed.
In metaphorical terms it is the principle of “if it ain’t broke, don’t fix it”. No football team would sack all of its players and then re-recruit them. It is straightforward to design an application and decision-making process that will do this through incorporating best value “local connection” criteria where the benefits of local networks and partnerships are recognised.
The second principle is less controversial. Evidence of past performance should be robust, independent and open to challenge. An organisation should not be penalised on the basis of anecdotal, biased or ill-informed evidence. Similarly an organisation should not be favoured only because they have a good relationship with the funder or they’re well-liked, or a single member of the panel has heard good things about their work.
This means that a core part of the process of commissioning and procurement should be to decide (and consult on?) what evidence of past performance is to be taken into account.
For example, it may be decided that the only evidence of past performance that will be taken into account is evidence independently collected by commissioning authorities concerning locally delivered, similar projects where the organisation has had a right to challenge the outcome of the evaluation. This is a highly bounded way of including evidence of past performance but would still need considerable clarification to make it robust. Decision would have to be made about what counts as a commissioning authority, how independent is independent, how local is “locally delivered”, what is a similar project and what counts as a right to challenge? Organisations that apply would be asked to submit a full list of “similar projects” within a given time period, who they were commissioned by, and what evaluation of performance took place, and asked to supply the evaluations.
To take another example, it may be decided that the reputation of an organisation with local public authorities will be taken into account. This could be done by asking the organisation to submit additional evidence in the form of a small number of references from public authority employees at a management level who have had direct experience of working in partnership with the organisation.
The critical point here is that the form of evidence is thought about and decided upon before the decision-making panel meets and all other evidence is excluded. No more will panel members have the freedom to bring to the table their own experience and knowledge in an unstructured way.
The third principle is that evidence of past performance should be used in a sophisticated way. In most cases it should be used to contribute to decision-making rather than as a blunt instrument of decision-making. The exception is when the evidence of failure is severe or there has been clear misconduct e.g. falsification of monitoring returns. Perhaps, in these cases organisations should be told that their application will not be considered.
Organisations applying for funding should be given an opportunity within the application form to contextualise both good and poor performance, explain what happened and why, and the action the organisation has taken in response. There may be good and understandable reasons for why a project has underperformed, and the organisation can show how it has learnt from the failure and what steps it has taken to prevent further problems. If a project has performed well then they need to be able to explain why they were successful, and how they will maintain and increase their success.
You may be reading this article and have come to the conclusion that it is just all too difficult, that there is no practicable and satisfactory way of including evidence of past performance, and indeed, many procurement processes avoid the issue or deal with it inadequately. They rely solely on what is written in an application form, so a “bad” organisation with a good bid writer has a better chance of being successful than a “good” organisation with a bad bid writer. Including past performance is one way of making that just a bit less likely.
It‘s easier to “duck” the issue but, of course, the people who pay are the service beneficiaries.