CPI 2018: Whose perceptions, and who does the work of collecting and processing the source data?

Civis Mundi Digitaal #84

door Michel van Hulten

Critics of the annual CPI-reports as issued by TI, often governments of countries which find themselves at the bottom end of the rankorder as presented by TI, concentrate their dissatisfaction on the end-result. Cameroon was loud and clear in the first year of the CPI, 1995, that it did not want to accept to be the last one on the list. And this was not the last country of which the government got a bit angry. The endresult was for them what counted.
If dissatisfied, the first question to ask, however, is what activity do we call ‘corrupt’, i.e. ‘what is corruption?’. And who will be in charge of pointing to what he/she/the society/the government considers to be ‘corrupt behavior’? Second comes the question: what data are used to allow the conclusion about the level of corrupt behaviour per country? Data about facts? Data about perceptions of the reality? For the CPI that choice has already been made: it is perceptions we collect. Subsequently: how to process those data in order to get a result that can be used in quantifying corruption per country, as an absolutely needed ingredient for the making of a rank order from best performing country to worst performing country in corrupt behaviour.

I will write about the corruption definition in a later paper (190410-04, forthcoming) of this series and restrict myself here to the observation that not only TI, but also others did not have a perfect definition for the concept ‘corruption’ in 1993, the year TI was founded, neither in the year 1995, when the CPI began life, neither at this moment after 25 years of TI-organised fighting against corruption. This hinders reaching success.

Ordinary people are involved?

In those years and at the moment TI knows perfectly well what it is doing and on whose behalf in particular it acts in fighting corruption. Read only the following quote.

‘Ordinary citizens often stand on the front line against corruption. It is citizens who face demands for bribes to access public services, such as school entry for their children or life-saving medical care. Transparency International believes that people’s experience and perceptions of corruption are key for understanding corruption risks around the world. The public also plays a vital role in holding governments accountable for their actions – or lack of action – in addressing graft. This is a summary report of the key findings from the ninth edition of Transparency International’s Global Corruption Barometer series – the world’s largest survey asking citizens about their direct personal experience of bribery in their daily lives, their perceptions of corruption challenges in their own countries, and their willingness to act against corruption. The results of this latest edition of the survey have been published via a series of regional reports. This summary brings together those reports and covers 119 countries, territories and regions around the globe. It is based on interviews with 162,136 adults from March 2014 until January 2017 and it identifies the key differences between the regions and key results by place. This report clearly demonstrates that bribery is a far too common occurance around the world, with nearly one in every four public service users having to pay a bribe each year. With the United Nations Sustainable Development Goals requiring governments to reduce corruption and bribery in all its forms by 2030, the results from the survey can be used to show governments just how far they must go before these goals will be realised.’
Source: People and corruption: Citizen’s Voices from around the world, Global Corruption Barometer 2017 by Coralie Pring, Research Expert, Transparency International Secretariat, ISBN:978-3-96076-067-2 .

The CPI represents ‘petty-bribe-thinking’?

This quote clearly demonstrates the in-house-thinking of Transparency International about corruption. Indeed it is overwhelmingly the petty bribes ordinary citizens pay the policeman, the teacher, the nurse, that are seen by TI. I quote from above that it is ‘citizens who face demands for bribes to access public services’, which means also that TI pays attention to what these ‘ordinary citizens’ perceive as corruption. Neglecting the millions euros and dollars which are not in reach by ordinary citizens but handled by the powerful in our world.

TI had neither personal nor financial means in the 1990’s to organise its own global research needed for the immense task to qualify countries according to the level corruption had reached per country all over the globe. The choice made was to look for intermediaries dealing with the subject, research and investigations done by others, collect their results, harmonize these results for further processing, and draw conclusions from this material resulting in what became known as Corruption Perceptions Index.

That fundamental choice is still followed today and the result is that we get presented the CPI 2018 as obtained from the combined information found in 13 research efforts dealing with aspects of governance in at least ‘a number’ of countries.

Quality?

A second choice was not to have any doubts about the quality of these sources and not to dispute the way in which all of these research efforts reached their results (to be used by TI). This is confirmed again in the CPI 2018 Report under the ‘Frequently asked questions’ in the CPI 2018 documentation, where TI confirms: (see also that the ‘ordinary citizens’ have been replaced by ‘experts and business executives’)

‘The CPI scores and ranks countries/territories based on how corrupt a country’s public sector is perceived to be by experts and business executives. It is a composite index, a combination of 13 surveys and assessments of corruption, collected by a variety of reputable institutions.’ Later on the same page: ‘The CPI contains informed views of relevant stakeholders, which generally correlate highly with objective indicators … ‘ .
We are invited to believe that the institutions (as also named below) are reputable, and that the stakeholders (meant are those that had been involved in the reviews done by these reputable institutions) are ‘relevant’ and ‘have informed views’. Is this anything more than relying on the ‘significance of their gut feelings’ and that we are tempted to believe inventive stories, and to neglect relevant information?

As these surveys and experts do not use information about the abuse of public power for private gain in our world, also called ‘corruption’, which has been observed, documented, described, collected and preserved in data-bases, what exactly is it that makes us trust that their judgments, and the qualifications they give to particular countries, are correct or at least useful for processing as data?

Conclusion

All the foregoing together leads me to the conclusion that we get a ‘perception of the presence of corruption’ as worded by a small bandwith of experts in this field, mostly male, white collar workers, between 35 and 55, with MBA or doctorate in economics or finance, in leading business or academic-research positions, often temporary residents in the country from which they report, employed by companies who make their living from studies in this field. It is not the ‘ordinary people’ involved in the work of these reviewers and investigators. Also, if their perceptions are captured in the right way, and are processed into an amalgam which represents their views correctly, still there is no way in which we can confirm that these perceptions are the same as we would get if we directed those same questionnaires to women, youth, blue overall workers, lower educated levels in society, the poor, expatriates, in short the powerless, etc. No one will doubt that asking trade-unionists about corruption will deliver other results than are received asking CEO’s and professors in business administration. Eventually, why should we pay more attention to the perceptions of those belonging to the first selection and not to the second?

Let us go over the list of the chosen 13 sources.

1. AfDB, African Development Bank, Country Policy and Institutional Assessment 2016

Type of assessment:Expert evaluations

The CPIA is carried out by a group of country economists with vast experience in policy analysis. The knowledge of these experts is complemented with that of local contacts that provide both quantitative and qualitative insights. Peer discussions are also used to monitor the quality of the findings.

2. Bertelsmann Stiftung Sustainable Governance Indicators 2018 (SGI)
Type of assessment:Combination of quantitative data and qualitative expert assessments
The indicators are calculated using quantitative data from international organisations and then supplemented by qualitative assessments from recognised country experts. For each SGI survey, individual countries are evaluated by two (or more) leading experts.

3. Bertelsmann Stiftung Transformation Index 2018 (TI)

Type of assessment:Qualitative expert survey

Assessments are provided by two experts per country. Country assessments consist of two sections: the written assessment of the state of transformation and management performance in a country (country report) and the numerical assessment of the state of transformation and management performance (country ratings). Scores are given by a country expert, which are then reviewed blind by a second country expert who also provides a second independent rating of the country. These scores by experts are then verified and discussed by regional coordinators to ensure intra and inter-regional comparability in ratings. In addition, Bertelsmanns Stiftung has also instituted an extra layer of verification to ensure the scores provided match the qualitative descriptions for each country.

4. Economist Intelligence Unit Country Risk Service 2018 (EIU)

Type of assessment: Risk evaluations based on qualitative and quantitative indicators

The EIU relies on teams of experts based primarily in London (but also in New York, Hong Kong, Beijing and Shanghai) who are supported by a global network of in-country specialists. Each country analyst covers a maximum of two or three countries/territories. The economic and political reports produced by EIU analysts are subjected to a rigorous review process before publication.

5. Freedom House Nations in Transit 2018 (FH)

Type of assessment: Qualitative expert assessment

The Nations in Transit (NIT) surveys were produced by Freedom House staff and consultants. The latter were recommended by relevant authorities and are regional or country specialists. A range of sources were used in compiling the report, including: multilateral lending institutions; non-governmental organisations; and other international organisations; local newspapers and magazines; and select government data.

6. Global Insights Business Conditions and Risk Indicators 2017 (GI)

Type of assessment: Commercial business expert assessment

The assessments are made by over 100 in-house country specialists, who also draw on the expert opinions of in-country freelancers, clients and other contacts. The ratings reflect IHS Global Insights expert perceptions of the comparative level of the problem in each country/territory. The ratings assess the broad range of corruption, from petty bribe-paying to higher-level political corruption and the scores assigned to each country are based on a qualitative assessment of corruption in each country/territory.

7.  IMD World Competitiveness Yearbook 2018 (IMD)

Type of assessment: Executive opinion survey

The WCY largely includes hard data but also a survey of senior business leaders who, together, reflect a cross-section of a nation’s corporate community. IMD calls upon local and foreign enterprises operating in a given economy, and surveys both nationals and expatriates, so as to add an international perspective on local environments. In 2018, more than 6,300 business executives responded.
The IMD World Competitiveness Centre works in collaboration with 54 partner institutes around the world to assure the validity and relevance of data.

https://www.imd.org/wcc/research-methodology/  

8.  Political and Economic Risk Consultancy Asian Intelligence 2018 (PERC)

Type of assessment: Executive opinion survey of local and expatriate businesspeople.

All responses were either collected in face-to-face interviews or in response to emails directed to specific people obtained from different national business chambers, conferences and personal name lists. All respondents provided scores and comments only for the country in which they are currently residing. Respondents for each country include local business executives who are nationals of the countries, academics and expatriate executives. The data is available to subscribers under http://www.asiarisk.com/subscribe/exsum1.pdf

9. The PRS Group International Country Risk Guide 2018 (PRS)

Type of assessment:Risk assessment

ICRG staff collect political information and convert it to risk points on the basis of a consistent pattern of evaluation. Political risk assessments and other political information form the basis of ICRG risk ratings. It is therefore possible for the user to check through the information and data so as to assess the ratings against their own assessments, or against some other risk ratings system.

10. World Bank, Country Policy and Institutional Assessment 2017 (WB)

Type of assessment: Expert assessment

The ratings are the product of staff judgment and do not necessarily reflect the views of the World Bank’s Board of Executive Directors or the governments they represent. The Bank has prepared guidance to help staff assess country performance, by providing a definition of each criterion and a detailed description of each rating level. Bank staff assess the countries’ actual performance on each of the criteria, and assign a rating. The ratings reflect a variety of indicators, observations and judgments based on country knowledge, originating with the Bank or elsewhere, and on relevant publicly available indicators.

11. World Economic Forum Executive Opinion Survey 2018 (WEF)

Type of assessment: Business executive survey

The Forum’s Global Competitiveness and Benchmarking Network works closely with over 160 partner institutes that administer the survey in their respective countries/territories: well-respected economics departments of national universities, independent research institutes or business organisations. The surveys are conducted according to detailed guidelines aiming at collecting a sample stratified by sector of activity and company size.
See chapter 1.3 of the Global Competitiveness Report 2013-2014 for further details: www.weforum.org/gcr.

12. World Justice Project Rule of Law Index 2017-2018 (WJP)

Type of assessment: Expert survey

The Index’s rankings and scores are the product of a rigorous data collection and aggregation process. Data comes from a global poll of the general public and detailed questionnaires administered to local experts (in-country practitioners and academics with expertise in civil and commercial law, criminal justice, labour law, and public health).

13. Varieties of Democracy Project 2018 (VDEM)

Type of assessment: Expert survey

It is a collaboration among more than 3,000 scholars worldwide which is co-hosted by the Department of Political Science at the University of Gothenburg, Sweden and the Kellogg Institute at the University of Notre Dame, USA. With four principal investigators, fifteen project managers with special responsibility for issue areas, more than thirty regional managers, 170 country coordinators, research assistants, and 2,500 country experts, the V-Dem project is one of the largest social science data collection projects focusing on research.

*****

Allow me to improvise an impromptu reaction on the description above under number 3.

What can possibly be the objective value of a score given by a country expert, reviewed blind by a second country expert, then verified and discussed by regional coordinators with as purpose to ensure intra and inter-regional comparability in ratings? In addition, Bertelsmann Foundation has also instituted an extra layer of verification to ensure that the scores provided match the qualitative descriptions for each country. (Think about what this can mean?).

Allow me also to put a question mark under number 5 where is stated: ‘The Nations in Transit (NIT) surveys were produced by Freedom House staff and consultants. The latter were recommended by relevant authorities and are regional or country specialists.’
What do you think that will happen when in particulat countries you ask the ‘relevant authorities’ for recommendations to include particular researchers for a review?

Close reading of several of the other descriptions as given (and I repeat that the descriptions are not mine but literally drawn from the CPI-text), give enough reason for doubts about the final outcomes.

In general the conclusion cannot be otherwise than that obviously as well those that participated as researcher or enqueteur, as those who answered, belonged to a rather small part of the population. We are far from any representativity. Should we dare to say that most nationals of those countries will not recognise as facts what over their country and countryman has been reported as the truth?

In a summary table, see the above again:

From the website https://www.transparency.org/cpi2018 (scroll down) have been taken the 13 ‘Source descriptions’. From these some selected information is reproduced below showing that most CPI-data find their origin with the ‘high and mighty’ in the economy and society.

Table 1, CPI 2018, 13 data providers, and of each one: which ‘type of Assessment’ is done and ‘who does the work?’

Obviously, it is not women, youth, blue collar workers, migrant labour (although ‘expatriates’ in executive positions?), handicapped, poor, less-educated, those lacking power, etcetera, that are being asked what they see as corruption. Whose ‘perceptions’ count?                               
*****