New Zealand rockets up the anti-Corruption ratings: a non-Spinoff investigation

18:00 Jan 27, 2017 19

New Zealand is again atop the Transparency International list of countries with the lowest perceived corruption.

With an index score of 90 out of a possible 100, the perception of non-corruption has also increased (last year, when New Zealand ranked forth, with a lowly 88). It’s not that the rest of the world is getting worse, New Zealand's apparently getting better. What is New Zealand doing right? I decided to investigate.

The Transparency International Survey of Corruption perceptions takes data from a range of international surveys that, in part, have questions on corruption. There are thirteen in all, including The African Development Bank Governance Ratings, the Economist Intelligence Unit Country Risk Ratings and World Justice Project Rule of Law Index. Corruption is a factor in governance, in economic risk, and in the rule of law, so Transparency International takes the data from the corruption parts of those rankings, and brings that together in one place.

New Zealand features in seven of the thirteen reports (Transparency International will give a country a rating if it features in at least three). Transparency International takes the information it gets from each of those surveys, turns that into a score out of 100, and then averages those scores to determine each country’s score.

So, how did New Zealand’s average score increase so drastically, up from a paltry 88, to a stellar 90?

Let’s look at each of the reports.

In the World Economic Forum Executive Opinion Survey (which surveys business executives), our corruption score fell from 92 to 90. That’s not good.

According to the IMD World Competitiveness Yearbook, a survey of business executives by IMD Business School in Switzerland, New Zealand’s corruption score increased from 93 to 95. That’s the two point increase we need to find, but all it does is cancel out the fall caused by the presumably different business executives that the World Economic Forum used.

In the Global Insight Country Risk Ratings, an assessment by in-country specialists who are part of the consulting firm IHS (now IHS Markit), New Zealand’s corruption score was unchanged: 83 in both 2014 and 2015.

New Zealand was up two points in the Economist Intelligence Unit’s Country Risk Ratings (which uses the experts in the Research Arm of the Economist to construct rankings), going from 88 to 90.

Unfortunately, New Zealand was down in its corruption rating in World Justice Project’s Rule of Law index, dropping from 83 to 79.

And New Zealand was down by five points in its corruption score according to score of risk analysis company Political Risk Services’ International Country Risk Guide, falling from 98 points to 93.

How then did New Zealand’s score increase? If you’ve been counting along, that’s only six reports, and New Zealand featured in seven. In the Bertelsmann Foundation’s Sustainable Governance Indicators report, New Zealand’s corruption score increased from 81 to 99. This is the sole reason for New Zealand’s index score increased.

How did the Bertelsmann Foundation arrive at its conclusions? Transparency International describes it in the following way:

The Sustainable Governance Indicators (SGI) examine governance and policymaking in all OECD and EU member states in order to evaluate each country's need for, and ability to carry out, reform.

The indicators are calculated using quantitative data from international organisations and then supplemented by qualitative assessments from recognised country experts.

What changed between 2015 and 2016 that accounts for the upgrade? The people who prepare the Bertelsmann Foundation’s report have offered their own insights:

New Zealand is one of the least corrupt countries in the world. Prevention of corruption is strongly safeguarded by such independent institutions as the auditor general and the Office of the Ombudsman. In addition, New Zealand has ratified all relevant international anti-bribery conventions of the OECD and the United Nations. All available indices confirm that New Zealand scores particularly high regarding corruption prevention, including in the private sector.

This synopsis footnotes the Freedom House: Freedom in the World 2015 index. And what did that report have to say about corruption in New Zealand:

C. Functioning of Government: 12 / 12

New Zealand is one of the least corrupt countries in the world. It was ranked 2 out of 175 countries and territories surveyed in Transparency International’s 2014 Corruption Perceptions Index. However, scandals involving political donations from migrant Chinese businessmen have hurt the government’s image. In May 2014, Minister Maurice Williamson resigned amid allegations of intervention in a domestic violence case involving a Chinese businessman who had made political contributions. In another case, donations were made to the National Party by a Chinese firm, one of whose board members is the husband of the Justice Minister.

So why, did Transparency International raise New Zealand’s anti-corruption score increase in 2016? According to the authors of the reports they relied on (and the authors of the reports they relied on) it’s because New Zealand was highly ranked on Transparency International rankings in 2014.

Now that's funny, and points out a pretty big problem with all of these indexes (although at least the Transparency International index is not quite as ridiculous as the index that suggested that North Carolina is no longer a democracy). but it's not actually the reason.

The Bertelsmann Foundation is interested in sustainable governance, of which corruption is only a part of its assessment. Indeed, the data transparency international uses from their assessment comes from a single question:

To what extent are public officeholders prevented from abusing their position for private interests?

The recognised country experts were told this question addresses:

… how the state and society prevent public servants and politicians from accepting bribes by applying mechanisms to guarantee the integrity of officeholders: auditing of state spending; regulation of party financing; citizen and media access to information; accountability of officeholders (asset declarations, conflict of interest rules, codes of conduct); transparent public procurement systems; effective prosecution of corruption.

Helpfully, the experts tasked with answering this question were warned:

Note: Please be aware that the Corruption Perceptions Index (CPI) of Transparency International uses the data and information given in response to question D4.4 for their assessments. To avoid circularity of assessments, please do not base your evaluation on the CPI.

They were asked to give a score from 1 to 10, with scores explained as following:

A score of 9 or 10 signifies that “Legal, political and public integrity mechanisms effectively prevent public officeholders from abusing their positions.”
A score of 6-8 would show that “Most integrity mechanisms function effectively and provide disincentives for public officeholders willing to abuse their positions.”

Quantitatively, this appears to be the source of the difference in New Zealand’s Transparency International index score between 2015 and 2016.

So, presumably, that question saw New Zealand get a score of 8 in 2015 (which Transparency International converted to 81), and a 10 in 2016 (converted to 99)?

Actually, no. The Bertelsmann Foundation's assessor gave New Zealand a 10 in 2016, and a 10 in 2015, and a 10 in 2014. It's just that in 2016, this 10 was converted by Transparency International to a 99 on a 100-point scale, and in 2015, that 10 was considered to be worth 81 by Transparency International.

Transparency International explains how this works:

Standardise data sources to a scale of 0-100 where a 0 equals the highest level of perceived corruption and 100 equals the lowest level of perceived corruption. This is done by subtracting the mean of the data set and dividing by the standard deviation and results in z-scores, which are then adjusted to have a mean of approximately 45 and a standard deviation of approximately 20 so that the data set fits the CPI’s 0-100 scale. The mean and standard deviation are taken from the 2012 scores, so that the rescaled scores can be compared over time against the baseline year.

I know a number of people much better at statistics than me occasionally read and comment here, so I won't try to explain it. In the end, I doubt understanding the exact process will help. Although I suspect it gives lie to Transparency International's outline of the results as shocking because so many countries score below 50:

OUT NOW: Corruption Perceptions Index 2016. More than 2/3 of countries score below 50. Check it out here: https://t.co/V3UTLQOuDF #cpi2016 pic.twitter.com/mM82LMcQ9w
— Transparency Int'l (@anticorruption) January 25, 2017

So what explains New Zealand's rise?

In 2015, when New Zealand's Bertelsmann Foundation score of 10 was standardised as an 81, Denmark's score of 10 scaled to 97. Finland and Sweden, whom New Zealand beat on the Bertelsmann Foundation's measure (they received 9s) got scores of 91. The Netherland's score of 7 got scaled to 97 (oops). And Canada's 8 got them 81 like us.

Why did New Zealand rocket up Transparency International's Anti-Corruption ratings in 2016? And why did the Netherland's fall from fifth to ninth? There were data entry errors in Transparency International's 2015 analysis. Fix that mistake, and New Zealand's index score didn't go up from 88 to 90, it went down from 91 to 90. And New Zealand would have been first equal (relying on rounding) in both years.

Assuming, of course, there aren't more errors. And ignoring that these sorts of rankings are stupid anyway.

Update: Several months after this post was published, Transparency International issued an errata for their 2014 and 2015 Indexes. This was accompanied by a press release, and a post on Medium explaining how the error occurred.

19 responses to this post

Raymond A Francis, 20:06 Jan 27, 2017

Well it doesn't really matter which way you choose to dice the cake, NZ is near the top of the World rankings with regard to anti corruption which is good and the alternative view pushed by the "Standard" and Bomber is bollocks!
Hurrah, unicorns all round.

45' South • Since Nov 2006 • 578 posts Report
linger, 21:31 Jan 27, 2017

Though rankings are relative measures only: even achieving a global "least worst" still equates on an absolute scale to "could try harder".

Tokyo • Since Apr 2007 • 1944 posts Report
Ian Dalziel, 08:23 Jan 28, 2017

Thiel be right mate!
- it's the Kiwi way...

Christchurch • Since Dec 2006 • 7953 posts Report
andin, 08:34 Jan 28, 2017

these sorts of rankings are stupid anyway
very good analysis from the laymans point of view ie me.
All I heard was a mild bit of nationalistic horn blowing and once over, really lightly, in MSM and then it disappeared. Which I guess is all, they decided, the general readership needed or wanted to know.Now back to sports news

raglan • Since Mar 2007 • 1891 posts Report
Joe Wylie, 08:36 Jan 28, 2017

There is no corruption in New Zealand
We can all keep perfectly calm...

flat earth • Since Jan 2007 • 4593 posts Report
simon g, in reply to Raymond A Francis, 12:22 Jan 28, 2017

NZ is near the top of the World rankings with regard to anti corruption which is good and the alternative view pushed by the “Standard” and Bomber is bollocks!
The opinions of fringe bloggers don’t mean much at all. The views of the leader of the opposition matter a good deal more.
And when that opposition party was National, and its leader was Don Brash, they called Helen Clark’s government “corrupt” (not implied it, but explicitly said it). When it was pointed out that NZ was top of the very same world rankings (and it was) they said “oops, our bad” … well no, of course they didn’t, they just said “corrupt” again. (“Alternative facts” didn’t begin with Trump).
So, apples with apples, please.
(Edit: just one example among many) – https://web.archive.org/web/20070510021623/http://www.national.org.nz/Article.aspx?ArticleID=7645
"... the most corrupt government in NZ history"

Auckland • Since Nov 2006 • 1333 posts Report
David Hood, 14:37 Jan 28, 2017

Without actually going and looking at any of the data (so putting in considerably less work than Graeme) I would focus on
Although I suspect it gives lie to Transparency International’s outline of the results as shocking because so many countries score below 50
It does not give lie- It is going to depend on how skewed the data is (where the commonest range is and if there are extreme values), but this seems to be further complicated by the difference with 2012 as well so what I am about to say is a bit of a simplification. If you have a few extremely (score near 1) countries and the vast bulk of countries with low levels (rating in the 50s) you could have a mean of 45 and few countries with high corruption. Similarly you can have a lot of countries with badish scores (30s) and a few extreme well behaved countries and also have a mean of 45.
Rather than that, the reason it is not shocking is that it is basically the same as the past few years. For the percentage of countries that scored below 50: 2016 = 69%, 2015 = 68%, 2014 = 70%, 2013=70%
(OK I went and looked at the data for the last part)

Dunedin • Since May 2007 • 1445 posts Report
Thomas Lumley, in reply to David Hood, 18:59 Jan 28, 2017

Although I suspect it gives lie to Transparency International’s outline of the results as shocking because so many countries score below 50:
I'm going to partly disagree with David Hood and agree with Graeme here. The standardisation doesn't guarantee that a lot of countries will be below 50, but it certainly tilts the scale that way. Worse than that, it's possible that the proportion below 50 could increase even if all the changes that occurred were improvements (or the reverse).
The fact that improvements in one country will tend to lower the values (not just the ranking, which is inevitable) for other countries is more fundamentally wrong, though. Reducing corruption in other countries is good for New Zealand, not bad. This isn't rugby, where someone has to lose for us to win.

There can be good reasons for scaling an index differently each year. The motivation for 'grading on a curve' in the literal sense in large college classes is that each year's exams are completely different, so it's plausible the population is more stable than the difficulty of the test. If that's the case, scaling away year-to-year differences in mean and variance is sensible.
Here, the idea is to have the same inputs to the index each year, and it's quite plausible that there are changes in the population -- that there are trends in corruption either around the world or in large groups of countries. In that situation it doesn't make a lot of sense to use a different scaling each year.
On the other hand, it is still true that given the actually-existing scaling, a country does have to be pretty bad to get a grade below 50. It's just that this is a contingent fact about the scaling, and might not be true in a different, better world.

Auckland • Since Feb 2013 • 50 posts Report
Graeme Edgeler, in reply to simon g, 01:11 Jan 29, 2017

And when that opposition party was National, and its leader was Don Brash, they called Helen Clark’s government “corrupt” (not implied it, but explicitly said it). When it was pointed out that NZ was top of the very same world rankings (and it was) they said “oops, our bad” … well no, of course they didn’t, they just said “corrupt” again.
This arose from a very specific set of facts:
1. The Labour Party had spent a bunch of money they were not entitled to spend on Pledge cards and leaflets (this bit adds a little to the concern, but isn’t particularly noteworthy).
2. More importantly, the Electoral Commission advised them before the election that the card were election advertising, and that the money spent on them was an election expense.
3. Number 2, above, should have been no surprise to anyone. While there might have been confusion about whether they money was able to be spent on the pledge cards, there should have been no confusion whatsover that the material was election advertising and that its cost was an election expense. This was not a difficult call.
4. Knowing that this spending would count toward their spending limit, and knowing that it may already have put them over their spending limit, Labour decided that the rules oughtn’t to apply to them, and kept spending money on election expenses, knowing that this would mean they spent well in excess of the spending limit we put in place to ensure fair elections.
5. Deliberately spending more money on election expenses than the spending limit is a corrupt practice. It can result in prison, and anyone involved in the decision who was an MP could be convicted as a party to that corrupt practice, and would lose their seat.
6. The Electoral Commission referred this deliberate over-spending to the police. The Police interviewed the Prime Ministers chief of staff who told police that there was a dispute about whether the spending on the pledge card was permitted within Parliamentary rules, and that this meant no-one had deliberately overspent. This was false. Even if the auditor-general had ruled that the spending on the pledge card and leaflet were proper uses of Parliamentary funds, this would have made no difference to whether the spending counted toward the overall spending limit. It clearly did, and there is literally no argument against this.
Having been presented with a government that had deliberately committed a corrupt practice, for the purpose of winning an election, Don Brash and others called the government corrupt. The title was truly earned, and a bunch of people are lucky they didn’t end up in prison, or kicked out of Parliament.
The incompetence of the police, in accepting the legal interpretation of the Electoral Act given by a suspect over that offered by the Electoral Commission, and the short time-frames that then applied for filing prosecutions are the only reasons they got off.

Wellington, New Zealand • Since Nov 2006 • 3215 posts Report
LeighKennaway, in reply to andin, 11:30 Jan 29, 2017

Now back to sports news
Apparently Richie McCaw was asked to comment, but he is still away on honeymoon.

Lower Hutt • Since Feb 2016 • 9 posts Report
Lynn Yum, 20:22 Jan 29, 2017

I wouldn't say this index is "stupid", but I always thought TI's report is pretty much a PR exercise because it is a PERCEPTION index, not a measure of actual corruption. I'm sure the lower ranked countries are really corrupted, but perception and reality can come apart.
This is from their FAQ:
Why is the CPI based on perceptions?
Corruption generally comprises illegal activities, which are deliberately hidden and only come to light through scandals, investigations or prosecutions. There is no meaningful way to assess absolute levels of corruption in countries or territories on the basis of hard empirical data. Possible attempts to do so, such as by comparing bribes reported, the number of prosecutions brought or studying court cases directly linked to corruption, cannot be taken as definitive indicators of corruption levels. Instead, they show how effective prosecutors, the courts or the media are in investigating and exposing corruption. Capturing perceptions of corruption of those in a position to offer assessments of public sector corruption is the most reliable method of comparing relative corruption levels across countries.
Which is to say they don't really want to do the hard yards and find a better metric for measuring corruption.

Auckland • Since Dec 2016 • 38 posts Report
Sacha, in reply to Graeme Edgeler, 20:25 Jan 29, 2017

The incompetence of the police
Have they ever successfully prosecuted anyone under the Electoral Act? Seem to wimp out every time I've seen.

Ak • Since May 2008 • 19745 posts Report
izogi, in reply to Lynn Yum, 07:29 Jan 30, 2017

I wouldn't say this index is "stupid", but I always thought TI's report is pretty much a PR exercise because it is a PERCEPTION index, not a measure of actual corruption.
From past surveys I was sure I'd read somewhere that it was built from journalists' input and perceptions, which must be a false memory or maybe a different survey. Reading down Graeme's list, it (NZ's rating at least) seems to be far more weighted by perceptions of business leaders and executives, economists and other consulting firms whose primary clients are businesses.
This doesn't make it meaningless and ability to do business in a country isn't a useless metric for transparency, but it does (to me) seem more consistent with how the index might reach conclusions which seem at odds with a government's general conduct towards its people. eg. All the Dirty Politics stuff wasn't necessarily 'bad for business' as long as you were on the right side of it, and this current government and its senior Ministers have a strong emphasis on meeting with and listening to business lobbyists.

Wellington • Since Jan 2007 • 1142 posts Report
Lynn Yum, in reply to izogi, 13:17 Jan 30, 2017

I'm not sure if it is more biased towards business POV. TI's index is like meta index. Three of the source indexes are: Political Risk Services, World Justice Project and The Bertelsmann Foundation, which are statedly not catering to business but academics, the general public and governments.
I think it is more of a case of New Zealand banking on previous reputation. All the recent Dirty Politics stuff, all the gory John Banks/Kim Dotcom mess don't seem to register externally. Or maybe there are simple data entry errors as Graeme mentioned.
No one should be complacent about corruption in New Zealand, it should have no place here (or anywhere else) but it still happens. But I'm sure some politicians would like to pat themselves on their back having seen this TI report.

Auckland • Since Dec 2016 • 38 posts Report
David Hood, in reply to Lynn Yum, 08:44 Jan 31, 2017

I’m not sure if it is more biased towards business POV.
for generating the relative placing of countries between sources, they are imputing against the sources with more that 50% country coverage. So the most important sources, in order, are:
Global Insight Country Risk Ratings (covers all countries) 176 coverage
World Economic Forum EOS 141 coverage
PRS International Country Risk Guide 140 c0verage
Economist Intelligence Unit Country Ratings 129 coverage
Bertelsmann Foundation Transformation Index 129 coverage
World Justice Project Rule of Law Index 110 coverage

Dunedin • Since May 2007 • 1445 posts Report
Rob Stowell, 09:15 Jan 31, 2017

Dirty Politics, Panama papers, various scandals around political contributions have failed to move the 'perception'?
These rankings invite complacency and self-congratulation. We have little reason to be complacent.

Whakaraupo • Since Nov 2006 • 2120 posts Report
Joe Wylie, in reply to Rob Stowell, 09:53 Jan 31, 2017

These rankings invite complacency and self-congratulation. We have little reason to be complacent.
Too true. It's more than a little reminiscent of that brief fool's paradise atmosphere that existed following the Christchurch quake of September 2010, when NZ's "first world" regulatory standards were touted as having saved us from the kind of carnage that had happened in Haiti.

flat earth • Since Jan 2007 • 4593 posts Report
David Hood, 12:02 Jan 31, 2017

Incidentally, about 8% (depending on how you round it) of the movement in countries from 2015 to 2016 was caused by the inclusion of a new source.
Some R code:
https://github.com/thoughtfulbloke/transparency/blob/master/transparency.R

Dunedin • Since May 2007 • 1445 posts Report
Graeme Edgeler, in reply to David Hood, 17:58 Feb 2, 2017

Incidentally, about 8% (depending on how you round it) of the movement in countries from 2015 to 2016 was caused by the inclusion of a new source.
I knew there'd be a bunch of extra things going on in there :-)

Wellington, New Zealand • Since Nov 2006 • 3215 posts Report