You've just finished your Uber ride. By the general standards of taxi rides, it was very good.
The driver came fast, the car was newish and in good condition and tidy inside. The driver was courteous and asked how you were. Because you felt talkative, you had a discussion. Because you were having a discussion in a language that they were not native to, they were a little distracted, but not in a way that was particularly dangerous, and since they followed Google anyway, you can be pretty sure the route was a reasonably good one, except perhaps that last bit you had to talk them through as you drove down a long right-of-way that's not on any map.
At the end you jumped out and it was all done. As taxi rides go it was one of the good ones. But you've used Uber many times, and it definitely wasn't your best ride ever. What? No free water, no aux cable, no breath mint? No heated leather seats? The conversation was all right, but they didn't really show that much knowledge or interest in your life. Their banter was only at a canter. So should you 5 star them, or give them an average score?
Tough one. What is an average score? Normally, when the scale is from 1 to 5, the average score would be 3. A 5 would mean exceptionally good. You'd expect on this kind of scale for driver ratings to be normally distributed around a mean of 3. And you'd be half right. The ratings are approximately normally distributed around a mean. The mean just doesn't happen to be 3. It's somewhere between 4.5 and 5.
I collected the ratings of 25 drivers in our association, and the mean score was 4.8, with a standard deviation of 0.07.
Here's how that looks on the 1 to 5 scale.
This data is probably on the high side. Uber's own meagre insight is that the top 25% of drivers have an average rating of 4.79. We don't get the actual mean or variance, these are only known by Uber.
So how do you rate an average driver on a scale of one to 5, when the average driver rating is 4.8 (say). If you give:
- 5. Their mean will go up by 0.0004. Effectively this does almost nothing.
- 4. Their mean will go down by 0.0016. Still not much, but 4 times as much movement as 5
- 3. Their mean will go down by 0.0036. This is 9 times as influential on their rating as a 5
- 2. Their mean will go down by 0.0056. This is 14 times as influential on their rating as a 5
- 1. Their mean will go down by 0.0076. This is 19 times as influential on their rating as a 5
For any score below 4 you have to give a reason. Since it will take 9 consecutive perfect scores of 5 to undo the effect of your 3, this is fair, although it might be puzzling to anyone accustomed to 3 meaning "average".
So how do you signal that your driver is good? Currently, there is only one way with the app. You give them a 5 and you also leave a nice comment. The comment won't affect their rating, but it will show up on their main home screen and make them feel good.
Outside of the app you could signal it in two main ways - you could say something nice to the driver (for most people this is less effort than writing a comment), or you could even give them a tip. But this is NZ. Most people never tip, on principle.
Why is it this way? Isn't it rather perverse to have a system in which you can really only signal average, bad, really bad, shockingly bad, and fire-this-person? That depends on what you think the purpose of the system is. Clearly, it's not to reward the driver, because there is no way to reward them with anything other than keeping their job, and perhaps the afterglow of a kind word. Clearly, the whole purpose is to punish drivers to varying degrees.
Drivers are, unfortunately, very much in the dark about the specifics of their own ratings. We never know which trips received bad ratings or complaints. We don't even know how many of each rating value we have. The only numbers given are: How many 5 stars, how many rated trips, how many total trips, and what the mean rating is across the last 500. We can also see the top category of complaint.
What do I mean by punishing? Is a bad rating that much of a problem?
I have received many distressed phone calls in the last few months from drivers who have been disconnected permanently for low ratings. Uber's process is to "deactivate" drivers at will and without warning, and then when they inquire as to what has happened, they are told that they have to go on a course at the driver's expense, after which they are given a chance to improve or face permanent deactivation.
No specific target rating is ever given, the "city minimum average" is unknown, but it would seem that anyone below 4.6 in Auckland is at risk. The subsequent deactivation is given with a brutal message "Your final payment will occur within the next week. There is no need to come into the office, as our decision will stand." Attempts to come to terms with what has actually happened to the rating, what bad trips happened, where and when, are stonewalled. There is no natural justice whatsoever applied to this.
This was the fate of Sreeman, a veteran driver of over 1000 trips, on Sunday. Having a rating of 4.49 he had been deactivated and asked to do the training course to give him pointers about how to get a good rating. He accepted that the way he'd been doing it before was not as good as the average Uber standard and set about following the instructions given. His mean ratings over the last three weeks were around the 4.7 mark (he was careful to take some screenshots). But since he was fighting the dead weight of a 4.49 score based on 500 previous trips, it was clearly going to take him quite some time to fight his way back to an acceptable score. He fought, getting steady improvement:
Despite "Smashing It", on the 17th of September he was deactivated on a rating of 4.53.
No entreaty given to the staff in the office made a lick of difference. It did not matter that he had done their course and that he had significantly improved since then. It did not matter that he had taken on a significant investment in choosing to work for Uber, specifically buying a Prius, and gaining all of the compliance possible, the P Endorsement, his own PSL, a CoF for the vehicle. It certainly didn't matter that this was his only job and that he relied on it to support his family. His reduction to tears and begging made them a little uncomfortable and they tapped away at the screen for a while, before calling security to have him escorted from the premises. Now he can't support his family of 4, and is seeking work.
This, dear readers, is why Ubers are high quality. Because they have a rating system and termination processes that resemble no workplace practices you'd see in NZ. The refusal to ever give a useful mean and variance on driver ratings is not so strange when you consider it could be used to work out just how arbitrary their disconnection practices are. Are they ridding themselves of the bottom 1%? Or 10%?
Uber drivers are no longer called "Partners" – in the last few months that phrasing was quietly dropped.
Is this our model for the grand future of employment? Consider how that might work in your own workplace, for your own job.
PS: Ironically, on the way to meeting Sreeman, Arden MacDonald from the NZUDA used an Uber with a 4.3. I'm a little confused about where the Auckland minimum average rating must be, since 4.53 is clearly below it, but 4.3 doesn't seem to be.
