Chinese apps are letting public juries settle customer disputes
This story first appeared in China Report, MIT Technology Review’s newsletter about technology in China. Sign up to receive it in your inbox every Tuesday.
Have you ordered food delivery lately?
If you have, you probably know that particular feeling of frustration when you have to wait too long for your order or, when you finally receive it, the food isn’t what you asked for. These feelings are then often exacerbated by the difficulty of trying to make things right via app.
Meituan, the most popular food delivery app in China, has proposed one solution: inviting ordinary users to serve on “juries” that weigh in on disputes between other customers and restaurants. It could be anything from missing rice to not-spicy-enough noodles to the food being completely cold.
And beyond helping resolve grievances for others, it turns out users are having quite a bit of fun being “cyber judges,” as they’ve been calling themselves.
The feature isn’t new, but it’s getting a lot of attention right now on Chinese social media, especially for particularly ridiculous disputes—a trend I wrote about earlier this week.
While the jury function has evolved a bit since it launched in 2020, its essence remains the same: it’s a way for merchants to appeal customers’ bad reviews that they believe are fraudulent or malicious. If a restaurant chooses to open a case, the jurors can read the customer’s review, evaluate the original order and delivery records, and consider additional information provided by the restaurant before casting a vote for one side. If they find in favor of the restaurant, the review is removed.
Some of the jurors are very serious about it: they look closely at the order notes, zoom in on the photos of food uploaded by the customers, and scrutinize the delivery time stamps. Others are more interested in getting a good laugh out of the complaints; it’s like judging an “Am I the Asshole” case on Reddit, except it’s mostly about food.
To know more about how jurors and restaurants feel about this program, you can read my story here.
As I mention in the piece, Meituan is not the only popular Chinese app to have tried crowdsourcing platform management decisions. Alibaba, Tencent, Baidu, Douyin, Zhihu—basically every successful app in China has had its own version of the feature.
I’m fascinated by these experiments that give power back to the users and let them make decisions on platform rules and in individual disputes. While most of the jury features were discontinued after just a few years, they nevertheless provide interesting insight into basic human behavior on the internet and the nature of grassroots governance.
One of the oldest programs was from Alibaba, the e-commerce giant. From 2012 to 2018, Alibaba allowed any user, whether a seller or a buyer, to vote on inappropriate behaviors on the platform or in transaction disputes from unsatisfied buyers. By the end of the program, this jury system had processed over 16 million cases, with over 1.7 million users casting over 100 million votes.
The system had some built-in mechanisms to prevent people from abusing it by judging cases where their personal interests were involved. There was a multi-step randomized distribution system that made it impossible to predict what case you’d get assigned, and the app proactively suspended users who kept skipping over cases. (It also gamified its design, in which jurors could get experience points and level up after they judged more cases. It meant nothing in real life, but you got bragging rights for being able to say you were among “the top 100 jurors around the world.”)
Even though the exact program doesn’t exist anymore today, the data it generated can teach us more about how these crowd-voting systems really work.
Angela Zhang, a law professor at the University of Hong Kong, wrote a paper in 2021 with two fellow academics in which they studied 630,000 jury cases on Alibaba in a 20-month period. Over 150,000 jurors participated in these cases; more than 80% of them were buyers on the platform.
Zhang’s analysis found that crowdsourced dispute resolution is significantly more efficient than the alternative. While a normal complaint would take three to four business days for the platform to process, the online jury normally took about 73 minutes to reach a decision. (However, it can take some time to secure a jury trial; for instance, one merchant told me that could take days on Meituan.)
But the researchers also found a major problem: bias. In a buyer-seller dichotomy, people are more likely to vote for the group they belong to. On average, a juror who sold on Alibaba’s platform was 10% more likely to vote for the seller’s side than a juror who was also a buyer. That bias increased when jurors saw an ambiguous case or after they witnessed a few cases in which their own “side” lost.
Perhaps that’s not surprising. Participants in the program didn’t need to go through intensive jury training; nor were there any real accountability mechanisms to check their decisions.
Zhang found the best way to minimize bias was for jurors to spend more time voting on these disputes. It can also be reduced if the platform institutes a more sophisticated distribution system that diversifies the jurors on a case.
Zhang, whose book on Chinese online platform regulation will be published next year, remains hopeful about the feature as a way to make consumer internet platforms better.
“I view these crowd-judging systems as a form of platform decentralization. Essentially, platforms are delegating some of their authority to their users, creating a more collaborative and democratic governance structure,” she tells me. “Meituan’s crowd-judging feature is not only helpful in improving the efficiency of dispute resolution but also in bolstering the legitimacy of the platform’s decisions.”
Metuan’s system is only three years old now, half the age of Alibaba’s jury system when it ended. There hasn’t been any similar study on Meituan’s jury data yet, but I imagine the app would experience some of the same difficulties with bias and abuse that Alibaba did.
Though perhaps the newfound public interest in the program will inspire efforts to improve the design. I certainly look forward to seeing more attempts to create grassroots governance on Chinese apps, and I definitely won’t hate seeing more fun screenshots of heated customer-merchant debates either.
What do you think of the crowd-judging features on Chinese apps? Do you think other tech companies could learn from them? Let me know at [email protected].
Catch up with China
1. One year after Chinese people took to the streets to protest rigid zero-covid policies, participants are reflecting on the movement’s impact on themselves and the country. (Associated Press)
- Personally, I can’t believe it’s been a full year since I wrote about the protest for the first time. (MIT Technology Review)
2. Shein, Gen Z’s favorite ultrafast-fashion retailer from China, has confidentially filed to go public in the US. It was last valued at $66 billion. (CNBC)
3. Meanwhile Temu, the other Chinese e-commerce website bombarding you with ads, is still burning through cash and straining its logistics systems to compete with Shein, TikTok, and Amazon. (Wired $)
4. As Apple moves its manufacturing capacity from China to India, many Chinese engineers have been sent abroad to train Indian workers. (Rest of World)
5. An investigation using satellite imagery shows how nearly 2,000 mosques in China have gone through modifications since 2018 that removed Arabic features and sometimes added traditional Chinese designs. (Financial Times $)
6. Geely, the Chinese automaker that owns the brand Volvo, signed a deal with Chinese electric-vehicle startup Nio to make more EV models that rely on battery swaps. (Reuters $)
- My colleague Casey Crownhart has written about the rise of battery swapping as a way to charge EVs more efficiently. (MIT Technology Review)
7. China’s southern island province Hainan has almost finished building the first phase of an underwater data center that uses ocean water to cool down the computers. (New Scientist $)
Lost in translation
This Black Friday and Cyber Monday, did you order anything from Chinese e-commerce platforms? As the Chinese publication Ciwei Gongshe reported, companies like TikTok, Shein, Temu, and AliExpress have all thrown themselves into the frenzy of the US shopping battlefield, offering extensive promotions and flash sales. They have begun to pose significant challenges to traditional players like Target and Costco, many of which saw a decline in sales this past November.
Chinese companies may be particularly well suited for this competition. This year, Black Friday sales have felt more extended, with many websites starting their discounts as early as late October. The Chinese players are already good at fighting long campaigns: Black Friday–like events in the country have traditionally dragged on for weeks or even months.
One more thing
Wow, this urinal in Shanghai can conduct a health test of your urine and send you the result on WeChat in two minutes.
If I had a nickel for every time a Shanghai public restroom installed unnecessarily intelligent equipment that knows too much information about the users, I’d have two nickels. Which isn’t a lot, but it’s weird that it’s happened twice.
