When a young mother from Chengdu wanted to return home from a visit to Beijing in May 2016, the only option she has was to travel for 20 hours in a rickety train to complete the 1,800-kilometer journey.
The woman, who told reporters her surname was Wei, had been put on a government blacklist that prevented her from purchasing certain items and services that required identification verification—including tickets for air and high-speed rail travel.
Wei, who had divorced a year earlier, had become entangled in a legal dispute with her ex-husband who, unbeknownst to her, had filed a suit against her over visitation rights to their son.
Much has been written about China’s emerging tools for social control. But few topics have garnered as much attention as the country’s nascent Social Credit System, a framework to monitor and manipulate citizen behavior using a dichotomy of punishments and rewards.
The idea is simple: By keeping and aggregating records throughout the government’s various ministries and departments, Chinese officials can gain insight into how people behave and develop ways to control them.
The goal writes Rogier Creemers, a postdoctoral scholar specializing in the law and governance of China at Leiden University in The Netherlands, is “cybernetic” behavioral control, allowing individuals to be monitored and immediately confronted with the consequences of their actions. In so doing, authorities can enhance the county’s expanding surveillance apparatus.
Some draw comparisons to the British/US science fiction television series Black Mirror and its speculative vision of the future. Others see parallels with dystopian societies penned by 20th-century writers such as George Orwell. In nearly all cases, the labels of the Social Credit System have been misappropriated.
Despite its name, it isn’t a single system, and it’s not monolithic, as many reports claim. Not every one of the country’s 1.4 billion citizens is being rated on a three-digit scale. Instead, it’s a complex ecosystem containing numerous subsystems, each at various levels of development and affecting different people.
Blacklists—and “redlists”—form the backbone of the Social Credit System, not a much-debated “social credit score.” Blacklists punish negative behavior while redlists reward positive. According to the planning outline released by the State Council, China’s cabinet, in mid-2014, the system’s objective is to encourage individuals to be trustworthy under the law and dissuade against breaking trust to promote a “sincerity culture.”
China’s Social Credit System: AI-driven panopticon or fragmented foundation for a sincerity culture?
Even so, an intricate web of social credit systems is coming to China—only perhaps not in the way, or at the speed, that’s generally expected. Many obstacles curb the implementation of a fully-fledged national system, including inadequate technology, insular mindsets among government ministries that jealously guard their data, and a growing awareness of the importance of privacy among China’s educated urban class.
Early experiments
The concept of a system of social credit first emerged in 1999 when officials aimed to strengthen trust in the country’s emerging market economy. However, the focus quickly shifted from building financial creditworthiness to encompass the moral actions of the country’s enterprises, officials, judiciary, and citizens.
More recently, in 2010, Suining County, in eastern China’s Jiangsu Province, began experimenting with a system to rate its citizens. Established to quantify individuals’ behavior, points could be deducted for breaking laws, but also for deviating from social norms and political positioning. Residents were initially awarded 1,000 points. Running a red light, driving while drunk, bribing a public official, or failing to support elderly family members resulted in a 50-point deduction.
The total would be then be used to assign an A to D rating. A-ratings were above 970 points, while those with less than 599 points were given D-ratings. Lower-rated citizens had a harder time accessing social welfare and government housing. More than half of an individual’s points related to social management.
Residents and the media lambasted the system, saying the government had no right to rate the country’s citizens, let alone use public services as a means of punishment and reward. To make matters worse, it was also compared to the “good citizen” identity cards that were issued by the Japanese to Chinese citizens as a form of social management during World War II. City officials eventually disbanded the A to D rating. State-run media outlet Global Times later referred to it as a “policy failure.”
Rising from the ashes of that disastrous experiment, new models for rating individuals have emerged around China. There are over now over 30 of these cities, despite there being no mention of assigning quantitative ratings in the 2014 planning outline. This highlights how the details of implementation are left to local governments, resulting in a scattered application.
In Rongcheng, Shandong Province, each of the city’s 740,000 adult residents start out with 1000 points, according to a report by Foreign Policy. Depending on their score, residents are then rated from A+++ to D, with rewards for high ratings ranging from deposit-free shared bike rental and heating subsidies in winter.
The city of Shanghai is also experimenting with social credit. Through its Honest Shanghai app residents can access their rating by entering their ID number and passing a facial recognition test. The data is drawn from 100 public sources.
Xiamen, a city in the eastern province of Fujian, has launched a similar system. Adults over 18 years old can use the Credit Xiamen official account on popular messaging app WeChat to check their scores. Those with high scores can skip the line for city ferries, and don’t need to pay a deposit to rent shared bikes or borrow a book from the library.
Jeremy Daum, a senior fellow at Yale Law School’s Paul Tsai China Center who has translated many of the government’s social credit-related documents, said that systems rating individuals—like the ones in Rongcheng, Shanghai, and Xiamen—have little effect since very few people are aware of their existence.
The scores are meant to form part of an education system promoting trustworthiness, says Daum. “This is supposed to get people to focus on being good,” he says. If punishments do occur, they are because of violations of laws and regulations, not “bad social credit,” he said.
In the 1990s, China went through a period of radical reformation, adopting a market-based economy. As the number of commercial enterprises mushroomed, many pushed for growth at any cost, and a host of scandals hit China.
In an editorial from 2012, Jiangxi University of Finance and Economics professor Zhang Jinming drew attention to the emerging appearance of low-quality goods and products and their effects on the populace. “These substandard products could result in serious economic losses, and some may even be health hazards,” he wrote.
In 2008, for example, contaminated milk powder sickened nearly 300,000 Chinese children and killed six babies. Twenty-two companies, including Sanlu Group, which accounted for 20% of the market at the time, were found to have traces of melamine in their products. An investigation found that local farmers had deliberately added the chemical to increase the protein content of substandard milk.
In 2015, a mother and daughter were arrested for selling $88 million in faulty vaccines. The arrests were made public a year later when it was announced that the improperly-stored vaccines had made their way across 20 provinces, causing a public outcry and loss in consumer confidence.
A question of trust
Incidents like these are driving the thinking behind the Social Credit System, Samm Sacks, a US-based senior fellow in the Technology Policy Program at the Centre for Strategic and International Studies (CSIS), who has published extensively on the topic, told TechNode. The idea is that greater supervision and increased “trust” in society could limit episodes like these, and in turn, promote China’s economic development.
China Tech Talk 59: China’s cybersecurity law: GDPR for the Middle Kingdom with Samm Sacks
The most well-developed part of social credit relates to businesses and seeks to ensure compliance in the market. Has your company committed fraud? It may be put on a blacklist. Along with you and other representatives. Have you paid your taxes on time? The company may be placed on a redlist, making it easier to bypass bureaucratic hurdles.
Government entities then share industry-specific lists and other public data through memorandums of understanding. This creates a system of cross-departmental punishments and rewards. If one government department imposes sanctions on a company, another could do the same within the scope of their power.
If a company were added to a blacklist for serious food safety violations it could be completely banned from operating or be barred from government procurement. Companies on redlists face fewer roadblocks when interacting with government departments.
A critical feature of the system to link individuals to businesses, explains Martin Chorzempa, a research fellow at the Peterson Institute for International Economics, based in Washington, DC. The idea is that while companies are supervised in their market activities, executives and legal representatives are also held responsible if something goes wrong.
But it’s not just business people that can be included on blacklists, as Wei, the young mother from Chengdu, found out.
One of the most notorious blacklists is the “List of Dishonest Persons Subject to Enforcement.” Reserved for those who have willfully neglected to fulfill court orders, lost a civil suit, failed to pay fines, or conducted fraudulent activity. Punishments include bans from air and high-speed rail travel, private school education, high-end hotels, and purchasing luxury goods on e-commerce platforms. Other sanctions include restrictions from benefiting from government subsidies, being awarded honorary titles, and taking on roles as a civil servant or upper-management at state-owned enterprises.
Jia Yueting, former CEO of embattled conglomerate LeEco, also landed on the blacklist in December 2017. Six months later he was banned from buying “luxury” goods and travel for a year—including air and high-speed rail tickets. He had failed to abide by a court order holding him responsible for his debt-ridden company’s dues. Jia fled to the US in late 2017 and defied an order to return to China. He has been back in the news recently after becoming embroiled in a battle with a new investor in Jia’s electric vehicle company Faraday Future.
Blacklist boom
It is uncertain whether the government is incorporating private sector data in social credit records. However, information does flow the other way. Companies like Alibaba and JD.com have integrated blacklist records into their platforms to prohibit defaulters from spending on luxury items.
Reports claiming that the social credit scoops up social media data, internet browsing history, and online transaction data conflate the government’s systems with commercial opt-in platforms like Ant Financial’s Sesame Credit.
Despite being authorized by the People’s Bank of China (PBoC), Sesame Credit is distinct from the government system. The platform, which is integrated into Alipay, rates users on a scale of 350 to 950. Those with higher scores gain access to rewards, including deposit free use of power bricks and shared bicycles, as well as reduced deposits when renting property. It functions like a traditional credit rating platform mixed with a loyalty program. The company was not willing to comment on social credit.
Experts believe that the collection of data by the government is currently limited to records held by its various departments and entities. It is information the government already has but hasn’t yet shared across departments, says Chorzempa.
Liang Fan, a doctoral student at the University of Michigan who studies social credit, explains that he is aware of 400 sources of information, although the total number of types of data that are compiled is unknown to him.
Nonetheless, private industry is picking up on signals from the government, some implicit and others explicit. Private credit systems have been developed off the back of the government’s broader plan. The PBoC was integral in the development of these systems. Although information might not be shared, the companies are benefiting from the troves of data they collect.
The lifeblood of social credit is data. And China has heaps of it. But there are still significant threats to the development of a far-reaching social credit system. Honest Shanghai app users have reported problems ranging from faulty facial recognition tech to the app just not accepting their registration.
“The user experience is terrible. I can’t verify my real name and it failed when I scanned my face,” said one of numerous similar reviews in the iOS App Store. Many of the reviewers posted one-star ratings.
But there exists a much more entrenched problem—individual government departments don’t like sharing their data, says Chorzempa. It holds significant commercial and political value for those who control it. This creates enormous difficulty when attempting to set up a platform for cross-departmental sharing. While there is a national plan to set up a centralized system for the coordination of data, there are currently no notable incentives for sharing. In addition, creating a broader system results in more labor for individual departments, with agencies essentially taking on more work for the benefit of others.
Other challenges are societal. Reports about the proliferation of the social credit system often ignore an important factor that could hinder its overreach: the agency of Chinese individuals. There is a growing awareness of how private data is used. This was evident in the Suining experiment and could have more wide-ranging effects for social credit. “It’s not the free-for-all that it may have been even in 2014 when the social credit plan was released,” said Sacks of CSIS. “There’s been a change in ways that could make aspects of that system illegitimate in the eyes of the public.”
Someone to watch over
Real-name verification is essential for social credit. Everyone in China is required to prove their identity when buying a SIM card, creating or verifying social media accounts, and setting up accounts for making online payments, in part, is dictated by the 2017 Cybersecurity Law.
Everyday activities are being linked to individual identities with more success, reducing anonymity, says Daum. He believes that’s what the government is doing with social credit. “They’re saying: ‘First, we need a system where people are afraid to not be trustworthy. Then we need a system where it’s impossible to not be trustworthy,’ because there’s too much information on you.”
For Wei, the blacklisted woman in Chengdu, it wasn’t the prospect of an arduous cross-country rail journey that bothered her. Instead, she was fearful that her future actions and freedom could be restricted by her past record. What if, for example, her employer wanted her to go on a business trip?
In the late 1800s, British social theorist Jeremy Bentham proposed the idea of a panopticon—an institution in which a single corrections officer could observe all inmates without them knowing whether they were being watched. In the Social Credit System framework that is emerging in China, the lack of anonymity, through both real-name verification and publicly-published blacklists, creates a system of fear even if no one is watching—much like Bentham’s notorious panopticon.