ChatGPT has become the talk in China’s tech and business communities these days, with major Chinese tech companies racing to prove they have a similar capability or are developing similar services. TechNode talked to John Zhang, CEO of StarBitech, a digital asset startup based in Shanghai and supported by Microsoft for Startups, on why Chinese tech majors are rushing to push out their own versions of ChatGPT. Below is an edited version of the conversation.
1. Why are Chinese tech companies developing their own AI chatbots like ChatGPT? For example, Baidu announced last week that its look-a-like product, ERNIE Bot, or Wenxin Yiyan in Chinese, will be launched in March.
There are three reasons for this. First, from a market perspective, ChatGPT is currently not available to Chinese users. They can’t use it as easily as overseas users. So it’s inevitable that there will be a local ChatGPT-like service to satisfy demand.
Second, from a technological perspective, most large language models (LLMs) currently available on the market, like ChatGPT, are trained on English as the primary language. Their natural language processing (NLP) performance in Chinese is still inferior to that of English. So a model trained with Chinese as the primary language will further improve user effectiveness.
The third reason is data security. AI generates content after going through a large amount of data training. And OpenAI seems to gradually shift from being a non-profit project to a market-oriented one, so there could be uncertainty in the future. Additionally, mainland China requires all data to be locally stored, but OpenAI does not have a team in the country, making it difficult to meet regulatory requirements for local data storage and maintenance.
2. Can China’s AI chatbot compete with ChatGPT and its peers?
In the short term, it’s still difficult for Chinese AI chatbots to compete. OpenAI entered the stage of large-scale GPU cluster training after getting investment from Microsoft. It’s said that OpenAI owns thousands of Nvidia A100 chips, and Microsoft’s billion-dollar investment was mostly in Microsoft’s Azure cloud resources. Microsoft and OpenAI have just begun the next round of financing and collaboration, which means that in three years, they have burned billions of dollars in cloud resources on training. Such a large-scale investment is very rare in China’s internet circle, especially in underlying infrastructure technology. Most of the big investments in China are more focused on the application side.
But in the long run, China’s AI chatbot will become more powerful in the future. The country has superior algorithm engineers, a unified large market, abundant application scenarios, and data sources, and cost advantages over Microsoft Azure compared to Alibaba Cloud and Tencent Cloud.
3. Do you think China is ready in terms of big data and language models?
In terms of big data, China is ahead of the game. It’s highly digitized, so has access to abundant data and a complete industrial chain. However, when it comes to language models, there’s still room for improvement. Currently, models like GPT-3.5 used in chatGPT are large models that require significant investment and are slower in seeing returns, which isn’t an attractive option for many Chinese investors. As a result, only a few major internet companies have participated, with limited investment, slowing China’s progress in language models. But the popularity of ChatGPT offers a good warning for both Chinese investors and internet companies. I expect to see larger investments in the future.
4. How would Chinese AI chatbots differ from others, regarding application and regulations?
Currently, in China, large-scale chatbots are applied in NLP tasks such as machine translation, intelligent customer service, and Q&A platforms. As the development of LLM progresses, China will also popularize AI chatbots based on LLM.
AI chatbots developed in China should be: first, eloquent in Chinese expressions. That is, they need to be able to understand Chinese commands. In addition, for a better communication experience, the chatbot must have knowledge of Chinese culture and history, and communicate in a way that fits the Chinese language style and expression. For example, the same word may have different meanings and emotions in different contexts. Furthermore, the chatbot will provide more personalized services based on Chinese users’ habits and needs, such as different payment methods or ethnic customs unique to China.
Chinese-developed chatbots also need to comply with Chinese laws and regulations, including its Data Security Law, Cybersecurity Law, Personal Information Protection Law, and Administrative Measures for Internet Information Services. These laws aim to protect personal information (prevent its illegal acquisition, use, and dissemination), prevent information leaks and misuse, safeguard network security, prevent network attacks and fraudulent activities, and regulate internet information services. With the increasing popularity of chatbots and the continuous improvement of Chinese laws and policies, it is expected that more comprehensive and targeted regulations will be developed in the future to regulate chatbots.
5. Has your team used GPT (Generative Pretrained Transformer, OpenAI’s language model upon which ChatGPT is developed)? What challenges and limitations do you see with this tool?
- Biases. The model is trained on a large amount of text data. If trained data contains biases, the model will also exhibit them. For example, if there is a lack of Chinese language data, particularly in Chinese history, culture, and society, the model may output biased information.
- The model lacks a broad, bird-eye view perspective. Although GPT can maintain a sense of coherence in context, it lacks the ability to think more broadly.
- Lack of language diversity. GPT is trained mostly based on English material, limiting its compatibility and understanding of other languages.
- High computation cost. GPT is a very large neural network model, with parameter counts ranging from millions to tens of millions. The model size ranges from tens of megabytes to several gigabytes, going up to hundreds of gigabytes. Training such a model costs a significant amount of computing resources and time.
6. Has your team used any China-developed AI language models? How do they compare to GPT?
Currently, with self-developed Chinese AI language models:
- Some can support different voice responses, which are not currently supported by GPT.
- Regarding language support, there is a greater focus on Chinese-language communication, while GPT has a deeper understanding of English.
- In the application field, Chinese models are more narrowly focused on dialogue generation. To compare, GPT is a language generation model that can be used in text generation, code writing, and more.
- In terms of communication, Chinese models tend to deliver short-sentence communication, while GPT has a strong understanding of long sentences.
7. What are some features or functions that your team would like to achieve using AI language models, but have yet to do?
Current AI-powered chatbots may have achieved impressive results, but there is still room for improvement. One area is the understanding of context and emotions. Chatbots have a limited understanding of things such as one word having different meanings based on the context.
Another issue is that chatbots can lack coherence in continuous communication on the same topic. Moreover, they lack creativity, as they primarily integrate and sort existing knowledge. This means they do not meet the requirement for independent thinking and creating new ideas.
8. Could you give us an introduction to your company?
StarBitech is a digital content asset technology company founded in 2015. It is jointly invested in by the Shanghai Tree-Graph Blockchain Research Institute and Fengyuzhu and is located at the Microsoft Accelerator in the Caohejing Development Zone in Shanghai. The company focuses on providing individuals and businesses with algorithm-driven digital asset creation and publishing services. StarBitech has worked with companies such as China Merchants Bank, Huawei, LVMH, Shanghai Public Security Jing’an Branch, and the Shanghai Technology Exchange.
The company has recently received support from Microsoft and OpenAI and will leverage its strengths in Chinese natural language processing and local compliance to develop AIGC (AI-generated content) services in fields such as chatbots, visual content creation, and marketing content creation. These services will be supported by GPT, DALL-E, and reinforcement learning, providing AI capabilities for industries such as marketing, gaming, animation, culture and tourism, and government.