A ride in a Baidu self-driving taxi

Baidu’s Apollo autonomous driving program has thrust the search giant into the spotlight. Named after NASA’s moon missions, the self-driving program recently enjoyed a series of wins when Baidu came out on top in annual self-driving reports released by authorities in California and Beijing.

Waymo has said the reports do not provide “relevant insights” or distinguish their company’s “performance from others in the self-driving space.” Kyle Vogt, the CTO of General Motors-backed Cruise, shared similar sentiments. “The idea that disengagements give a meaningful signal about whether an AV is ready for commercial deployment is a myth,” he wrote in a blog post.

Still, much is expected of Baidu’s self-driving efforts. The company has launched autonomous ride-hailing services in Changsha, the capital of Hunan province, as well as in Cangzhou, in north China’s Hebei province, with a fleet of 30 cars. Baidu’s autonomous driving tests have covered more than 3 million kilometers on public roads across 23 Chinese cities.

Where are Baidu’s self-driving cars headed in 2020? What is the outlook for Baidu in autonomous ride-hailing? We will start with our recent experience in a Baidu robotaxi in Changsha and move on from there.

A ride in a Baidu robotaxi

Robotaxis are all the rage. Around the world, startup and tech giants alike are fighting the war for self-driving supremacy, and autonomous taxis have become the new battleground.

Companies including Baidu, Pony.ai, and WeRide have launched robotaxi pilots across China. Baidu, the country’s designated self-driving champion, began offering its robotaxi service in Changsha last September.

Three months later, TechNode arrived in downtown Changsha. Standing outside a well-known culture and arts center on a sun-washed December afternoon, we waited for a Baidu self-driving taxi to pull up.

The trip showed us how companies are taking vastly different approaches to developing their self-driving technologies, and just how difficult it is to create global benchmarks detailing how these vehicles should perform.

Baidu runs its autonomous taxis in and around Changsha’s downtown Xiangjiang New Area. The trial operation is more of a geo-fenced test on public roads; passengers can pick one of three fixed five-kilometer routes, all starting from the city’s grand theater.

The tech giant has partnered with Chinese state-owned automaker FAW Group, which provides the vehicle for its autonomous system. As the luxury Hongqi model arrived to pick us up on that balmy December afternoon, we quickly took one photo before we were told that pictures were not allowed.

Shortly after we got into the car and entered Changsha traffic, Baidu’s approach to its self-driving program became evident. It was like going for a ride with a nervous student driver.

Companies that develop self-driving technology need to consider not only the safety of their passengers but also the comfort of the ride. Baidu places more emphasis on safety than we had expected, resulting in a trip that was less smooth than AV rides we’d experienced from companies that squeeze more efforts to the comfort of their passengers.

“Our top priority is zero accidents on the road,” our vehicle’s safety driver said while we waited at a traffic light. He offered a glimpse into how the company’s safety precautions are meant to protect the trial project from any sort of controversy. “All of us are required to take a 10-minute break for each hour of work,” the driver told us.

During our trip, Baidu’s robotaxi traveled at speeds of around 30 kilometers per hour and stopped by itself every now and then to yield to pedestrians. Traffic was heavy, with cars filling the six-lane Meixi Lake Road, downtown Changsha’s main avenue.

When the vehicle stopped at a red light in the middle of an intersection, we got to see firsthand the safety precautions that our driver had described: After a few minutes of waiting, the human driver had to take over. Situations like these are typically evaluated as “too risky” for the autonomous system to navigate. Baidu says it has reported “zero accidents” in the past few years because of its “safety-first” approach.

The company has requested that its fleet of dozens of vehicles in Changsha log a certain amount of mileage each day, our safety driver told TechNode, without revealing any further details. Meanwhile, working hours are very limited since the company has not been allowed to test during rush hour. Therefore, overtime work during weekends has become common.

A conservative driving strategy

Baidu is taking a more conservative approach to its AV road testing, emphasizing safety over comfort, a self-driving car engineer said, commenting on TechNode’s observations of our robotaxi ride.

Slower driving speeds, hesitation when turning or changing lanes, and constant stops when facing dangerous scenarios are among the passive driving strategies that result, the engineer said, who asked not to be named because he was not permitted to speak to the media.

A focus on safety, alongside a goal of fewer human interventions, can be achieved by developing a cautious algorithm, helped by some of the high-performance hardware that acts as the eyes of self-driving vehicles.

For years, safety and comfort have been among the top priorities for robotaxi companies offering driverless experiences. “No doubt safety is the key to getting autonomous cars on the roads,” but a better solution could be a wider range of driving styles with safety guarantees to ensure more comfort for passengers, the engineer said. There should have been some “more decisive driving policies” he said, referring to how the vehicle could have taken proactive measures to avoid dangerous situations, such as changing lanes.

Key metrics on AV testing

Baidu’s prudence could be part of the reason the company came out on top in the recent self-driving report released by California’s Department of Motor Vehicles.

Baidu beat Google’s self-driving unit Waymo by reporting the least number of disengagements among all companies operating such vehicles in the state. A disengagement is defined as any time a human driver is required to take over from an autonomous system during self-driving tests.

But within the industry, questions over the relevance of such metrics are on the rise, with experts saying that the measure has limits when trying to gauge whether a company’s technologies are ready to be deployed commercially.

AV companies themselves have also highlighted the report’s limited usefulness. In an announcement, Baidu said disengagement is more of an internal reflection of the speed of technical iterations, and therefore comparison between companies is “not that meaningful.”

However, if disengagement rates offer few relevant insights into the technology, what are the measurable metrics that could indicate progress? Two experts that TechNode spoke with gave the same answer: the variety and complexity of testing scenarios in which a robocar can operate.

Keeping within a lane in urban traffic, recognizing traffic signals, or turning left at an intersection without a “green arrow” traffic signal are some of the most typical and frequently seen scenarios identified and tested by AV players.

However, the real difficulty is to get autonomous cars to operate under “edge cases,” or unusual circumstances, such as a nearby vehicle changing lanes abruptly, a motorcycle coming out of nowhere, or drunk driving behavior from other road users.

These scenarios could be used to create a benchmark dataset that enables companies to train and evaluate their algorithms and compare accuracy rates to effectively evaluate their technologies, much like ImageNet, a renowned computer vision dataset of more than 14 million photographs widely used to evaluate the performance of AI systems.

“The more driving scenarios your cars can handle, the more you can prove the safety of the technology,” said one of the experts. Nevertheless, problems persist because the industry has not reached a consensus on standards.

The self-driving industry has now evolved from being driven by research and development of AV technologies to being mostly pushed forward by testing efforts. The development of key technologies, such as environment perception and car control, have mostly been completed; the priority now is to gain experience in as many driving cases as possible and learn how to deal with them, the experts added.

Every new experience helps a self-driving car to learn, and that’s where some of the world’s AV leaders are ramping up their efforts. Last year, Cruise almost doubled its testing and validation miles from the year prior, and “every mile Cruise tested in California was driven in the very complex urban environment of San Francisco,” it said in its individual filing.

The company, which is mainly backed by General Motors, operates a fleet of 228 vehicles that drove more than 831,000 miles last year, nearly eight times that of Baidu. As of last December, the Chinese search giant claimed its vehicles had traveled a total of more than 3 million kilometers (1.86 million miles).

But wider tests in China are coming as more local governments join in the race to open their roads to robotaxi companies, allowing them to collect more data and develop better evaluation methods. We’ll have to wait and see who comes out in pole position.