Big data has exploded over the past few years. It is expected that by 2020 every human will create the equivalent of 1.7 megabytes of data every second. More than 90% of all the data that exists has been generated since 2015.

The data comes from numerous sources: social media, financial transactions, governments, and sensors. With the rise of the internet of things (IoT), the amount of data collected is expected to increase exponentially. It is believed that by 2030 there will be more than 125 billion IoT devices in service, representing a 360% increase from 2017.

However, problems arise when data is collected from multiple sources. It becomes unstructured, messy, and disparate.

Start your free trial now.

Get instant access to all our premium content, archives, newsletters, and online community.

Monthly Membership

Yearly Membership

What you get

Full access to all premium content and our full archives

Members'-only newsletters

Preferential access and discounts to all TechNode events

Direct access to the TechNode newsroom

Start your free trial now.

Get instant access to all our premium content, archives, newsletters, and online community.

Monthly Membership

Yearly Membership

Chris Udemans

Christopher Udemans is a Shanghai-based data and graphics reporter. He covers Chinese artificial intelligence, mobility, and cybersecurity. You can contact him at chrisudemans [at] technode [dot] com.