Is Machine Learning Revolutionizing Data Management?

Artificial intelligence, or AI for short, is without a doubt the hottest topic in IT these days. The world's largest hedge fund, Bridgewater Associates, is working on software to automate day-to-day operations, including hiring and firing as well as strategic decisions.

Babylon, a U.K.-based healthcare company, has raised $60 million to develop an AI doctor that can make diagnoses and estimate the likelihood of future health problems without human assistance.

According to analysts, 44 percent of organizations are already using AI or plan to use it in the next two years, and another 38 percent are considering it.

AI for 60 years

The whole thing is not surprising or sudden: AI has been researched for more than 60 years, but its use even for very small organizations has only recently come into consideration.

The reasons for this have mainly to do with the fact that AI and machine learning are only finally able to deliver meaningful results with more data and computing power. We have reached a point where it is technically possible and also affordable to store literally everything.

But finding the patterns, algorithms, and models behind the data requires another component that is now available: the power to process it.

Instead of investing in physical hardware on site, organizations can step outside and leverage thousands of servers with mature and specialized hardware. And for as long or as short as they are needed, courtesy of cloud infrastructure providers.

Sophisticated analytics and models can now be developed in reasonable time windows and learning results executed in real time. The age of machine learning and AI is here.

This leads to two conclusions for companies: Use it to do tasks that humans simply aren't capable of, or use it to enrich the current work of humans. JP Morgan Chase, for example, saves 360,000 man-hours a year and eliminates errors by automating the reading and interpretation of loan agreements.

First mover and winner

The Inova Translational Medicine Institute can use machine-learning algorithms to create personalized treatment plans for its patients. The algorithms use terabytes of clinical and genomic information to identify genetic factors for disease.

Enriching human work or triggering new areas is used in all industries. Nevertheless, there are clear "first movers" and winners: banks, retail and healthcare.

These industries have always collected relevant data. They were also among the first to use open source Big Data platforms to manage the data.

Now they can capitalize on this old habit and benefit from the development and proliferation of open source projects like Apache Spark, which enable machine learning and fit directly into their existing platforms.

AI is no longer the privilege of a few companies with niche applications. With an integrated technology stack that brings together the masses of data and computing power, come more and more mature use cases that underscore the benefits of the approach. Such a stack makes machine learning and AI possible.

All that is needed is a vision and a use case.