The Life And Times Of Machine Learning

A recent history of machine learning from the perspective of those who lived it


AI and machine learning have come a long way since the Alan Turin first created his famous test. Even in the 1950's, the idea that machines could one day be so indistinguishable from us that an intelligence test would be needed was presumed.

Since then, advancement after advancement has ensued at an increasingly accelerated pace. For example, in 1952, IBM employee, Arthur Samuels, who first coined the term 'machine learning', created the worlds first computer learning program. He wrote a program which played checkers for the company's first commercial computer, IBM 701. It got better with the more games it played, exemplifying the first fundamentals of artificial intelligence.

This incredible achievement is what inspired Nobel Prize winner, Herb Simon to make the now infamous prediction, 'within ten years, a computer will be the world's chess champion.' He made that claim in 1958 and it would be 40 years before that prediction came true in the form of IBM chess machine, Deep Blue.

Compare that with the story of Alpha Go and Alpha Go zero. The Chinese board game, Go, is widely considered the worlds most complex game with a near infinite number of possible moves. It was first won in 2016 by Googles DeepMind algorithm, AlphaGo, a feat many experts didn't expect to happen for another decade at best. The very next year, Google released AlphaGo Zero and this time only supplied the program with the rule set. Not only did it teach itself how to play Go, but after 3 days of playing itself, it beat AlphaGo.

The point is, while AI and machine learning have been on the minds of masses for decades, the advancements we have made in the last 10-20 years dwarfs those of the half century that came before it.

Hence, in order to get a grasp of how much the industry and attitudes have changed during this pivotal time in AI's history, I talked to some professionals who experienced these advancements first hand.

Computing power - stronger, faster.

Whether it computer scientists or movie directors, we have always imagined future tech years before we came close to creating them. The human race's imagination has always outpaced its progress, and no deeper is this exemplified than in the case of computing power.

Tom Andriola, Chief Information Officer with the University of California and speaker at this year's Machine Learning Summit, talked about this struggle between the theoretical and practical. Speaking about his time as a student he said, "It's interesting, many, many years ago when I was an undergraduate, I was exposed through my major, which was very small and niche at the time, to some of these [machine learning] techniques, but there weren't a lot of applications for them. The reason for that was these models are very data and computing intensive, and a generation ago we had neither. We simply didn't have the capability to utilize them, so they sat on the shelf."

The need for technology to catch up to our imaginations has been a constant feature of AI since the first spark of an idea flickered into existence. This is because no matter how far it comes, there will always be a new generation for whom it is still inadequate. Sam Zimmerman, CTO & Co-founder at Freebird and another one of our speakers, describes the immense amount of progress which has been made since he entered the machine learning world less than 8 years ago. "When I first entered the field in 2011, machine learning was just beginning to extend outside of advertising and finance into domains like sentiment analysis and computer vision. Largely this was a migration from quite clear optimizations of well-defined outcome variables (like click-through-rates and PnL) to much more abstract, subjective, and ill-defined outcome variables (like the “sentiment” of a sentence or the “setting” of a photo)."

The unpredictability of progress

These advancements have only been made possible because every year that has gone by, not only have computers become faster and stronger, but they have also become cheaper. With a reduction in price comes greater accessibility, more people can put forward an idea, can learn and, teach. The more people involved, the more innovations are borne and the more applications are envisioned.

This is the shift Zimmerman points out at the start of his career, this real-time progression from AI in advertising and finance into more complicated areas like sentiment analysis. He puts it down to 2 major changes around that time, "1- cheaper computing power and 2, the development of several very important regularization techniques (like dropout)-- both of which propelled powered the successful application of deep learning to new domains."

"However," he continues, "I think equally important as those two changes, the industry also developed massive training sets around these softer outcome variables to power these models. Without this influx of (largely manual) efforts to create massive well-labeled datasets to define these softer outcome variables, we certainly would not be where we are today."

Zimmerman's last point is key to understanding a commonly overlooked factor which has contributed so much to the progress of machine learning. Much like how the digital age didn't really kick off until computers went from taking up entire rooms in companies to fitting on desks in our homes, the more accessible machine learning has become, the faster it has innovated.

"All of a sudden, computing capability has caught up and you can compute at massive scale," Andriola explains, "whether it's with the government or providers like the Google's, the Amazon's, or the Microsoft's of the world. These techniques allow you process huge amounts of data, which is now everywhere as we essentially live in a digital world...this has become very in vogue at universities where our job is really to create the future."

It can feel like the future and present are interchangeable as so much progress is being made so quickly. Complete paradigm shifts can now happen within the space of a 10-year career, meaning we all have to be more adaptable than ever before. Hugh Williams, Machine Learning Data Science manager at Uber and another one of this year's speakers, puts it perfectly. "Honestly, I don’t know about 'the industry', but I can speak for the companies I’ve worked for. In general, I’ve seen ML become a more formalized staple of business strategy as opposed to some lofty theoretical concept. It is no longer a pipe-dream to use an-ML based strategy to generate efficiency. Those who already used ML are opening their mind to more complex and powerful techniques and willing to push the boundaries now in terms of research. It’s exciting.

As a regular practitioner, it’s greatly helped elevate the conversation. I spend a lot less time advocating for whether to use ML and much more time brainstorming how to use ML."

Hugh williams pyramid

Read next:

The Pyramid Of Data Needs (And Why It Matters For Your Career)