How to Think About Machine Learning

Machine learning is a large and interdisciplinary field of study.

You can achieve impressive results with machine learning and find solutions to very challenging problems. But this is only a small corner of the broader field of machine learning often called predictive modeling or predictive analytics.

In this post, you will discover how to change the way you think about machine learning in order to best serve you as a machine learning practitioner.

After reading this post, you will know:

• What machine learning is and how it relates to artificial intelligence and statistics.
• The corner of machine learning that you should focus on.
• How to think about your problem and the machine learning solution to your problem.

Let’s get started.

How to Think About Machine Learning
Photo by Rajarshi MITRA, some rights reserved.

Overview

This post is divided into 3 parts; they are:

1. You’re Confused
2. What is Machine Learning?
3. Your Machine Learning

You’re Confused

You have a machine learning problem to solve, but you’re confused about what exactly machine learning is.

There’s good reason to be confused. It is confusing to beginners.

Machine learning is a large field of study, and not all much of it is going to be relevant to you if you’re focused on solving a problem.

In this post, I hope to clear things up for you.

We will start off by describing machine learning in the broadest terms and how it relates to other fields of study like statistics and artificial intelligence.

After that, we will zoom in on the aspects of machine learning that you really need to know about for practical engineering and problem solving.

What is Machine Learning?

Machine learning is a field of computer science concerned with programs that learn.

The field of machine learning is concerned with the question of how to construct computer programs that automatically improve with experience.

— Machine Learning, 1997.

That is super broad.

There are many types of learning, many types of feedback to learn from, and many things that can be learned.

This could encompass diverse types of learning, such as:

• Developing code to investigate how populations of organisms “learn” to adapt to their environment over evolutionary time.
• Developing code to investigate how one neuron in the brain “learns” in response to stimulus from other neurons.
• Developing code to investigate how ants “learn” the optimal path from their home to their food source.

I give these esoteric examples on purpose to help you really nail down that machine learning is a broad and far reaching program of research.

Another case that you may be more familiar with is:

• Developing code to investigate how to “learn” patterns in historical data.

This is less glamorous, but is the basis of the small corner of machine learning in which we as practitioners are deeply interested.

This corner is not distinct from the other examples; there can be a lot of overlap in methods for learning, fundamental tasks, ways of evaluating learning, and so on.

What About Artificial Intelligence?

Machine learning is a subfield of artificial intelligence.

It overlaps with machine learning.

Artificial intelligence is also an area of computer science, but it is concerned with developing programs that are intelligent, or can do intelligent things.

Intelligence involves learning, e.g. machine learning, but may involve other concerns such as reasoning, planning, memory, and much more.

This could encompass diverse types of learning such as:

• Developing code to investigate how to optimally plan logistics.
• Developing code to investigate how to reason about a paragraph of text.
• Developing code to investigate how to perceive the contents of a photograph.

Artificial intelligence is often framed in the context of an agent in an environment with the intent to address some problem, but this does not have to be the case.

Machine learning could just as easily be named artificial learning to remain consistent with artificial intelligence and help out beginners.

The lines are blurry. Machine learning problems are also artificial intelligence problems.

What About Statistics?

Statistics, or applied statistics with computers, is a sub-field of mathematics that is concerned with describing and understanding the relationships in data.

This could encompass diverse types of learning such as:

• Developing models to summarize the distribution of a variable.
• Developing models to best characterize the relationship between two variables.
• Developing models to test the similarity between two populations of observations.

It also overlaps with the corner of machine learning interested in learning patterns in data.

Many methods used for understanding data in statistics can be used in machine learning to learn patterns in data. These tasks could be called machine learning or applied statistics.

Your Machine Learning

Machine learning is a large field of study, and it can help you solve specific problems.

But you don’t need to know about all of it.

• You’re not an academic investigating an esoteric type of learning as in machine learning.
• You’re not trying to make an intelligent agent as in artificial intelligence.
• You’re not interested in learning more about why variables relate to each other in data as in statistics.

In fact, when it comes to learning relationships in data:

• You’re not investigating the capabilities of an algorithm.
• You’re not developing an entirely new theory or algorithm.
• You’re not extending an existing machine learning algorithm to new cases.

These may be activities in the corner of machine learning that we may be interested in, but activities for academics, not practitioners like you.

So what parts of machine learning do you need to focus on?

I think there are two ways to think about machine learning:

1. In terms of the problem you are trying to solve.
2. In terms of the solution you require.

Your Machine Learning Problem

Your problem can best be described as the following:

Find a model or procedure that makes best use of historical data comprised of inputs and outputs in order to skillfully predict outputs given new and unseen inputs in the future.

This is super specific.

First of all, it discards entire sub-fields of machine learning, such as unsupervised learning, to focus on one type of learning called supervised learning and all the algorithms that fit into that bucket.

That does not mean that you cannot leverage unsupervised methods; it just means that you do not focus your attention there, at least not to begin with.

Second of all, it gives you a clear objective that dominates all others: that is model skill at the expense of other concerns such as model complexity, model interpretability, and so on.

Again, this does not mean that these are not important, just that they are considered after or in conjunction with model skill.

Thirdly, the framing of your problem this way fits neatly into another field of study called predictive modeling. That is a field of study that borrows methods from machine learning with the objective of developing models that make skillful predictions.

In some areas of business, this area may also be called predictive analytics and encompasses more than just the modeling component to include related activities of gathering and preparing data and deploying and maintaining the model.

More recently, this activity can also be called data science, although that phrase also has connotations of inventing or discovering the problem in addition to working it through to a solution.

I don’t think it matters what you call this activity. But I do think it is important to deeply understand that your interest in and use of machine learning is highly specific and different from some other uses by academics.

It allows you to filter the material you read and the tools you choose in order to stay focused on the problem you’re trying to solve.

Your Machine Learning Solution

The solution you require is best described as the following:

A model or procedure that automatically creates the most likely approximation of the unknown underlying relationship between inputs and associated outputs in historical data.

Again, this is super specific.

You need an automatic method that produces a program or model that you can use to make predictions.

You cannot sit down and write code to solve your problem. It is entirely data-specific and you have a lot of data.

In fact, problems of this type resist top-down hand-coded solutions. If you could sit down and write some if-statements to solve your problem, you would not need a machine learning solution. It would be a programming problem.

The type of machine learning methods that you need will learn the relationship between the inputs and outputs in your historical data.

This framing allows you to think about what that real underlying yet unknown mapping function might look like and how noise, corruption, and sampling of your historical data may impact approximations of this mapping made by different modeling methods.

Without this framing, you will wonder things like:

• Why there isn’t just one super algorithm or set of parameters.
• Why the experts can’t just tell you what algorithm to use.
• Why you can’t achieve a zero error rate with predictions from your model.

It helps you see the ill-defined nature of the predictive modeling problem you’re trying to solve and sets reasonable expectations.

Next Step

Now that you know how to think about machine learning, the next step is to change the way you think about the process of solving a problem with a machine learning solution.

For a hint, see the post:

• A Gentle Introduction to Applied Machine Learning as a Search Problem

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Posts

• Gentle Introduction to Predictive Modeling
• How Machine Learning Algorithms Work
• What is Machine Learning?
• A Gentle Introduction to Applied Machine Learning as a Search Problem
• Where Does Machine Learning Fit In?

Articles

• Machine learning on Wikipedia
• Predictive modelling on Wikipedia
• Predictive analytics on Wikipedia

Summary

In this post, you discovered how to change the way you think about machine learning in order to best serve you as a machine learning practitioner.

Specifically, you learned:

• What machine learning is and how it relates to artificial intelligence and statistics.
• The corner of machine learning that you should focus on.
• How to think about your problem and the machine learning solution to your problem.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.