Skip to content
Knowledge Hub/Article
Article

What Is Machine Learning? Deep Learning and Neural Networks

We cover the types of machine learning, its core concepts, example algorithms, evaluation metrics, deep learning and how models move into production.

Oğuzhan Gerçek··9 min read
What Is Machine Learning? Deep Learning and Neural Networks

Short answer: Machine learning (ML) is the general name for methods that let a computer make predictions or generate content by learning from data, without being explicitly programmed; it is a subfield of artificial intelligence. During training, a model learns the relationships in example data and then applies them to new data. The main types are supervised, unsupervised, reinforcement and self-supervised learning. Deep learning is the branch of machine learning that uses multi-layer artificial neural networks, and large language models belong to this family. A model's value is measured by how well it performs on data it has never seen.

What is machine learning?

According to Google's Machine Learning Crash Course, machine learning is the process of training a piece of software, called a model, to make useful predictions or generate content from data. Google explains the difference with rainfall prediction: a traditional approach builds a physics-based representation of the atmosphere and computes massive amounts of fluid dynamics equations, while an ML approach gives a model enormous amounts of weather data until it learns the relationship between weather patterns and rainfall on its own. NIST's glossary likewise speaks of computer systems that adapt and learn from data with the goal of improving accuracy.

According to IBM, machine learning sits directly beneath AI (AI article, glossary). ISO/IEC 23053, published in June 2022, provides a conceptual framework and shared terminology for AI systems that use machine learning; according to ISO, deep learning is explicitly covered by that framework.

Core concepts: model, features, labels and training

  • Features and labels: According to Google, the label is the "answer" the model is asked to predict, and features are the values used to predict it. In a rainfall model, temperature and humidity are features and rainfall amount is the label.
  • Model, training and inference: According to Microsoft Learn, a model is a software application that encapsulates a function that calculates an output from inputs. Defining that function is called training; using it to predict new values is called inference.
  • Loss and gradient descent: Loss measures how wrong a model's predictions are, and training tries to minimize it; gradient descent iteratively finds the weights and bias that produce the lowest loss.
  • Training, validation and test sets: According to Google, the model trains on the training set, the validation set performs initial testing while the model is being trained, and the test set evaluates the trained model. Tuning over and over against the test set can make the model implicitly fit that set's peculiarities.
  • Overfitting and generalization: An overfit model memorizes the training set and fails to make correct predictions on new data; a model that generalizes well makes good predictions on new data too.

Types of machine learning

  • Supervised learning: According to Google, the model can make predictions after seeing lots of data with the correct answers. Two of its most common use cases are regression, which predicts a numeric value (the amount of rain), and classification, which predicts the likelihood that something belongs to a category (whether an email is spam).
  • Unsupervised learning: The model looks for meaningful patterns in a dataset; many unsupervised models rely on clustering, which organizes similar data into groups.
  • Reinforcement learning: The model learns by getting rewards or penalties for actions it performs within an environment. Google's examples are robots learning to walk around a room and AlphaGo playing Go.
  • Self-supervised learning: According to IBM, the model generates implicit labels from unlabeled data. Language models such as GPT are trained this way, by predicting the next word.

Machine learning algorithms: four examples

Four basic examples from the scikit-learn documentation:

  1. Linear regression: Treats the target value as a linear combination of the features and fits coefficients that minimize the residual sum of squares between observed and predicted values.
  2. Logistic regression: Despite its name, a classification model; it models the probabilities of the possible outcomes with a logistic function. In binary classification, the default threshold is 0.5.
  3. Decision tree: A supervised method that learns simple decision rules from the data. Trees are simple to understand and interpret, but over-complex trees overfit, so pruning or a maximum depth is needed.
  4. K-means: A clustering algorithm that splits unlabeled data into n groups by minimizing the within-cluster sum of squares (inertia); the number of clusters has to be specified in advance.

How is a model evaluated? Accuracy, precision and recall

According to Google, the three basic metrics are:

  • Accuracy: The proportion of all classifications that were correct.
  • Precision: The proportion of the model's positive classifications that are actually positive.
  • Recall: The proportion of all actual positives that were correctly classified as positive.

Accuracy is misleading on imbalanced data: in Google's example, if the positive class appears in 1% of cases, a useless model that predicts negative every time scores 99% accuracy. Precision and recall also tend to move in opposite directions. Which one to favor depends on whether a missed case or a false alarm costs more.

What is deep learning?

According to the freely available book "Deep Learning" by Goodfellow, Bengio and Courville (MIT Press, 2016), deep learning lets computers learn from experience and understand the world in terms of a hierarchy of concepts, each defined through its relation to simpler concepts. The book's introduction explains that the graph of these concepts is deep, with many layers, which is where the approach gets its name; deep learning is a type of machine learning.

ACM gave the 2018 A.M. Turing Award to Yoshua Bengio, Geoffrey Hinton and Yann LeCun for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing. According to AWS, large language models are very large deep learning models (LLM article). We explain why training relies on GPUs in our GPU article, and our glossary has a short definition of the term.

How do artificial neural networks work?

According to Google, neural networks are a family of model architectures designed to find nonlinear patterns in data:

  • Layers and neurons: The layers between the input layer and the output layer are called hidden layers, and their nodes are called neurons. According to the Deep Learning book, they are called hidden because the training data does not show the desired output for each of these layers.
  • Weights and bias: Each neuron multiplies its inputs by a weight specific to each connection, sums the products and adds a bias; the weights and biases are the model's parameters.
  • Activation function: Linear operations stacked on linear operations are still linear, so nonlinear relationships require an activation function. Common examples are sigmoid, tanh and ReLU.
  • Backpropagation: According to Google, it is the most common training algorithm for neural networks. The book stresses that backpropagation only computes the gradient; another algorithm, such as stochastic gradient descent, uses that gradient to do the learning.

According to the book, these networks are called neural because they are loosely inspired by neuroscience, and deep learning should not be viewed as an attempt to simulate the brain.

Machine learning in the enterprise: data, bias and production

According to Google Cloud, the quality and quantity of data are crucial to performance. NIST's SP 1270, published in March 2022, sorts bias into three groups: systemic bias, which results from the procedures and practices of institutions; statistical and computational bias, which stems from errors when the sample is not representative of the population; and human bias, which reflects systematic errors in human thought. According to NIST, even a representative dataset may still exhibit entrenched historical and systemic biases.

We cover collecting and cleaning data from source systems in our ETL article and data readiness for AI in our data platform article; for data at very large scale, see the big data entry in our glossary. Versioning, monitoring and retraining a model in production is the subject of our MLOps article. According to IBM, in IT operations machine learning helps with anomaly detection, root cause analysis and event correlation; we cover this area on our AIOps and intelligent operations page, and the infrastructure that runs the models in our AI stack design and engineering and datacenter and infrastructure engineering services.

Frequently asked questions

What is ML? ML stands for machine learning: methods that let a computer learn from data to make predictions or generate content without being explicitly programmed.

What is deep learning? It is the branch of machine learning that uses multi-layer artificial neural networks. The model learns, from data, hierarchies of concepts built on top of simpler concepts.

What are artificial neural networks? They are model architectures built from layers of neurons to find nonlinear patterns in data. Each neuron multiplies its inputs by weights, sums them, adds a bias and passes the result through an activation function.

What are some machine learning algorithms? Linear and logistic regression, decision trees and k-means are basic examples. The first three are used in supervised learning, and k-means is used to cluster unlabeled data.

What is unsupervised learning? It is the type of machine learning that looks for meaningful patterns in unlabeled data. Many such models rely on clustering, which groups similar data together.

What is reinforcement learning? It is the type of machine learning in which a model learns by getting rewards or penalties for the actions it performs within an environment. It has been used to train robots and to play games such as Go.

Sources