Machine Learning: An Introduction

Arthur Samuel, an American Pioneer in the field of Artificial Intelligence, defined Machine Learning in 1959. His definition succinctly captures the essence of this transformative technology. Samuel defined machine learning as:

"Field of study that gives computers the ability to learn without being explicitly programmed."

Arthur Samuel's definition highlights the core concept of machine learning, emphasizing the capacity of computers to learn and improve their performance over time through experience and exposure to data, without the need for explicit programming for every possible scenario. Samuel's groundbreaking work laid the foundation for development and evolution of machine learning as we know it today. 

In 1962, Arthur Samuel wrote an essay "Artificial Intelligence: A Frontier of Automation", he wrote:

"Programming a computer for such computations is at best, a difficult task, not primarily because of any inherit complexity in the computer itself but, rather, because of the need to spell out every minute step of process in the most exasperating detail. Computers, as any programmer will tell you, are giant morons, not giant brains."

Machine learning is like a regular programming, a way to get computers to compute a specific task. But Machine learning is similar to traditional programming in that it is a method of instructing computers to do a certain computation. However, how would you go about using regular programming to do a task that is challenging, such as differentiating between images of cats and dogs? In most cases, when we are building a program, it is not difficult for us to write down the steps that need to be taken in order to do a job. In most cases, we will construct a function that seems to be something like this: 

Fig 1. Traditional Programming

His basic idea was this: instead of telling the computer the exact steps required to solve a problem, instead show it examples of the problems to solve, and let it figure out to solve itself! This turned out to very effective, by 1961 his checkers playing program had learned to do so much that it beat Connecticut State Champion!

Here's how he described his idea: Suppose we arrange for some automatic means testing the effectiveness of any current weight assignment in terms of actual performance and provide a mechanism for altering the weight assignment so as to maximize the performance. We need not go into the details of such procedure to see that it could be made entirely automatic and to see that a machine so programmed would "learn" from its experience. 

Fig 2. A Program using Weight Assignment

A machine so programmed would "learn" from its experience. Learning would become entirely automatic when the adjustment of the weights was also automatic when instead of us improving a model by adjusting its weights manually, we relied on an automated mechanism that produced adjustments based on performance. 

The full picture of Samuel's idea of training a machine learning model.

Fig 3. Training a Machine Leaning Model

Also note that, once the model is trained, that is once we have chosen our final, best favorite weight assignment -- then we can think of the weights as being part of the model, since we are not varying them anymore. Therefore, actually using a model after its trained looks like:

Fig 4. Using a Trained Model as a Program

Samuel was working in 1960s, but terminology has changed. Here is the modern deep learning terminology:

• The functional form of the model is called its architecture (but be careful, sometimes people use model as a synonym of architecture, so this can get confusing).
• The weights are called parameters.
• The predictions are calculated from the independent variables, which is the data not including the labels. 
• The result of the model is called parameters.
• The measure of the performance is called the loss.
• The loss depends not only the predictions, but also the correct labels (also known as targets or dependent variable) e.g. "dog" or "cat".

Fig 5. Machine Learning