Ryan Saxe's Blog

Designing Transparent Neural Networks

Thu, 04 Mar 2021 00:00:00 GMT

Most systems we interact with are part of some pipeline that integrates Machine Learning (ML). Sometimes we interact with an ML model directly, like Spotify’s recommender system for songs. Other times, this interaction is more detatched; we post a comment on Twitter or Facebook, and this comment is used to train some language model at the respective company. As ML models become more and more prevalent, interpreting and explaining the decisions these models make become increasingly important.

Neural networks (NNs) are a popular class of machine learning algorithms, which are notorious for being difficult to interpret. They are often referred to as “black boxes” or “opaque”. Linear Models, Generalized Linear Models (GLMs), and Generalized Additive Models (GAMs) are examples of popular machine learning algorithms that are not “opaque”, but instead “transparent”. This transparency often leads linear and additive models to be favorable choices over classic neural networks even though the functions neural networks can learn are a superset of the functions these other models can learn. The goal of this blog post is to demonstrate that, with a particular asterix over the architecture of neural networks, they can be just as transparent.

Prior to jumping into such a neural architecture, it’s important to understand the fundamental transparent algorithms. The following section will provide a quick background on the math and fundamentals for Generalized Additive Models, which is what will inspire our neural architecture design. GAMs are a special kind of GLM, and if you are not familiar with GLMs and how they are connected to Neural Networks, I would recommend reading the blog post below before reading this one:

Prerequisite: This post builds on From Linear Models to Neural Networks.

Generalized Additive Models

Generalized Additive Models (GAMs), introduced in (Hastie & Tibshirani, 1986), take another step towards reducing the restrictions within linear models. There are two modifications that GAMs make to classic GLMs, which truly moves from rigid assumptions to flexible modeling:

Allow non-linearity: GAMs wrap each component $X_i$ with a function $h_k$ , where $h_k$ is some learned function that can be non-linear, but must be smooth¹. It is also usually non-parametric.
Feature interaction: The systematic component can be an equation that contains non-linear feature interaction like $h_k(X_i,X_j, \cdots)$ .

Hence, equations for GAMs could look like this:

$g(\mu(X)) = \beta_0 + h_1(X_1) + h_2(X_2, X_3) + \cdots + h_m(X_n) + \epsilon$

Technically, this makes GLMs a special case of GAMs where all functions $h_k$ simply multiply their corresponding input feature(s) by a single parameter $\beta_k$ . However, unlike GLMs, these functions require a more convoluted fitting mechanism. If you are interested in the history of GAMs and how they are fit, please refer to the original papers on the backfitting algorithm: (Friedman & Stuetzle, 1981), (Breiman & Friedman, 1985).

Don’t worry if you are not familiar with some of the terms (e.g. b-spline), as they won’t be relevant to the main body of this blog post about transparent neural networks. What follows is a simplification to provide intuition on what these $h_k$ functions are.

$h_k(x_i) = \sum_{j=1}^n b_j(x_i)\beta_j$

Where $b_j$ is a b-spline, $\beta_j$ is a learned coefficient corresponding to $b_j$ , and $n$ is a hyperparameter describing the number of b-splines to use to fit the GAM. A linear combination of b-splines uniquely describes any spline function sharing the same properties as the b-splines (Prautzsch et al., 2002), which means these $h_k$ functions are spline functions. Below is an example of fitting a smooth function ( $h_k(x_i) = sin(x_i)$ ) using a linear combination of b-splines.

As you can see, a linear combination of many cubic b-splines was sufficient to fit this smooth function. GAMs try and fit spline functions to transform each individual dependent variable, and model the independent variable as a linear combination of these spline functions. This maintains transparency because, once fit, we can inspect the learned functions to understand exactly how our model makes predictions according to individual features.

There is so much more to learn about GAMs, such as how these splines are learned, methods of interpreting them, and important regularization penalties to ensure higher degrees of smoothness. There are even some newer methods of fitting GAMs using decision trees instead of splines. If you would like a more extensive review on GAMs, please refer to this wonderful mini-website-textbook, however that’s out of the scope of this blog post.

Transparent Neural Networks

(Hornik et al., 1989) is the original paper suggesting NNs are a type of universal approximator. The theory of this contribution was explored in multi-layer perceptrons in (Pinkus, 1999) and generally formalized by (Csáji, 2001). The theorem can be summarized by:

A neural network with a single hidden layer of infinite width can approximate any continuous function.

If neural networks can be used to approximate any continuous function, then they can be used to approximate the non-linear, non-parametric, functions ( $h_k$ in the previous section) necessary for Generalized Additive Models. Furthermore, neural networks can describe a wider set of functions than GAMs because continuous functions don’t have to be smooth, while smooth functions have to be continuous.

Above are images of three functions. The first function is a linear combination of the second and third function². Observe that all three functions are continuous. We would like to build a model that can fit the first function, while maintaining feature-wise interpretability such that we can see that it properly learns the second and third functions. Unfortunately, it is not reasonable to expect a GAM to achieve this because, while the functions are continuous, they are not smooth. Below is a GAM with very low regularization and smoothness penalties in order to let it try and fit non-smooth functions. Furthermore, it is trained and tested on the same dataset. This is a strong demonstration that GAMs cannot approach these problems, because they can’t even overfit to the solution.

from pygam import LinearGAM
from pygam import s as spline
#fit a classic GAM with no regularization or smoothing penalties to try and 
#let it overfit to non-smooth functions. It still fails!
gam = LinearGAM(
    spline(0,lam=0,penalties=None) + 
    spline(1,lam=0,penalties=None),
    callbacks=[]
).fit(X, y)

While these fits aren’t terrible in terms of prediction error, that’s not what we care about. We care about learning the proper structure for the functions that explain the relationship between individual features and our output. The above plots are a clear demonstration that GAMs can’t even overfit to provide that.

Luckily, because these functions are continuous, we can use a neural network to approximate them!

Generalized Additive Neural Networks

The trick is to use a different neural network for each individual feature, and add them together just like how GAMs work! By replacing the non-linear, non-parametric, functions in GAMs by neural networks, we get Generalized Additive Neural Networks (GANNs), introduced in (Potts, 1999). Unfortunately, this contribution did not take off because we didn’t have the technical capacity to train large networks as we do today. Luckily, now it is quite easy to fit such a model.

GANNs are simply a linear combination of neural networks, where each network only observes a single input feature³. Because each of these networks take a single input feature, and provide a single output feature, it becomes possible to plot a two-dimensional graph where the x-axis is the input feature and the y-axis is the output feature for each network. This graph is hence a fully transparent function describing how the neural network learned to transform the input feature as it contributes, additively, to the prediction. Hence this type of neural architecture is sufficient for creating a model as transparent as classic linear and additive models described in this blog post so far.

Below is code for creating a GANN using Keras. As you can see, this model is capable of solving the regression problem while maintaining feature-wise transparency on piecewise continous functions!

import torch
import torch.nn as nn

#define a simple multi-layer perceptron
class NN(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            # relu helps learn more jagged functions if necessary.
            nn.Linear(1, 8), nn.ReLU(),
            # softplus helps smooth between the jagged areas from above
            #     as softplus is referred to as "SmoothRELU"
            nn.Linear(8, 8), nn.Softplus(),
            nn.Linear(8, 8), nn.Softplus(),
            nn.Linear(8, 8), nn.Softplus(),
            nn.Linear(8, 1),
        )

    def forward(self, x):
        return self.net(x)

#define a Generalized Additive Neural Network for n features
class GANN(nn.Module):
    def __init__(self, n_features):
        super().__init__()
        self.n_features = n_features
        # initialize MLP for each input feature
        self.components = nn.ModuleList([NN() for _ in range(n_features)])
        # create final layer for a linear combination of learned components
        self.linear_combination = nn.Linear(n_features, 1)

    def forward(self, x):
        #split up by individual features
        individual_features = torch.split(x, 1, dim=1)
        components = []
        #apply the proper MLP to each individual feature
        for f_idx, individual_feature in enumerate(individual_features):
            component = self.components[f_idx](individual_feature)
            components.append(component)
        #concatenate learned components and return linear combination of them
        components = torch.cat(components, dim=1)
        return self.linear_combination(components)

X_tensor = torch.tensor(X.to_numpy(), dtype=torch.float32)
y_tensor = torch.tensor(y.to_numpy(), dtype=torch.float32).unsqueeze(1)
loader = DataLoader(TensorDataset(X_tensor, y_tensor), batch_size=32, shuffle=True)

model = GANN(n_features=2)
model.compile()
optimizer = optim.Adam(model.parameters())
loss_fn = nn.MSELoss()

for epoch in range(50):
    epoch_loss = 0.0
    for xb, yb in loader:
        pred = model(xb)
        loss = loss_fn(pred, yb)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        epoch_loss += loss.item()
    if (epoch + 1) % 1 == 0:
        print(f'Epoch {epoch+1}/50, Loss: {epoch_loss/len(loader):.4f}')

At first glance, these results look good, but not perfect. The first plot demonstrates this model achieved a near perfect fit of the actual task, while the last two plots look like the GANN was capable of learning the general shapes of the functions, but is off on the intercepts for all of them. This should not discount the validity of the model. The corresponding math demonstrates perfectly fitting intercepts of an additive model cannot be guaranteed.

Let $h_i(X_i) = \alpha_i + f(X_i)$ where $f(X_i)$ represents all of the aspects of $h(X_i)$ that dependent on $X_i$ , and $\alpha_i$ represents the intercept.

\begin{aligned} y & = \beta_0 + \sum_{i=1}^n \beta_i h_i(X_i) \\ & = \beta_0 + \sum_{i=1}^n \beta_i (\alpha_i + f_i(X_i)) \\ & = \beta_0 + \sum_{i=1}^n \beta_i \alpha_i + \sum_{i=1}^n \beta_i f_i(X_i) \end{aligned}

The only way to tease apart these intercepts is via $\beta$ . Imagine the proper fit of this equation, for every $i$ , had $\beta_i = 1$ and $\alpha_i = 2$ . In this case, if half of the learned $\alpha_i$ s are zero, and the other half are four, that would yield the exact same result for $\sum_{i=1}^n \beta_i \alpha_i$ as the proper fit. Hence, by way of contradictory example, it is impossible to guarantee learning correct intercepts for the individual components of any additive model.

The plots below are what happens when we simply adjust the learned intercepts for these functions. As you can see, by simply changing the intercept, we are able to show a near perfect fit of these functions, which is the best we can ever hope to do! Furthermore, we can explore the derivatives of these functions to measure the goodness of fit because the rate of change of the function is entirely independent of the intercept.

This is a clear demonstration that Generalized Additive Neural Networks are capable of overfitting to piecewise continuous functions while maintaining transparency!

The next section provides a brief overview of Neural Additive Models (NAMs), which are a special variant of GANNs that are designed to be able to fit “jumpy functions”, as this is more reminiscient of real-world data.

Neural Additive Models

Real world data doesn’t look like beautifully continuous functions. It’s often messy, and there will be multiple data points with extremely similar features that have noticeably different results. Many machine learning models have existing structure that prevents learning functions that look crazy and jump all over the place. Linear regression has to learn the best fitting line because $X^T\beta$ can’t describe a “jumpy” function. GAMs are regularized to enforce smoothness to prevent this as well. (Agarwal et al., 2020) explores a particular modification to the mathematic computations made on nodes in GANNs in order to let them fit “jumpy” functions. They call these modified nodes “exp-centered hidden units” or “ExU units”, and they work as follows:

Let $w$ and $b$ be the respective weight and bias parameters on a node with activation function $f$ . Then, when the node is passed input $x$ it computes: $f(e^w * (x - b))$ . Recall that the normal computation on these nodes is simply applying the activation function to a linear computation: $f(w * x + b)$ . The reason exponentiation of the weights accomplishes the goal of learning “jumpy” functions is that small changes in the input can have drastic changes on the output, enabling small weights to still represent a function with a steep slope. The excerpt below is taken directly from (Agarwal et al., 2020) and demonstrates the difference between a non-regularized neural network’s ability to overfit to data when using normal node computation (a) versus ExU unit computation (b).

While this clearly demonstrates ExU’s ability to overfit to jumpy functions, this isn’t ideal. Overfitting isn’t a good thing. Hence, the rest of this paper demonstrates ExU can be regularized to learn functions that are relatively smooth, yet “jumpy” at very specific junctions. This is more reminiscient of real-world solutions, and the paper shows that this can outperform current state-of-the-art GAMs while maintaining feature transparency. Their regularization strategies are:

Dropout: randomly zero-out nodes inside each NN.
Weight Decay: add a penalty corresponding to the L2 norm of weights inside each NN.
Output Penalty: add a penalty corresponding to the L2 norm of the outputs of each NN.
Feature Dropout: randomly exclude entire features from the NAM.

ExU and these regularization techniques are the only difference between NAMs and GANNs, and it fundamentally changes the loss landscape in a way that enables learning “jumpy” functions. GANNs are a very general architectural description, and there are many perturbations on the architecture, such as NAMs, that can make them well suited for your problem. The final section in this blog post will discuss the computational graph, and how to leverage understanding a specific problem to design a transparent neural network.

Designing Your Computational Graph

Hopefully the review of the NAM paper motivated some curiosity and creativity. The nodes and feature-networks in GANNs don’t need to be the classic feed-forward-neural-network you may be familiar with. Furthermore, there is no reason all of these feature-networks have to be the same. They can have different numbers of layers, different activation functions, and different architectures all together.

A neural network can be described as a computational graph. It’s a graph that describes a directed order of computation in order to go from some input to some output. The general description of the order of computations of a GANN is:

Define a finite set of modules (NNs), and specify which features go into which modules.
Make your prediction as a linear combination of the output of these modules.

The second point is what makes the model a GANN. It’s what specifies the model as strictly additive. We can make a slight modification to this description to further generalize into transparent models that are not strictly additive.

Define a finite set of modules, and specify which features go into which modules.
Define a function that specifies how the outputs of each module are used for prediction.

The second point here is more general than that of a GANN, because “a linear combination of the output of these modules” is one such example of “a function that specifies how the outputs of each module are used”. As this blog post has demonstrated, transparency is all about control of the input features. That last operation that is a linear combination of all these feature-neural-networks doesn’t have to be how they are used! Based on an expert understanding of your data and your problem, you can define a function you would like to fit. And, as long as you code up a computational graph that doesn’t entangle all the features in opaque ways, you can fit that function with a collection of neural networks just like how GANNs work!

Let’s walk through a simple example:

Let’s say you have 5 features $[x_1, x_2, x_3, x_4, x_5]$ . And let’s say that you believe, according to features $x_1$ and $x_2$ , the solution to your problem should treat the other features differently. We can describe our equation as follows:

Let $f_1(x_1, x_2) \in \{0,1\}$

Let $f_2(x_3, x_4, x_5) \in \mathbb{R}$

Let $f_3(x_3, x_4, x_5) \in \mathbb{R}$

Then, we want to solve:

\hat{y} = (f_1(x_1, x_2)) * f_2(x_3, x_4, x_5) + (1 - f_1(x_1, x_2)) * f_3(x_3, x_4, x_5)

Specifically, the purpose of $f_1$ is to learn some binary feature to determine how the model handles the other features. if $f_1(x_1, x_2) = 0$ , then the model will use $f_3$ , otherwise it will use $f_2$ . You can solve this problem with three neural networks, one for each function. Then, rather than having the prediction be a normal linear combination of those three functions like a GANN, you set the prediction to correspond to the equation above.

I have simulated a dataset that defines $f_1$ as whether $x_1,x_2$ are inside the unit circle, and defines $f_2$ and $f_3$ as basic linear models⁴. The code below defines a a neural network that learns these three functions to make a prediction. And below I show the results to demonstrate that we learn the correct circle boundaries and linear models, which maintains full transparency!

Additional Note: this specific example requires learning a conditional function $f_1$ , which is not continuous and hence requires some tricks to learn via neural networks. You can see my code below for the BinaryNN, which learns a continuous function, modifies it to be binary, but applies backpropigation as if that modification was the identity function. This is called the Straight-Through estimator, and was introduced by Hinton in his 2012 Coursera lectures. Refer to (Bengio et al., 2013) If you’d like further reading on conditional computation.

class BinaryNN(nn.Module):
    def __init__(self, threshold=0.5):
        super().__init__()
        self.threshold = threshold
        self.net = nn.Sequential(
            nn.Linear(2, 8), nn.ReLU(),
            nn.Linear(8, 8), nn.ReLU(),
            nn.Linear(8, 8), nn.ReLU(),
            #sigmoid to ensure the output is in the range 0,1
            # before we round to be binary
            nn.Linear(8, 1), nn.Sigmoid(),
        )

    def forward(self, x):
        x = self.net(x)
        return self.round_to_binary(x)

    def round_to_binary(self, x):
        """
        x is a number between 0 and 1, and is set
            to be 0 if less than self.threshold, 1
            if greater than self.threshold.

        We use a straight-through estimator: the forward
            pass rounds to binary, but the backward pass
            passes gradients through as if rounding didn't happen.
        """
        hard = (x > self.threshold).float()
        # straight-through: forward uses hard, backward uses x
        return x + (hard - x).detach()

class TransparentNN(nn.Module):
    def __init__(self, threshold=0.5, binary_idx=2):
        super().__init__()
        self.binary_idx = binary_idx
        #layer for learning a function that maps to a binary
        # label according to a specified threshold
        self.binary_layer = BinaryNN(threshold=threshold)
        #two different linear regression models where linear_1
        # is used when the binary layer specifies 1, otherwise
        # we use linear_2
        self.linear_1 = nn.Linear(3, 1)
        self.linear_2 = nn.Linear(3, 1)
        #note: if you want to model this as a complex non-linear
        # function, you can replace these linear layers with an MLP

    def forward(self, x):
        #get features to pass to the binary layer
        binary_slice = x[:, :self.binary_idx]
        #get features to pass to the linear layers
        linear_slice = x[:, self.binary_idx:]
        #compute binary flag
        binary_flag = self.binary_layer(binary_slice)
        #return the function we are trying to model
        return (
            binary_flag * self.linear_1(linear_slice) +
            (1 - binary_flag) * self.linear_2(linear_slice)
        )

x_tensor = torch.tensor(x, dtype=torch.float32)
y_tensor = torch.tensor(y, dtype=torch.float32).unsqueeze(1)
loader = DataLoader(TensorDataset(x_tensor, y_tensor), batch_size=32, shuffle=True)

model = TransparentNN(threshold=0.5)
model.compile()
optimizer = optim.Adam(model.parameters())
loss_fn = nn.MSELoss()

for epoch in range(5):
    epoch_loss = 0.0
    for xb, yb in loader:
        pred = model(xb)
        loss = loss_fn(pred, yb)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        epoch_loss += loss.item()
    if (epoch + 1) % 1 == 0:
        print(f'Epoch {epoch+1}/50, Loss: {epoch_loss/len(loader):.4f}')

As you can see above, our neural network designed to fit our specified function was properly able to learn both weights of the linear models and the circular boundary to determine which linear model to use and is hence interpretable.

This whole blog post built up to an understanding of how additive models can maintain feature-wise interpretability, and that it’s possible to design a neural network architecture that leverages that. However, the computational graph generalizes even further, and I hope the example above helped demonstrate that. Focus on the math. Focus on the specifics of the problem at hand. You can design interpretable architectures as a function of these feature-networks as long as you meticulously control how the features interact according to the computational graph. Then, because you controlled and isolated features in the computational graph, you can easily discern how those features were used in order to make the final prediction $\hat{y}$ , which is the definition of a transparent model!

References

Hastie & Tibshirani, “Generalized Additive Models”, Statistical Science, 1986. ↩
Friedman & Stuetzle, “Projection Pursuit Regression”, Journal of the American Statistical Association, 1981. ↩
Breiman & Friedman, “Estimating Optimal Transformations for Multiple Regression and Correlation”, Journal of the American Statistical Association, 1985. ↩
Prautzsch, Boehm & Paluszny, Bézier and B-Spline Techniques, 2002. ↩
Hornik, Stinchcombe & White, “Multilayer feedforward networks are universal approximators”, Neural Networks, 1989. ↩
Pinkus, “Approximation theory of the MLP model in neural networks”, Acta Numerica, 1999. ↩
Csáji Balázs, “Approximation with Artificial Neural Networks”, 2001. ↩
Potts, “Generalized Additive Neural Networks”, KDD ‘99, 1999. ↩
Agarwal et al., “Neural Additive Models: Interpretable Machine Learning with Neural Nets”, arXiv:2004.13912, 2020. ↩
Bengio, Léonard & Courville, “Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation”, 2013. ↩

Footnotes

The “smoothness” of a function is described by the continuity of the derivatives. The set of functions with a smoothness of 0 is equivalent to the set of continuous functions. The set of functions with a smoothness of 1 is the set of continuous functions such that their first derivative is continuous. So on, and so forth. Generally, a function is considered “smooth” if it has “smoothness” of $\infty$ . In other words, it is infinitely differentiable. ↩
The actual function being fit here is $f(x_1,x_2) = a(x_1) + b(x_2)$ , however I plot the function $f(x_1 + x_2) = a(x_1) + b(x_2)$ in order to project it as two-dimensional, as that is easier for readers to look at. ↩
Technically, this could be fit where the sub-networks take more than a single feature as input, but this comes at a cost of interpretability. It is still possible to explore the relationship between both features and the output, however it becomes high-dimensional, entangled, and hence more difficult to interpret. ↩
I am setting $f_2$ and $f_3$ as linear models for simplicity in demonstrating a transparent model. This methodology ports over to any function you would want to define as $f_2$ and $f_3$ . ↩

From Linear Models to Neural Networks

Wed, 03 Mar 2021 00:00:00 GMT

Neural Networks (NNs) are an extremely popular type of machine learning algorithm known for, theoretically, being able to approximate any continuous function (Hornik et al., 1989). Furthermore, they are incredibly non-linear and can be difficult to interpret, unlike linear models. However, it is possible to understand how they work with only the math background of linear models.

Linear models are one of the simplest approaches to supervised learning. The general goal of supervised learning is to discover some function $f$ that minimizes an error-term $\epsilon$ given a set of input features $X$ and a corresponding target $y$ such that $y = f(X) + \epsilon$ . Additionally, the output of a supervised model is often written as $\hat{y} = f(X)$ because $f(X)$ is our best approximation of $y$ .

Simple linear models learn to make predictions according to functions of the form:

\begin{aligned} y &= X^T\beta + \epsilon \\ &= \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_n X_n + \epsilon \end{aligned}

Where $\beta_i$ represents learned coefficients with respect to $X_i$ , and $T$ is the transpose operation, which in this case is basically identical to computing the dot product of two n-dimensional vectors.

Linear Regression

Linear regression is arguably the simplest linear model, and comes with four assumptions:

Linearity: The relationship between $X$ and the mean of $y$ is linear.
Independence: The set of feature vectors $[X_1,X_2,\cdots,X_n]$ is linearly independent¹.
Normality: $y$ given any $X$ comes from a normal distribution.
Homoscedasticity: The variance of the error is the same for any value of $X$ .

These assumptions can be nicely described by one math equation:

\begin{aligned} y & \in \mathcal{N}(X^T\beta, \sigma^2 I) \\ & \Rightarrow \mathbb{E}[y|X] = X^T\beta \end{aligned}

Unfortunately, these assumptions are quite rigid for the real world. Many datasets do not conform to these restrictions. So why do we still use linear regression when we have algorithms that can perform the regression task without such rigid assumptions? The common answers to this question are:

Occam’s Razor: Don’t add complexity without necessity.
Little Data: Ordinary Least Squares (OLS) is a closed form solution to linear regression², hence there are no issues with convergence and a solution can always be computed regardless of the amount of data.
Interpretability: $y$ can be explained with respect to how $X$ interacts with the $\beta$ coefficients.

Generalized Linear Models

Generalized Linear Models (GLMs), introduced in (Nelder & Wedderburn, 1972), loosen the constraints of normality, linearity, and homoscedasticity described in the previous section. Furthermore, GLMs break down the problem into three different components:

Random Component: The probability distribution of $y$ (typically belonging to the exponential family³).
Systematic Component: the right side of the equation for predicting $y$ (typically $X^T\beta$ ).
Link Function: A function $g$ that defines the relationship between the systematic component and the mean of the random component.

This yields the following general equation for GLMs:

$g(\mathbb{E}[y|X]) = X^T\beta + \epsilon$

Observe that if the random component is a normal distribution with a constant variance, and the link function is the identity function ( $g(y) = y$ ), then the corresponding GLM is exactly linear regression! Hence, the functions that GLMs can describe are a superset of the functions linear regression can describe.

Selecting a link function according to the random component is what differentiates GLMs. The intuition behind a link function is that it transforms the distribution of $y$ to the range $(-\infty,+\infty)$ , as that is the expected range of $X^T\beta$ . There are a variety of types of link functions depending on the random component. As an example, binary logistic regression assumes the probability distribution of $y$ is a Bernoulli distribution. This means that the average of the distribution, $\mu$ , is between 0 and 1. We need some function $g: [0,1] \rightarrow \mathbb{R}$ , and the logit function is sufficient for this:

$g(\mu) = log(\frac{\mu}{1 - \mu})$

Now, we can fit a simple linear model to $g(y) = X^T\beta + \epsilon$ . Unfortunately, introducing a non-linear transformation to this equation means that Ordinary Least Squares is no longer a reasonable estimation method. Hence, learning $\beta$ requires a different estimation method. Maximum Likelihood Estimation (MLE) estimates the parameters of a probability distribution by maximizing the likelihood that a sample of observed data belongs to that probability distribution. In fact, under the assumptions of simple linear regression, MLE is equivalent to Ordinary Least Squares as demonstrated on page 2 of these CMU lecture notes. The specifics of MLE are not necessary for the rest of this blog post, however if you would like to learn more about it, please refer to these Stanford lecture notes.

The Building Blocks of Neural Networks

How are neural networks (NNs) connected to linear models? Aren’t they incredibly non-linear and opaque, unlike GLMs? Sort of. On the macro-level, NNs and GLMs look very different, but the micro-level tells a different story. Let’s zoom into the inner workings of neural networks and see how they relate to GLMs!

Neural networks are built of components called layers. Layers are built of components called nodes. At their heart, these nodes are computational-message-passing-machines. They receive a set of inputs, perform a computation, and pass the result of that computation to other nodes in the network. These are the building blocks of neural networks.

The first layer of a neural network is called the input layer, because each node passes an input feature to all nodes in the next layer. The last layer of a neural network is called the output layer, and it should represent the output you are trying to predict (this layer has one node in the classic regression case). Lastly, any layers between the input and output layers are called hidden layers.

In the classic fully-connected feed-forward neural network, this structure of layers is ordered and connected such that every node $n_j$ in layer $L_i$ receives the output of every node in the preceding layer $L_{i-1}$ , does some computation with those outputs, and passes the corresponding output to each node in the succeeding layer $L_{i + 1}$ . The image above displays a neural network with $N$ input features, a single hidden layer, and a single output prediction $\hat{y}$ .

Each node in layer $L_i$ contains some set of weights ( $w$ ) and a bias ( $b$ ), where the dimension of the weight vector is equal to the number of nodes in layer $L_{i - 1}$ . When the node receives the output of all the nodes in the preceding layer, it performs the following computation: $L_{i - 1}^Tw + b$ .

This should look familiar! It is quite literally $X^T\beta$ : the classic computation from linear models on the outputs of the preceding layer!

However, before this node passes $X^T\beta$ to the next layer in the network, it is passed through an activation function $f$ . Activation functions often introduce non-linearity to the NN, similar to link functions in GLMs. This non-linearity is important to increase the expressive power of the network. The composition of two linear functions is also linear. Because the computation on the node in a NN is the same as linear models, passing that to a non-linear activation function is necessary to expand the space of functions we can learn to span all continous functions.

The image below isolates a single neuron from the image above, taking input from the previous layer, and making a prediction by transforming the output of the neuron with an activation function $\hat{y} = f(X^T\beta)$ .

In fact, observe that if the activation function is invertible, this computation is equivalent to $f^{-1}(\hat{y}) = X^T\beta$ , which is exactly a GLM with link function $f^{-1}$ . This demonstrates that the computation of a single node in a neural network is, conceptually, a GLM on the output of the previous layer!

Furthermore, this means that a neural network with zero hidden layers and a linear activation function on the output layer is exactly equivalent to linear regression, as the lack of hidden layers maintains independence. And, if we change the activation function to the inverse of the logit function (this is the sigmoid activation function), this neural network becomes exactly equivalent to logistic regression! The code below is a simple prototype of building linear and logistic regression with PyTorch, and tests it on a simulated dataset.

import torch
import torch.nn as nn

class LinearRegression(nn.Module):
    def __init__(self, n_features):
        super().__init__()
        # zero hidden layers with a linear output
        self.output_layer = nn.Linear(n_features, 1)

    def forward(self, x):
        return self.output_layer(x)

class LogisticRegression(nn.Module):
    def __init__(self, n_features):
        super().__init__()
        # zero hidden layers with a sigmoid activation on one output node
        self.output_layer = nn.Linear(n_features, 1)

    def forward(self, x):
        return torch.sigmoid(self.output_layer(x))

Hopefully if the math wasn’t convincing enough, these plots convince you that our neural network with a single input layer, zero hidden layers, and a single output node with a linear/sigmoid activation function is mathematically equivalent to linear/logistic regression.

Now that you have an understanding of all the components of neural networks with respect to linear models, the last piece to understand is the hidden layers.

Understanding Hidden Layers

As described in the previous section, hidden layers are the layers between the input and the output layer. They often have an activation function, which introduces non-linearity to the function we fit. Furthermore, introducing hidden layers are what is responsible for the opacity of neural networks. It’s why they are referred to as “black boxes”, meaning they are difficult to interpret.

This opacity doesn’t actually come from the non-linearity, but rather feature interactions. Because the nodes in a neural network compute the sum of the output of the nodes in the previous layer, once a single hidden layer is introduced, every node after the first hidden layer is computing a function that is dependent on every single input feature. This makes it difficult to see the individual contribution of a single input feature on the output of the network. The image below isolates the computation of single a node in a hidden layer between other hidden layers.

Let’s look at the math for regression using a neural network with $k$ hidden layers, where $w_{i,j}$ and $b_{i,j}$ are the weights and bias of the j^th node in layer $L_i$ with activation function $f_i$ . As additional context, with the notation below, $f_i(L_{i - 1}^Tw_{i,3} + b_{i,3})$ corresponds to the computation of the highlighted node in the image above: the third node in the i^th layer.

\begin{aligned} L_0& = \big [ X_1, X_2, \cdots, X_n \big ] \\ L_1 & = \big [\hspace{0.5em}f_1(L_0^Tw_{1,1} + b_{1,1}), \hspace{0.5em}f_1(L_0^Tw_{1,2} + b_{1,2}), \cdots \big ] \\ & \vdots \\ L_k & = \big [\hspace{0.5em}f_k(L_{k - 1}^Tw_{k,1} + b_{k,1}), \hspace{0.5em}f_k(L_{k - 1}^Tw_{k,2} + b_{k,2}), \cdots \big ] \\ \hat{y} & = L_k^T \beta \end{aligned}

When you think of a hidden layer, think of it as a collection of models. Hidden layer $L_k$ in the math above is kind of like a list of Generalized Linear Models. The first element in $L_k$ is $f_k(L_{k - 1}^Tw_{k,1} + b_{k,1})$ , which is basically like a GLM with link function $f_k^{-1}$ where the input features are the vector $L_{k - 1}$ , the outputs of the nodes in the preceding layer. However, the analogy to GLMs is not perfect for a few reasons. First, The elements in $L_{k - 1}$ are not independent because they all are linear combinations of the input features. And second, activation function $f_k$ isn’t always invertible (e.g. ReLU. The point I am trying to make is that the math is familiar, as it’s intimately related to that of GLMs.

So why does this work so well? What is the intuition behind it?

Look at the last line in the equation above. The ouput prediction: $\hat{y} = L_k^T \beta$ . If we think of $L_k$ as a collection of non-linear models dependent on every input feature as I described above, then we can also think of the output of each model in $L_k$ as an engineered feature. The purpose of hidden layers is basically automated feature engineering. The last hidden layer contains a bunch of engineered features that come from adding up a bunch of linear combinations of the input features and applying non-linear activation functions. These engineered features are then thrown into a linear model to predict the target.

So, neural networks really aren’t that different from linear models. They’re just instructions for learning the best features one can engineer from the input features, and predicting a linear combination of the engineered features!

Ryan Saxe's Blog

Designing Transparent Neural Networks

Generalized Additive Models

Transparent Neural Networks

Generalized Additive Neural Networks

Neural Additive Models

Designing Your Computational Graph

References

Footnotes

From Linear Models to Neural Networks

Linear Regression

Generalized Linear Models

The Building Blocks of Neural Networks

Understanding Hidden Layers

Further Reading

References

Footnotes