Know Why Generalized Linear Model is a Remarkable Synthesis Model!

Understanding the basics

GLM is very famous among individuals who deal with different regression models starting from Classical Linear Regression Models to Models for Survival Analysis. The term generalized linear model (GLIM or GLM) was coined and familiarized by McCullagh (1982) and Nelder (2nd edition 1989). GLM, in the simplest form as described in Rutherford 2001,  Data = Model + Error. It has a useful framework, which is the foundation of various statistical tests. 

Top Machine Learning Courses & AI Courses Online

Revisiting the class of models

• Classical Linear Regression (CLR) Models, also referred to as Linear Regression models
• Analysis of Variance (ANOVA) models.
• Models which predict the odds of winning like the probability of machine failure 
• Models used for explaining and predicting event counts
• Models for estimating lifespans of living and non-living things such as a processor or biological age of a plant etc.

Generalized Linear Model, as the name suggests, is like a canopy for all the above-given models with improved calculations and approximations.

Trending Machine Learning Skills

Enrol for the Machine Learning Course from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

The Structure of Generalized Linear Models

A generalized linear model (or GLM1) consists of three major components:  

1. Random Component: A random component known as a noise model or error model is the probability distribution of the response variable (Y).
2. Systematic Component : A linear predictor is a linear function of regressors, as mentioned below:

 ηi = α + β1Xi1 + β2Xi2 +···+ βkXik   

3. Link Function (denoted by η or g(μ) ): As the name suggests its the link between systematic and random components

Example : μi = E(Yi), to the linear predictor g(μi) = ηi = α + β1Xi1 + β2Xi2 +···+ βkXik

Generalized Linear Model applies to data by the process of maximum likelihood. This provides the estimates of the regression coefficients and estimated asymptotic standard errors of the coefficients.

The basic GLM for count data is the Poisson model with a log link. However, when the response variable is a count, its conditional variance increases more rapidly than its mean, producing a condition termed overdispersion and invalidating the use of the Poisson distribution. The quasi-Poisson GLM adds a dispersion parameter to handle overdispersed count data.

In general terms, quasi-likelihood estimation is one way of allowing for overdispersion, which is more significant variability in the data than expected from the statistical model used.

A similar model is based on the negative binomial distribution, which is not an exponential family. Negative-binomials in Generalized Linear Model cannot be determined by maximum likelihood. The zero-inflated Poisson regression model may be best suitable when there are more zeroes in the data than consistent with a Poisson distribution.

Read: Machine Learning Models Explained

Advantages of the Generalized Linear Model over traditional Ordinary Least Square (OLS) regression

There are many advantages of General Linear Models over the OLS regression, which can be summarised as below:

• Unlike OLS regression, the response Y is not required to be transformed every time to have a normal distribution. 
• Modeling is more flexible as choosing a link is different from choosing a random component.
• A constant variance is NOT needed if the link gives additive effects.
• We have the optimal properties of the estimators as the models are attached via Maximum Likelihood estimation.
• All the inference tools and model checking for log-linear and logistic regression models apply for other GLMs too.
• There is usually only one process (procedure or function) in a software package to capture all the models listed in the table above; take, for instance, glm() (R Language) or PROC GENMOD (SAS). 

Must Read: Machine Learning Project Ideas Explained

Disadvantages of the Generalized Linear Model

Apart from the above-listed advantages, there are two major disadvantages which are important to know:

• Some restrictions like Linear function can have only a linear predictor in the systematic component.
• Responses cannot depend on each other. 

Popular AI and ML Blogs & Free Courses

Conclusion 

If we summarize all the above information, we found that GLM is convenient with lower complexity. With GLM, response variables can have any form of exponential distribution type. Apart from this, it can deal with categorical predictors. The general linear model is relatability easy to interpret and allows a clear understanding of how each predictor is influencing the outcome.

If you’re interested to learn more about machine learning, check out IIIT-B & upGrad’s PG Diploma in Machine Learning & AI which is designed for working professionals and offers 450+ hours of rigorous training, 30+ case studies & assignments, IIIT-B Alumni status, 5+ practical hands-on capstone projects & job assistance with top firms.