Which regression to use

Use Poisson regression to model how changes in the independent variables are associated with changes in the counts. Poisson models are similar to logistic models because they use Maximum Likelihood Estimation and transform the dependent variable using the natural log. For example, homicides per month.

Example : An analyst uses Poisson regression to model the number of calls that a call center receives daily. Not all count data follow the Poisson distribution because this distribution has some stringent restrictions.

Fortunately, there are alternative analyses you can perform when you have count data. Negative binomial regression : Poisson regression assumes that the variance equals the mean. When the variance is greater than the mean, your model has overdispersion. A negative binomial model, also known as NB2, can be more appropriate when overdispersion is present. Zero-inflated models : Your count data might have too many zeros to follow the Poisson distribution.

In other words, there are more zeros than the Poisson regression predicts. Zero-inflated models assume that two separate processes work together to produce the excessive zeros.

One process determines whether there are zero events or more than zero events. The other is the Poisson process that determines how many events occur, some of which some can be zero. An example makes this clearer! Suppose park rangers count the number of fish caught by each park visitor as they exit the park. A zero-inflated model might be appropriate for this scenario because there are two processes for catching zero fish:.

Originally posted here. Views: Share Tweet Facebook. Join Data Science Central. Sign Up or Sign In. Powered by. To not miss this type of content in the future, subscribe to our newsletter. Archives: Book 1 Book 2 More. Follow us : Twitter Facebook. Write For Us 7 Tips for Writers. Regression Analysis with Continuous Dependent Variables Regression analysis with a continuous dependent variable is probably the first type that comes to mind.

Linear regression OLS produces the fitted line that minimizes the sum of the squared differences between the data points and the line. There are some special options available for linear regression. Most companies use regression analysis to explain a phenomenon they want to understand e. Whenever you work with regression analysis or any other analysis that tries to explain the impact of one factor on another, you need to remember the important adage: Correlation is not causation.

The regression shows that they are indeed related. The goal is not to figure out what is going on in the data but to figure out is what is going on in the world.

Redman wrote about his own experiment and analysis in trying to lose weight and the connection between his travel and weight gain. He noticed that when he traveled, he ate more and exercised less. So was his weight gain caused by travel? Not necessarily. He had to understand more about what was happening during his trips.

And this is his advice to managers. Use the data to guide more experiments, not to make conclusions about cause and effect. Always ask yourself what you will do with the data. What actions will you take? What decisions will you make? The chart below explains how to think about whether to act on the data. After you fit your model, determine whether it aligns with theory and possibly make adjustments.

For example, based on theory, you might include a predictor in the model even if its p-value is not significant. If any of the coefficient signs contradict theory, investigate and either change your model or explain the inconsistency. You might think that complex problems require complex models, but many studies show that simpler models generally produce more precise predictions.

Given several models with similar explanatory ability, the simplest is most likely to be the best choice. Start simple, and only make the model more complex as needed.

The more complex you make your model, the more likely it is that you are tailoring the model to your dataset specifically, and generalizability suffers. Verify that added complexity actually produces narrower prediction intervals.

As you evaluate models, check the residual plots because they can help you avoid inadequate models and help you adjust your model for better results. For example, the bias in underspecified models can show up as patterns in the residuals, such as the need to model curvature. The simplest model that produces random residuals is a good candidate for being a relatively precise and unbiased model.

In the end, no single measure can tell you which model is the best. Statistical methods don't understand the underlying process or subject-area. Your knowledge is a crucial part of the process! Minitab Blog. They strive to achieve a Goldilocks balance with the number of predictors they include. You could also use transformations to correct for heteroscedasiticy, nonlinearity, and outliers.

Some people do not like to do transformations because it becomes harder to interpret the analysis. Thus, if your variables are measured in "meaningful" units, such as days, you might not want to use transformations.

If, however, your data are just arbitrary values on a scale, then transformations don't really make it more difficult to interpret the results. Since the goal of transformations is to normalize your data, you want to re- check for normality after you have performed your transformations. Deciding which transformation is best is often an exercise in trial-and-error where you use several transformations and see which one has the best results.

The specific transformation used depends on the extent of the deviation from normality. If the distribution differs moderately from normality, a square root transformation is often the best.

A log transformation is usually best if the data are more substantially non-normal. An inverse transformation should be tried for severely non-normal data. If nothing can be done to "normalize" the variable, then you might want to dichotomize the variable as was explained in the linearity section. Direction of the deviation is also important. If the data is negatively skewed, you should "reflect" the data and then apply the transformation. To reflect a variable, create a new variable where the original value of the variable is subtracted from a constant.

The constant is calculated by adding 1 to the largest value of the original variable. If you have transformed your data, you need to keep that in mind when interpreting your findings.

For example, imagine that your original variable was measured in days, but to make the data more normally distributed, you needed to do an inverse transformation. Now you need to keep in mind that the higher the value for this transformed variable, the lower the value the original variable, days. A similar thing will come up when you "reflect" a variable. A greater value for the original variable will translate into a smaller value for the reflected variable.

Simple Linear Regression Simple linear regression is when you want to predict values of one variable, given values of another variable.

For example, you might want to predict a person's height in inches from his weight in pounds. Imagine a sample of ten people for whom you know their height and weight. You could plot the values on a graph, with weight on the x axis and height on the y axis. If there were a perfect linear relationship between height and weight, then all 10 points on the graph would fit on a straight line.

But, this is never the case unless your data are rigged. If there is a nonperfect linear relationship between height and weight presumably a positive one , then you would get a cluster of points on the graph which slopes upward. In other words, people who weigh a lot should be taller than those people who are of less weight. See graph below. The purpose of regression analysis is to come up with an equation of a line that fits through that cluster of points with the minimal amount of deviations from the line.

The deviation of the points from the line is called "error. Simple linear regression is actually the same as a bivariate correlation between the independent and dependent variable.

Standard Multiple Regression Standard multiple regression is the same idea as simple linear regression, except now you have several independent variables predicting the dependent variable. To continue with the previous example, imagine that you now wanted to predict a person's height from the gender of the person and from the weight.

You would use standard multiple regression in which gender and weight were the independent variables and height was the dependent variable. The resulting output would tell you a number of things. First, it would tell you how much of the variance of height was accounted for by the joint predictive power of knowing a person's weight and gender. This value is denoted by "R2". The output would also tell you if the model allows you to predict a person's height at a rate better than chance.

This is denoted by the significance level of the overall F of the model. If the significance is. In other words, there is only a 5 in a chance or less that there really is not a relationship between height and weight and gender.

For whatever reason, within the social sciences, a significance level of. If the significance level is between. In addition to telling you the predictive value of the overall model, standard multiple regression tells you how well each independent variable predicts the dependent variable, controlling for each of the other independent variables.

In our example, then, the regression would tell you how well weight predicted a person's height, controlling for gender, as well as how well gender predicted a person's height, controlling for weight. To see if weight was a "significant" predictor of height you would look at the significance level associated with weight on the printout.