InsideDarkWeb.com

Normalizing zero inflated predictors for multiple regression

Hope I got it right, as this is my first active post 🙂
I was trying to find a solution the whole day for my problem.

I am trying to predict a continuos variable based on 20 different predictors. The predicted variable is an average m’3 per transportation unit across all palets. the 20 different predictors are the ordered volume per department.

The data looks like this:

RAW DATA

You can see the predictors in the columns VS01 – VS20. Lots of them are zero, because not every order buys something from a department.

In order to create a linear regression I was trying to normalize data with different approaches.

  • Log is not working due to the zeros
  • Square Root is giving me kind of good distribution but does not solve the issue with the zeros
  • Yeo Johnson is the same as the Sqare Root try

What I’ve found is that the solutions for zero inflated issues are focusing only on the predicted value, not on the predictors.

So I am trying to ask the community, if you have another approach to try out?

Bellow you can find the 2nd picutre about 2 of the department’s volume from the orders. Almost all of them look like this:

  • Once transformed with Square Root
  • Once transformed with Yeo Johnson
  • Once the original data set, where it is visible that the data is not only skewed also that there is the gap between the zeros and the next bin of values

Transformations

Additional what would be nice to understand a bit more is how to transform such a non linear distribution into something normal distributed

NonLinear

What I would also be open, if someone could suggest a different approach than linear regression. Maybe:

  • A decision tree approach
  • A PCA approach

Thanks a lot

Cross Validated Asked by Eugen Cuic on November 14, 2021

0 Answers

Add your own answers!

Related Questions

A/B Testing – Repeated Sampling

0  Asked on January 18, 2021 by thomas-moore

   

Clustering with Likert items and N/A option

1  Asked on January 17, 2021 by matthias

     

How to derive Bias of PCA estimator

0  Asked on January 16, 2021 by baz

     

Mean squared error of OLS smaller than Ridge?

5  Asked on January 15, 2021 by aristide-herve

       

Ask a Question

Get help from others!

© 2021 InsideDarkWeb.com. All rights reserved.