Are “complex” ML Models Always Better?

Share on facebook
Share on twitter
Share on linkedin
Share on email
Share on whatsapp
Share on telegram
Share on pocket

After mentoring data professionals for a few years, I notice that there is always a handful of recent graduates (from universities or boot camps) who are caught by the mental trap of “always go for the most COMPLEX machine learning model!”

For instance, some of them would always go for stepwise regression (bi-directional elimination) to estimate customer lifetime value. When asked the reason why they are not testing with forward selection and backward elimination as well, the usual answer is, “I did not do so because I thought that stepwise regression is sufficient given its complexity.”

So let me set this straight. When we are doing supervised learning, we have a target (Y) and several features (Xs). Our job as data scientists is to find the “hidden” relationship between the target – what we want to predict – from the individual features. Now I do not deny that complex machine learning models do help us to model complex relationships (a mixture of quadratic, cubic, setting “curly” decision boundaries etc) but at the end of the day, we must know that it is a “curve fitting” exercise, as mentioned by Judea Pearl in this article. Now the “curve” need not necessary be curved, but can be a linear relationship (a.k.a when X changes, Y also proportionally changes as well).

What I am saying is, we cannot totally discount a machine learning model just because it is not “complex” enough. We should only discount a machine learning model when it has been tested and we do not get better “predictions” from it (not modelling the relationship between the target and features accurately) or it does not meet the business requirements, for instance not meeting the transparency needed, or the cost of implementation is prohibitive.

In conclusion, as data scientists, we need to test all possible machine learning algorithms, regardless of its “complexity” (usually defined by how complicated the underlying math is) as each class of machine learning model provides a different way to “curve fit” the hidden relationship between the features and the target.

Do keep this in mind at all times! Have fun in your data science learning journey! If the post has been useful, do share it.

Author

  • Koo is an experienced Data Scientist & Instructor with an MBA degree with more than 15 years relevant experience. He is a strong and passionate advocate of Data Science & Artificial Intelligence, hoping more people, both business and talents, get on the bandwagon to derive tremendous value from DS/AI. He co-founded DataScience.Sg, one of the largest and most active Data Science community in Singapore. Koo is also experienced educator, conducting Data Science training in many institutions such as NUS SCALE and SMU Academy. https://www.linkedin.com/in/koopingshung/

Share on facebook
Share on twitter
Share on linkedin
Share on email
Share on whatsapp
Share on telegram
Share on pocket

Leave a Comment

Previous

The Most Important Data Science Tool for Market and Customer Segmentation

Train a Viable Model in 45 minutes for AWS DeepRacer Beginner Challenge Virtual Community Race 2020

Next