Using Decision tree creates a step function, on the basis of which regression and classification is done. But a step function has many corners in it which make the graph discontinuous and also slightly biased in prediction on that particular corner points. So would'nt it be better if we can use curve smoothing on the model to make the model more slightly more accurate and unbiased.
In Decision trees the vertical lines indicate the decision boundaries while the horizontal ones indicate the average of the points (or values) so in essence, smoothing the curve would mean increasing the number of decision boundaries which will happen as we go further and further down the tree creating more and more branches. In doing so we would be taking into account the noise points and hence will be overfitting the model to the data, which we wouldn't want to do.
For the kind of curve smoothing you are suggesting, you have to resort to Polynomial Regression.
Please login to comment
4 Comments
This comment has been removed.
Using Decision tree creates a step function, on the basis of which regression and classification is done. But a step function has many corners in it which make the graph discontinuous and also slightly biased in prediction on that particular corner points. So would'nt it be better if we can use curve smoothing on the model to make the model more slightly more accurate and unbiased.
Upvote ShareHi,
If we try to fit the curve, would it not ovefit? Let me know what do you think.
Thanks.
1 Upvote ShareIn Decision trees the vertical lines indicate the decision boundaries while the horizontal ones indicate the average of the points (or values) so in essence, smoothing the curve would mean increasing the number of decision boundaries which will happen as we go further and further down the tree creating more and more branches. In doing so we would be taking into account the noise points and hence will be overfitting the model to the data, which we wouldn't want to do.
For the kind of curve smoothing you are suggesting, you have to resort to Polynomial Regression.
Upvote Share