Gradient Boost Part 1 (of 4): Regression Main Ideas

Описание

Gradient Boost is one of the most popular Machine Learning algorithms in use. And get this, it's not that complicated! This video is the first part in a series that walks through it one step at a time. This video focuses on the main ideas behind using Gradient Boost to predict a continuous value, like someone's weight. We call this, "using Gradient Boost for Regression". In the next video, we'll work through the math to prove that Gradient Boost for Regression really is this simple. In part 3, we'll walk though how Gradient Boost classifies samples into two different categories, and in part 4, we'll go through the math again, this time focusing on classification.

This StatQuest assumes that you already understand....
Decision Trees: https://youtu.be/_L39rN6gz7Y
Regression Trees: https://youtu.be/g9c66TUylZ4
AdaBoost: https://youtu.be/LsK-xG1cLYA
...and the tradeoff between Bias and Variance that plagues Machine Learning: https://youtu.be/EuBBz3bI-aA

For a complete index of all the StatQuest videos, check out:
https://statquest.org/video-index/

This StatQuest is based on the following sources:

A 1999 manuscript by Jerome Friedman that introduced Stochastic Gradient Boost: https://statweb.stanford.edu/~jhf/ftp/stobst.pdf

The Wikipedia article on Gradient Boosting: https://en.wikipedia.org/wiki/Gradient_boosting

The scikit-learn implementation of Gradient Boosting: https://scikit-learn.org/stable/modules/ensemble.html#gradient-boosting

If you'd like to support StatQuest, please consider...

Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - https://statquest.gumroad.com/l/wvtmc
Paperback - https://www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - https://www.amazon.com/dp/B09ZG79HXC

Patreon: https://www.patreon.com/statquest
...or...
YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

...a cool StatQuest t-shirt or sweatshirt:
https://shop.spreadshirt.com/statquest-with-josh-starmer/

...buying one or two of my songs (or go large and get a whole album!)
https://joshuastarmer.bandcamp.com/

...or just donating to StatQuest!
https://www.paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
https://twitter.com/joshuastarmer

0:00 Awesome song and introduction
2:58 Gradient Boost compared to AdaBoost
5:50 Building the first tree to predict weight
10:37 Building the second tree to predict weight
13:28 Building additional trees to predict weight
13:50 Prediction with Gradient Boost
14:28 Summary of concepts and main ideas

#statquest #gradientboost