【ML】GBDT Libraries Summay
0. Summary of GBDT Libraries
There are three popular libraries in GBDT, LGBM, CatBoost, XGBoost.
This time, I summarize the features of each libraries briefly, the detail explanation is written in another article, so please check it when you need.
1. LGBM
Features
・Fast
・Accurate
・Memory Efficiency
Article
Detail Explanation with code.
2. CatBoost
Features
・Can handle categorical features as is.
・Less overfitting
・Robust to hyperparameters
Article
Detail Explanation with code.
3. XGBoost
Features
・Great scalability and flexibility.
Article
Detail Explanation with code.
4. In the end
The ensemble of models generally performs a greater score than one model. So, each library has each feature, please consider using all as models that are to be ensembled.
・Ensemble method
In my experience, the blending works in following order;
①good and highly different models > ②not good but highly different models ≒ ③good and slightly different models >> ④not good and not different models.
quote: [3]
Reference
[1] https://qiita.com/Ak_ki/items/70572ae1c1728335219e
[2] https://qiita.com/Ak_ki/items/38b85992547454005533
[3] https://www.kaggle.com/competitions/open-problems-multimodal/discussion/363230
Discussion