【Kaggle Method】ISIC2024 7th solution explained

This time, I'll explain the 7th solution of the ISIC2024 Kaggle competition.

Original Solution:
https://www.kaggle.com/competitions/isic-2024-challenge/discussion/532687

0. Highlights

Chapter 2: CV strategy that trained on data from five hospitals and predicted on seven hospitals. It makes sense because this strategy could imitate the real test set distribution with high quality.
Chapter 3.2: Multi-head model for target and lesion_id, helps model generalization and expression ability.

1. Overview

Their solution is the weighted ensemble of three tree-based models. In each tree-based model, they used the metadata and the prediction from CNN models as features.
For CNN models, they trained multi-head models (1 for the classification target, 1 for predicting lesion_id) using variants of backbone.

2. CV Strategy

They tried three cv strategies.

1. GroupKFold on patient_id
2. GroupKFold on patient_id and stratified on attribution (hospitals).
3. Trained on data from five hospitals and predicted on seven hospitals(dataset divided depends on attribution).

1 and 2 are not correlated to LB.
In the discussion, the scenario that the test dataset contains two unknown hospitals, so they tried the third strategy. It has a high correlation with LB and it was used until the end.

3. Image Models(CNN)

3.1 Models

・ResNet18
・EfficientNetB0
・EfficientNetB1
・Swin_Small
as backbones.

3.2 Other Things

・Another Head:
They thought that the data has lesion_id would be more suspicious than other samples. So they leveraged this lesion_id information by adding another head for lesion_id. This helped improve the pauc of CNN compared to using the classification target only.

・Pre-training:
Pre-trained on data from previous ISIC competitions.

・Ensemble:
The results of combining image and tabular data were better than using image data alone. But they didn't use it because it takes a lot of inference time.

4. Tree-based models

・Models:
CatBoost, XGBoost, and LGBM.
・Feature:
From the public kernel.
Group and aggregation information by patient_id, tbp_lv_location, and attribution.
・Sample weight: 1 for negative, x for lesion and y for positive (the tuned value for x and y vary on different tree lib).
・Cross-feature (OOF prediction) from CNN models.

5. Submission

The final submission was a weighted ensemble of three tree-based models and the neural network model on tabular data.
Their submissions with the highest score on CV are also the highest score on both public LB and private LB.

6. What didn't work

・Multi-label for CNN models with the target and confidence score.
・Stacking is complicated and not as good as the weighted ensemble.
・SVM, TabNet, and other classifiers.
・Remove hair from skin images using OpenCV as in the discussion.
・Under-sampling as suggested on the public kernel.

7. Summary

This time, I looked back at the 7th solution of ISIC2024, looking back at the solution gives learning and fun.

References

[1] https://www.kaggle.com/competitions/isic-2024-challenge/discussion/532687
[2] https://www.kaggle.com/code/greysky/isic-2024-only-tabular-data
[3] https://www.kaggle.com/competitions/isic-2024-challenge/discussion/519735