【ML】Explanation ML methods without Neural Network Part1

I noticed that I don't know details about machine laerning method without Newral Network while I learned about machine learning until now, so I summarize those this time.

1. Linear Regression

It's a simple method. This serch the line that most fitting feature and target by calculate loss like MAE, RMSE.

It return the data based to serached line when feeded new unknown data.

2. Logistic Regression

Logistic regression estimate the coefficients of linear function by gradients from maximum likelihood estimation (MLE) not by minimizing a typical loss function directly like mean squared error used in linear regression.

・Regression
$z = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + ... + \beta_n X_n$
$X$: Input features
$\beta$: Estimated parameters(coefficients(\beta_1,\beta_2...) and bias($\beta_0$))

logistic regression uses the log-likelihood as a function to maximize, which reflects how well the model parameters are predicting the actual class labels.
Gradient descent or other optimization algorithms can be used to find the parameter values that maximize this likelihood function.

It has same output with linear regression apparently, but internal process is quite different. It output unlimited value same as Linear Regression, but the output will modified to limited value by sigmoid function, and get few nonlinearity.

3. Decision Trees

A decision tree is a type of machine learning algorithm that is used for both classification and regression tasks. It's a model that predicts the value of a target variable by learning simple decision rules(ex: a input value over 5) inferred from the data features.

At each node, the algorithm selects the feature and a split point that best separates the data into branches. The "best" split is typically determined by measuring how well the classes are separated. Metrics such as Gini impurity or entropy are used in classification trees, and variance reduction is used in regression trees.

Advantages of decision trees include high interpretability, no need for feature normalisation, and the ability to handle both numerical and categorical data.

Summary

These are so famous machine learning method, and it even also useful. Especially, used when tackle non-linear problem or ensemble some complex model.

The next article, I'll explain about Support Vector Machines (SVM), K-Nearest Neighbors (KNN), and heuristics method(not ML though).