Leetcode
ML Code
Gradient Descent
- define loss function and solve
- if gradient exploding ➡ clipping
K-Means
Logistic Regression
- how to take derivatives?
- how to convert to matrix multiplication?
Linear Regression
- closed form?
- using gradient descent
Attention
- self attention
- cross attention
other ML algos from scratch
Project / Experience Specific
General ML Knowledge
Metrics
- accuracy
- precision
- recall
- F1
- ROC-AUC
naive bayes
- how to derive from original form to naive form?
SVM
- what’s supporting vector?
- what’s advantage
- why it’s not easy to overfitting?
- hinge loss
Logistic Regression
- how to get its loss function
Tree
- ID3
- C4.5
- CART
- pruning
- random forest
- adaboost
- gbdt
unsupervised
- K-Means
- PCA
DL Knowledge
NN
CNN
RNN
LSTM
Transformer
NLP stuffs
- like word2vec
(optional) recommendation system
- content based
- collaborative filtering
- matrix factorization
- DeepFM
- GNN
- lightGCN
Other Chores
Dropout
Activation Functions
- sigmoid
- tanh
- relu
- leaky relu
- prelu
- swish
Normalization
- batch norm
- layer norm
KD Divergence
Initialization
- Xavier
- He
- why didn’t random nor all 0 work?
Optimizer
- SGD
- momentum SGD
- adagrad
- rmsprop
- adam
ML Sys Design
➡ bytebytego