Machine Learning and Optimization

• Machine learning models optimize the function by minimizing the loss
• The process of optimizaion for the machine learning function represents learning

Linear Regression

• Basic machine learning and statistical analysis method which assumes the linear relationship between input and output
• We estimate the optimal coefficients to minimize the loss function

• Linear regression is one of the basic and fundamental machine learning methods
  

➡️ y = 2x

➡️ Learning is Optimization !!

Loss Function of Neural Networks

Iterative Algorithm-based Optimization

Gradient Descent

• loss 최소 부분을 찾는 것이 머신러닝의 핵심

등고선 Contour

• 3차원 -> 2차원
  

경사 하강법 심화

Learning Rate

• Learning Rate 를 적절하게 책정해야 한다.
• 너무 작아도 문제, 너무 커도 문제

Stochastic Gradient Descent

• Consider minimizing an average of functions
  

Global vs Local Minimum

Momentum

• 관성의 법칙에 따라 이동

reference : K-MOOC 실습으로 배우는 머신러닝