Function Approximator
) Neural networks are function approximators that stack affine transformations followed by nonlinear transformations.
Eval
과정은 with torch.no_grad():
설정해줘야 함iterator
통해, 돌면서 Batch Size
만큼 뽑아내며 eval
과정 처리.to(device)
이용해 Batch Size
만큼의 X 데이터/y 데이터로부터 예측 값/정답 값 정의view()
함수 : 필요 시, reshapedef func_eval(model,data_iter,device):
with torch.no_grad():
model.eval() # evaluate (affects DropOut and BN)
n_total,n_correct = 0,0
for batch_in,batch_out in data_iter:
y_trgt = batch_out.to(device)
model_pred = model(batch_in.view(-1, 28*28).to(device))
_,y_pred = torch.max(model_pred.data,1)
n_correct += (y_pred == y_trgt).sum().item()
n_total += batch_in.size(0)
val_accr = (n_correct/n_total)
model.train() # back to train mode
return val_accr
print ("Done")
전체 Dataset에서 정의한 iterator
통해, 돌면서 Batch Size
만큼 뽑아내며 train
과정 처리
[1] 포워드 과정
- (1)위에서 정의한
CustomModel
의forward(self, x)
함수와 (2)선택한loss
함수(크로스엔트로피 등)에 배치를 돌면서피드포워드
[2] 옵티마이저 및 로스 업데이트 과정 (backpropagation
)
- (1)
옵티마이저.zero_grad()
로 값 리셋, (2)출력로스.backward()
로 미분값 백워드, (3)옵티마이저.step()
로 값 업데이트
- code
print ("Start training.")
M.init_param() # initialize parameters
M.train()
EPOCHS,print_every = 10,1
for epoch in range(EPOCHS):
loss_val_sum = 0
for batch_in,batch_out in train_iter:
# Forward path
y_pred = M.forward(batch_in.view(-1, 28*28).to(device))
loss_out = loss(y_pred,batch_out.to(device))
# Update
optm.zero_grad() # reset gradient
loss_out.backward() # backpropagate
optm.step() # optimizer update
loss_val_sum += loss_out
loss_val_avg = loss_val_sum/len(train_iter)
# Print
if ((epoch%print_every)==0) or (epoch==(EPOCHS-1)):
train_accr = func_eval(M,train_iter,device)
test_accr = func_eval(M,test_iter,device)
print ("epoch:[%d] loss:[%.3f] train_accr:[%.3f] test_accr:[%.3f]."%
(epoch,loss_val_avg,train_accr,test_accr))
print ("Done")
Generalization
Cross-validations
Bias와 Variance
torch.optim
라이브러리 활용해 선언momentum
옵티마이저는 momentum=(확률값)
옵션 추가- code
### 임포트 : import torch.optim as optim
LEARNING_RATE = 1e-2
# Instantiate models
model_sgd = Model(name='mlp_sgd',xdim=1,hdims=[64,64],ydim=1).to(device)
model_momentum = Model(name='mlp_momentum',xdim=1,hdims=[64,64],ydim=1).to(device)
model_adam = Model(name='mlp_adam',xdim=1,hdims=[64,64],ydim=1).to(device)
# Optimizers
loss = nn.MSELoss()
optm_sgd = optim.SGD(model_sgd.parameters(), lr=LEARNING_RATE)
optm_momentum = optim.SGD(model_momentum.parameters(), lr=LEARNING_RATE, momentum=0.9)
optm_adam = optim.Adam(model_adam.parameters(), lr=LEARNING_RATE)
print ("Done.")
Adam
Adam
은 짧은 학습 시간부터 성능(정확도)이 확보됨Adam
에는adaptive learning
개념이 반영된 접근법. (adaptive learning
: 어떠한 파라미터에 대해서는 lr을 높이고, 다른 어떠한 파라미터에 대해서는 lr을 줄여나감)- 따라서, 똑같은 lr을 선언해주어도 더 빠르게 성능이 확보될 수 있다..! ✨
Momentum
은 학습 시간이 조금 길어지면 성능 확보됨. 반면, SGD는 같은 시간까지도 성능이 확보되지 않음Momentum
은 이전 배치의gradient
정보를 활용해 현재 배치 턴에서 사용하겠다는 접근법- 이러한 점에서 미니배치일 때,
SGD
보다 좋다..! ✨
SGD
는 데이터에서 큰 특징이 되는 파트 위주로 학습이 잘되고, 세부적으로는 잘 놓치는 모양새를 보임- 하지만,
SGD
는 오랜 학습 시간이 흘렀을 때에는 더 좋은 성능을 보일 순 있다.
Adam
/rAdam
을 초기에 사용하면 어느정도의 성능을 짧은 시간에 효율적으로 확보할 수 있다..! ✨Wonderful Content, Now i'm a massive believer for offer suggestions relating to internet websites to make sure you let the web page creators understandthat they’ve incorporated an issue good to make sure you just a couple of! ufamobile
This is cool post and i quite like to see this post. your site is fantastic and you have good staff in your blog. nice sharing thanks. łóżko dla chłopca
i read a lot of stuff and i found that the way of writing to clearifing that exactly want to say was very good so i am impressed and ilike to come again in future..http://wwscc.org/evinfo/pages/1xbet_promo_code_for_registration___sign_up_bonus_india.html
Double your deposit with Linebet’s promo code BNB777! Claim a 100% bonus up to €130. Enjoy free spins, sports bets, and exclusive rewards today.code promo linebet cote d'ivoire
This really which means delightful not to mention original. I just absolutely adore typically the styles not to mention anyone who will become it again in your mailing could be cheerful. mimi t shirt
I’m inspired with the surpassing and preachy listing that you furnish in such little timing. 江古田英会話
Fantastic Put up, I'm sure a giant believer through post feed-back concerning webpages towards allow the blog page consultants recognise that they’ve further a specific thing positive towards the web! door handles
Great submit, the an extremely great website which you have the following, maintain the nice perform, will probably be again. toto12
Positive site, where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work. Online Harassment Lawyer
You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. 五反田 英語 アルバイト
Your current tunes can be remarkable. You've got a number of quite accomplished musicians. My spouse and i would like anyone the top involving good results. 신용카드 현금화
Wow, What an Outstanding post. I found this too much informatics. It is what I was seeking for. I would like to recommend you that please keep sharing such type of info.If possible, Thanks. 해외선물 분양
POL88 adalah situs slot online gacor terpercaya dengan RTP tinggi dan game dari provider ternama. Daftar sekarang untuk nikmati jackpot besar serta bonus harian yang banyak. slot gacor
Great article, Thanks a lot pertaining to expressing This specific know-how. Outstandingly prepared content, only when most web owners presented a similar a higher level written content because you, the world wide web has to be superior position. Remember to continue! compare broadband services
This article was written by a real thinking writer. I agree many of the with the solid points made by the writer. I’ll be back. jay rufer
Keep up the good work , I read few posts on this web site and I conceive that your blog is very interesting and has sets of fantastic information. agenolx
Really a great addition. I have read this marvelous post. Thanks for sharing information about it. I really like that. Thanks so lot for your convene. BATMAN138
It all is impressive to read the paper these types of interesting together with different article content for your web-sites. ผลการแข่งขัน
Thank you for such a well written article. It’s full of insightful information and entertaining descriptions. Your point of view is the best among many. DeepL
This is my first time i visit here and I found so many interesting stuff in your blog especially it's discussion, thank you. sandibet
A very awesome blog post. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. hptoto alternatif
A very awesome blog post. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. pass prop firm
A very awesome blog post. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. toto login
Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. olxtoto
Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. situs togel terpercaya
I’m going to read this. I’ll be sure to come back. thanks for sharing. and also This article gives the light in which we can observe the reality. this is very nice one and gives indepth information. thanks for this nice article... hptoto situs slot
Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. sandibet
You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers.olxtoto
Excellent article. Very interesting to read. I really love to read such a nice article. Thanks! keep rocking. slot qris
It was wondering if I could use this write-up on my other website, I will link it back to your website though.Great Thanks. Bulenox coupon code
Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.gaspol189
It was wondering if I could use this write-up on my other website, I will link it back to your website though.Great Thanks. bandar toto macau
Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.M88
I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.situs toto
I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.situs toto
I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.situs toto
I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.situs toto
I think this is an informative post and it is very useful and knowledgeable. therefore, I would like to thank you for the efforts you have made in writing this article.olxtoto
Thanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info. keluaran toto macau
Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.keytoto
Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.keytoto
Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.olxtoto
Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. olxtoto link alternatif
Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. toto slot 777 login
Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.olxtoto
This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here keep up the good worktoto togel
This is my first time i visit here and I found so many interesting stuff in your blog especially it's discussion, thank you.toto 4d
Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. togel online
Thanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info.keluaran toto macau
I have read all the comments and suggestions posted by the visitors for this article are very fine,We will wait for your next article so only.Thanks! olxtoto
Your own songs is actually incredible. You've a few really gifted designers. We desire a person the very best associated with achievement. olxtoto alternatif Your own songs is actually incredible. You've a few really gifted designers. We desire a person the very best associated with achievement. daftar evostoto Your own songs is actually incredible. You've a few really gifted designers. We desire a person the very best associated with achievement. toto macau Your own songs is actually incredible. You've a few really gifted designers. We desire a person the very best associated with achievement. olxtoto alternatif Your own songs is actually incredible. You've a few really gifted designers. We desire a person the very best associated with achievement. situs togel 4d
When your website or blog goes live for the first time, it is exciting. That is until you realize no one but you and your.kantorbola slot
Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include.situs toto 4d
It was wondering if I could use this write-up on my other website, I will link it back to your website though.Great Thanks. olxtoto login
Thanks For sharing this Superb article.I use this Article to show my assignment in college.it is useful For me Great Work.toto macau
Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. olxtoto
Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. jpdewa
It was wondering if I could use this write-up on my other website, I will link it back to your website though.Great Thanks. slot dax69
I haven’t any word to appreciate this post.....Really i am impressed from this post....the person who create this post it was a great human..thanks for shared this with us. olxtoto link alternatif
Fine Posting, We're an important believer around writing commentary for web pages so that you can allow the site freelancers realise that they’ve increased a little something valuable so that you can the ether!tennis scholarships USA