[DL Basic] 딥러닝 기본 및 최적화

gromit·2022년 2월 9일
0
post-thumbnail

1. Deep Learning

딥러닝의 구분

  1. AI (인공지능)
    • Mimic human intelligence, 사람의 지능을 모방
  1. ML (기계학습)
    • Data-driven approach, 데이터를 기반으로 무언가를 (기계)학습
  1. DL (딥러닝)
    • Neural Networks, 그 안에서 뉴럴 네트워크를 사용하여 데이터를 가지고 무언가를 학습하는 세부 분야


딥러닝의 Key Components

  1. data
    • the model can learn from
  2. model
    • how to transform the data
  3. loss function
    • quantifies the bbadness of the model
  4. algorithm
    • to adjust the parameters to minimize the loss



기타

  • loss function을 줄이기만 하는 것이 목적이 아니라, 모델이 학습하지 않았던 테스트 데이터/실제 사용 환경에서 잘 동작하는 것이 목적..! -> 해당 parameter 값(weight, bias)을 찾는 것이 중요할 것
  • 모델의 layer를 깊게 쌓으면 문제가 생기는 이유 또한, train error는 적게 나오지만, test data에서 성능이 잘 안 나오기 때문 (+ 이는 ResNet 등장 후 보완이 됨)



Neural Networks

  • 인간의 뇌 구조를 모방한 구조
  • 뉴럴 네트워크는 함수를 근사하는 모델( Function Approximator )
  • 비선형 변환의 연속

Neural networks are function approximators that stack affine transformations followed by nonlinear transformations.



MLP(다층퍼셉트론) PyTorch로 구현해보기

- Evaluate Function 정의

  • Eval 과정은 with torch.no_grad(): 설정해줘야 함
  • 전체 Dataset에서 정의한 iterator통해, 돌면서 Batch Size만큼 뽑아내며 eval과정 처리
  • .to(device) 이용해 Batch Size만큼의 X 데이터/y 데이터로부터 예측 값/정답 값 정의
  • view() 함수 : 필요 시, reshape
def func_eval(model,data_iter,device):
    with torch.no_grad():
        model.eval() # evaluate (affects DropOut and BN)
        n_total,n_correct = 0,0

        for batch_in,batch_out in data_iter:
            y_trgt = batch_out.to(device)
            model_pred = model(batch_in.view(-1, 28*28).to(device))
            _,y_pred = torch.max(model_pred.data,1)
            n_correct += (y_pred == y_trgt).sum().item()
            n_total += batch_in.size(0)
        val_accr = (n_correct/n_total)
        model.train() # back to train mode 
    return val_accr
print ("Done")

- Train 코드

  • 전체 Dataset에서 정의한 iterator통해, 돌면서 Batch Size만큼 뽑아내며 train과정 처리

  • [1] 포워드 과정

    • (1)위에서 정의한 CustomModelforward(self, x) 함수와 (2)선택한loss 함수(크로스엔트로피 등)에 배치를 돌면서 피드포워드

  • [2] 옵티마이저 및 로스 업데이트 과정 (backpropagation)

    • (1)옵티마이저.zero_grad()로 값 리셋, (2)출력로스.backward()로 미분값 백워드, (3)옵티마이저.step()로 값 업데이트

- code

print ("Start training.")
M.init_param() # initialize parameters
M.train()
EPOCHS,print_every = 10,1

for epoch in range(EPOCHS):
    loss_val_sum = 0

    for batch_in,batch_out in train_iter:
        # Forward path
        y_pred = M.forward(batch_in.view(-1, 28*28).to(device))
        loss_out = loss(y_pred,batch_out.to(device))

        # Update
        optm.zero_grad()      # reset gradient 
        loss_out.backward()      # backpropagate
        optm.step()      # optimizer update
        
        loss_val_sum += loss_out
        
    loss_val_avg = loss_val_sum/len(train_iter)
    # Print
    if ((epoch%print_every)==0) or (epoch==(EPOCHS-1)):
        train_accr = func_eval(M,train_iter,device)
        test_accr = func_eval(M,test_iter,device)
        print ("epoch:[%d] loss:[%.3f] train_accr:[%.3f] test_accr:[%.3f]."%
               (epoch,loss_val_avg,train_accr,test_accr))
print ("Done")        



2. Optimization

  1. Generalization

    • 일반화 성능을 높이자
    • 학습 데이터 - 테스트 데이터 간 성능의 차이(Generalization Gap)가 적을 때를 지향
    • underfitting vs. overfitting 용어와 관련
  2. Cross-validations

    • 학습 데이터 / 테스트 데이터를 k개로 나누어서, (k-1)개로 training, 1개로 validation 일부만 학습에 사용
      • Cross-validations을 이용해 최적의 하이퍼 파라미터 값을 찾고, 이 하이퍼 파라미터 값을 가지고 모델을 학습하는 방법
  3. Bias와 Variance

    • 편향과 분산
    • bias와 variance는 Trade-off 관계를 갖는다.
    • Cost를 최소화하는 문제는, 즉, bias, variance, noise 세 가지를 최소화하는 것
  1. Bootstrapping
    • 학습 데이터가 고정되어 있을 때, 그 안에서 subsampling을 통해서 학습 데이터를 여러 개로 만들고, 이를 통해 만든 여러 개의 모델/metric을 통해, 전체 모델의 일치성/불확실성을 파악하는 접근법


선형회귀문제(음성신호 데이터) PyTorch로 optimizer 비교해보기

  • optimizers 정의
    • torch.optim라이브러리 활용해 선언
    • momentum 옵티마이저는 momentum=(확률값) 옵션 추가

- code

### 임포트 : import torch.optim as optim



LEARNING_RATE = 1e-2
# Instantiate models
model_sgd = Model(name='mlp_sgd',xdim=1,hdims=[64,64],ydim=1).to(device)
model_momentum = Model(name='mlp_momentum',xdim=1,hdims=[64,64],ydim=1).to(device)
model_adam = Model(name='mlp_adam',xdim=1,hdims=[64,64],ydim=1).to(device)

# Optimizers
loss = nn.MSELoss()
optm_sgd = optim.SGD(model_sgd.parameters(), lr=LEARNING_RATE)
optm_momentum = optim.SGD(model_momentum.parameters(), lr=LEARNING_RATE, momentum=0.9)
optm_adam = optim.Adam(model_adam.parameters(), lr=LEARNING_RATE)

print ("Done.")


- 💡 인사이트

  • Adam

    • Adam은 짧은 학습 시간부터 성능(정확도)이 확보됨
    • Adam에는 adaptive learning 개념이 반영된 접근법. (adaptive learning: 어떠한 파라미터에 대해서는 lr을 높이고, 다른 어떠한 파라미터에 대해서는 lr을 줄여나감)
    • 따라서, 똑같은 lr을 선언해주어도 더 빠르게 성능이 확보될 수 있다..! ✨

  • Momentum
    • Momentum은 학습 시간이 조금 길어지면 성능 확보됨. 반면, SGD는 같은 시간까지도 성능이 확보되지 않음
    • Momentum은 이전 배치의 gradient 정보를 활용해 현재 배치 턴에서 사용하겠다는 접근법
    • 이러한 점에서 미니배치일 때, SGD보다 좋다..! ✨

  • SGD
    • SGD는 데이터에서 큰 특징이 되는 파트 위주로 학습이 잘되고, 세부적으로는 잘 놓치는 모양새를 보임
    • 하지만, SGD는 오랜 학습 시간이 흘렀을 때에는 더 좋은 성능을 보일 순 있다.

  • 결론: Adam/rAdam을 초기에 사용하면 어느정도의 성능을 짧은 시간에 효율적으로 확보할 수 있다..! ✨
profile
AI, Big Data, Industrial Engineering

156개의 댓글

comment-user-thumbnail
2025년 5월 20일

i read a lot of stuff and i found that the way of writing to clearifing that exactly want to say was very good so i am impressed and ilike to come again in future..http://wwscc.org/evinfo/pages/1xbet_promo_code_for_registration___sign_up_bonus_india.html

답글 달기
comment-user-thumbnail
2025년 5월 20일

Double your deposit with Linebet’s promo code BNB777! Claim a 100% bonus up to €130. Enjoy free spins, sports bets, and exclusive rewards today.code promo linebet cote d'ivoire

답글 달기
comment-user-thumbnail
2025년 6월 12일

You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. 五反田 英語 アルバイト

답글 달기
comment-user-thumbnail
2025년 6월 17일

POL88 adalah situs slot online gacor terpercaya dengan RTP tinggi dan game dari provider ternama. Daftar sekarang untuk nikmati jackpot besar serta bonus harian yang banyak. slot gacor

답글 달기
comment-user-thumbnail
2025년 6월 22일

This article was written by a real thinking writer. I agree many of the with the solid points made by the writer. I’ll be back. jay rufer

답글 달기
comment-user-thumbnail
2025년 6월 22일

Keep up the good work , I read few posts on this web site and I conceive that your blog is very interesting and has sets of fantastic information. agenolx

답글 달기
comment-user-thumbnail
2025년 6월 22일

Really a great addition. I have read this marvelous post. Thanks for sharing information about it. I really like that. Thanks so lot for your convene. BATMAN138

답글 달기
comment-user-thumbnail
2025년 6월 23일

Thank you for such a well written article. It’s full of insightful information and entertaining descriptions. Your point of view is the best among many. DeepL

답글 달기
comment-user-thumbnail
2025년 6월 24일

This is my first time i visit here and I found so many interesting stuff in your blog especially it's discussion, thank you. sandibet

답글 달기
comment-user-thumbnail
2025년 6월 25일

A very awesome blog post. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. hptoto alternatif

답글 달기
comment-user-thumbnail
2025년 6월 25일

A very awesome blog post. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. pass prop firm

답글 달기
comment-user-thumbnail
2025년 6월 25일

A very awesome blog post. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. toto login

답글 달기
comment-user-thumbnail
2025년 6월 26일

Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. olxtoto

답글 달기
comment-user-thumbnail
2025년 6월 26일

Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. situs togel terpercaya

답글 달기
comment-user-thumbnail
2025년 6월 26일

I’m going to read this. I’ll be sure to come back. thanks for sharing. and also This article gives the light in which we can observe the reality. this is very nice one and gives indepth information. thanks for this nice article... hptoto situs slot

답글 달기
comment-user-thumbnail
2025년 6월 26일

Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. sandibet

답글 달기
comment-user-thumbnail
2025년 6월 28일

You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers.olxtoto

답글 달기
comment-user-thumbnail
2025년 6월 28일

Excellent article. Very interesting to read. I really love to read such a nice article. Thanks! keep rocking. slot qris

답글 달기
comment-user-thumbnail
2025년 6월 28일

It was wondering if I could use this write-up on my other website, I will link it back to your website though.Great Thanks. Bulenox coupon code

답글 달기
comment-user-thumbnail
2025년 6월 28일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.gaspol189

답글 달기
comment-user-thumbnail
2025년 6월 28일

It was wondering if I could use this write-up on my other website, I will link it back to your website though.Great Thanks. bandar toto macau

답글 달기
comment-user-thumbnail
2025년 6월 28일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.M88

답글 달기
comment-user-thumbnail
2025년 6월 28일

I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.situs toto

답글 달기
comment-user-thumbnail
2025년 6월 28일

I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.situs toto

답글 달기
comment-user-thumbnail
2025년 6월 28일

I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.situs toto

답글 달기
comment-user-thumbnail
2025년 6월 28일

I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.situs toto

답글 달기
comment-user-thumbnail
2025년 6월 28일

I think this is an informative post and it is very useful and knowledgeable. therefore, I would like to thank you for the efforts you have made in writing this article.olxtoto

답글 달기
comment-user-thumbnail
2025년 6월 28일

Thanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info. keluaran toto macau

답글 달기
comment-user-thumbnail
2025년 6월 29일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.keytoto

답글 달기
comment-user-thumbnail
2025년 6월 29일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.keytoto

답글 달기
comment-user-thumbnail
2025년 6월 29일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.olxtoto

답글 달기
comment-user-thumbnail
2025년 6월 29일

Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. olxtoto link alternatif

답글 달기
comment-user-thumbnail
2025년 6월 29일

Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. toto slot 777 login

답글 달기
comment-user-thumbnail
2025년 6월 29일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.olxtoto

답글 달기
comment-user-thumbnail
2025년 6월 29일

This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here keep up the good worktoto togel

답글 달기
comment-user-thumbnail
2025년 6월 29일

This is my first time i visit here and I found so many interesting stuff in your blog especially it's discussion, thank you.toto 4d

답글 달기
comment-user-thumbnail
2025년 6월 29일

Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading. togel online

답글 달기
comment-user-thumbnail
2025년 6월 30일

Thanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info.keluaran toto macau

답글 달기
comment-user-thumbnail
2025년 6월 30일

I have read all the comments and suggestions posted by the visitors for this article are very fine,We will wait for your next article so only.Thanks! olxtoto

답글 달기
comment-user-thumbnail
2025년 7월 1일

When your website or blog goes live for the first time, it is exciting. That is until you realize no one but you and your.kantorbola slot

답글 달기
comment-user-thumbnail
2025년 7월 1일

Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include.situs toto 4d

답글 달기
comment-user-thumbnail
2025년 7월 2일

It was wondering if I could use this write-up on my other website, I will link it back to your website though.Great Thanks. olxtoto login

답글 달기
comment-user-thumbnail
2025년 7월 2일

Thanks For sharing this Superb article.I use this Article to show my assignment in college.it is useful For me Great Work.toto macau

답글 달기
comment-user-thumbnail
2025년 7월 2일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. olxtoto

답글 달기
comment-user-thumbnail
2025년 7월 2일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. jpdewa

답글 달기
comment-user-thumbnail
2025년 7월 2일

It was wondering if I could use this write-up on my other website, I will link it back to your website though.Great Thanks. slot dax69

답글 달기
comment-user-thumbnail
2025년 7월 2일

I haven’t any word to appreciate this post.....Really i am impressed from this post....the person who create this post it was a great human..thanks for shared this with us. olxtoto link alternatif

답글 달기
comment-user-thumbnail
2025년 7월 3일

Wow, What a Excellent post. I really found this to much informatics. It is what i was searching for.I would like to suggest you that please keep sharing such type of info.Thankssitus slot

답글 달기
comment-user-thumbnail
2025년 7월 3일

Wow, excellent post. I'd like to draft like this too - taking time and real hard work to make a great article. This post has encouraged me to write some posts that I am going to write soon. SoundCloud MP3

답글 달기
comment-user-thumbnail
2025년 7월 3일

I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post. olxtoto link

답글 달기
comment-user-thumbnail
2025년 7월 3일

I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post. toto macau

답글 달기
comment-user-thumbnail
2025년 7월 3일

Great job for publishing such a beneficial web site. Your web log isn’t only useful but it is additionally really creative too.slot dax69

답글 달기
comment-user-thumbnail
2025년 7월 3일

Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include. bandar togel terpercaya

답글 달기
comment-user-thumbnail
2025년 7월 4일

Thanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info.olxtoto togel login

답글 달기
comment-user-thumbnail
2025년 7월 7일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. mawartoto

답글 달기
comment-user-thumbnail
2025년 7월 7일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. Lotto resmi

답글 달기
comment-user-thumbnail
2025년 7월 8일

I was reading some of your content on this website and I conceive this internet site is really informative ! Keep on putting up. rtp olxtoto

답글 달기
comment-user-thumbnail
2025년 7월 8일

What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much.prediksi togel online

답글 달기
comment-user-thumbnail
2025년 7월 9일

What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much. live toto macau

답글 달기
comment-user-thumbnail
2025년 7월 9일

What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much. olxtoto login slot

답글 달기
comment-user-thumbnail
2025년 7월 9일

This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here keep up the good work olxtoto

답글 달기
comment-user-thumbnail
2025년 7월 9일

This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here keep up the good work olxtoto

답글 달기
comment-user-thumbnail
2025년 7월 10일

I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much有道下载.

답글 달기
comment-user-thumbnail
2025년 7월 10일

I definitely enjoying every little bit of it. It is a great website and nice share. I want to thank you. Good job! You guys do a great blog, and have some great contents. Keep up the good work.link alternatif olxtoto

답글 달기
comment-user-thumbnail
2025년 7월 10일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. สล็อตเครดิตฟรี มาใหม่

답글 달기
comment-user-thumbnail
2025년 7월 10일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you. olxtoto

답글 달기
comment-user-thumbnail
2025년 7월 10일

You there, this is really good post here. Thanks for taking the time to post such valuable information. Quality content is what always gets the visitors coming.

답글 달기
comment-user-thumbnail
2025년 7월 10일

You there, this is really good post here. Thanks for taking the time to post such valuable information. Quality content is what always gets the visitors coming.wps官网

답글 달기
comment-user-thumbnail
2025년 7월 12일

Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include. วิเคราะห์บอล 7m

답글 달기
comment-user-thumbnail
2025년 7월 12일

We have sell some products of different custom boxes.it is very useful and very low price please visits this site thanks and please share this post with your friends. agenolx, slot online

답글 달기
comment-user-thumbnail
2025년 7월 13일

If more people that write articles really concerned themselves with writing great content like you, more readers would be interested in their writings. Thank you for caring about your content. slot If more people that write articles really concerned themselves with writing great content like you, more readers would be interested in their writings. Thank you for caring about your content.

답글 달기
comment-user-thumbnail
2025년 7월 13일

slotDewapoker[https://www.oraka2.com/properties14-6](부산 출장마사지)[https://www.oraka2.com/properties-1-4](대구 출장마사지)[https://www.oraka2.com/properties14-4](광주 출장마사지)https://www.oraka2.com/properties14-5

답글 달기
comment-user-thumbnail
2025년 7월 13일

What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much. 토토커뮤니티

답글 달기
comment-user-thumbnail
2025년 7월 13일

What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much. سایت شرط بندی فارسی

답글 달기
comment-user-thumbnail
2025년 7월 13일

Hey There. I found your blog using msn. This is a very well written article. I’ll be sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely return. olxtoto slot Hey There. I found your blog using msn. This is a very well written article. I’ll be sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely return. olxtoto alternatif Hey There. I found your blog using msn. This is a very well written article. I’ll be sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely return. bebtoto login Hey There. I found your blog using msn. This is a very well written article. I’ll be sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely return. slot online dana Hey There. I found your blog using msn. This is a very well written article. I’ll be sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely return. pengeluaran macau Hey There. I found your blog using msn. This is a very well written article. I’ll be sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely return. toto togel 4d

답글 달기
comment-user-thumbnail
2025년 7월 13일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! social media management

답글 달기
comment-user-thumbnail
2025년 7월 13일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! social media management

답글 달기
comment-user-thumbnail
2025년 7월 13일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! Seafood Buffet Bangkok

답글 달기
comment-user-thumbnail
2025년 7월 13일

What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much. schlüsselfertige Renovierung

답글 달기
comment-user-thumbnail
2025년 7월 14일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! Live casino games India

답글 달기
comment-user-thumbnail
2025년 7월 14일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! betting sites

답글 달기
comment-user-thumbnail
2025년 7월 14일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! GullyBet payment options

답글 달기
comment-user-thumbnail
2025년 7월 14일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! Gullybet

답글 달기
comment-user-thumbnail
2025년 7월 15일

I was reading some of your content on this website and I conceive this internet site is really informative ! Keep on putting up. result macau

답글 달기
comment-user-thumbnail
2025년 7월 16일

It is a great website.. The Design looks very good.. Keep working like that!.custom plush keychain singapore

답글 달기
comment-user-thumbnail
2025년 7월 16일

I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.keluaran sydney lotto

답글 달기
comment-user-thumbnail
2025년 7월 19일

I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here! keep up the good work... slot online

답글 달기
comment-user-thumbnail
2025년 7월 19일

I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here! keep up the good work... toto slot

답글 달기
comment-user-thumbnail
2025년 7월 20일

Excellent post. I was reviewing this blog continuously, and I am impressed! Extremely helpful information especially this page. Thank you and good luck.situs slot gacor

답글 달기
comment-user-thumbnail
2025년 7월 20일

What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much. 토토커뮤니티

답글 달기
comment-user-thumbnail
2025년 7월 20일

Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include.4D

답글 달기
comment-user-thumbnail
2025년 7월 20일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! limo hire

답글 달기
comment-user-thumbnail
2025년 7월 20일

You have a real ability for writing unique content. I like how you think and the way you represent your views in this article. I agree with your way of thinking. Thank you for sharing.macau jitu This is very interesting content! I have thoroughly enjoyed reading your points and have come to the conclusion that you are right about many of them. You are great. casino88 If you set out to make me think today; mission accomplished! I really like your writing style and how you express your ideas. Thank you. situs slot gacor Thank you for such a well written article. It’s full of insightful information and entertaining descriptions. Your point of view is the best among many. olxtoto alternatif Hey There. I found your blog using msn. This is a very well written article. I’ll be sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely return. toto toge Hey There. I found your blog using msn. This is a very well written article. I’ll be sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely return. situs toto 176 login

답글 달기
comment-user-thumbnail
2025년 7월 20일

I was reading some of your content on this website and I conceive this internet site is really informative ! Keep on putting up. data sgp

답글 달기
comment-user-thumbnail
2025년 7월 20일

I was reading some of your content on this website and I conceive this internet site is really informative ! Keep on putting up. flowers rio de janeiro

답글 달기
comment-user-thumbnail
2025년 7월 20일

This particular is usually apparently essential and moreover outstanding truth along with for sure fair-minded and moreover admittedly useful My business is looking to find in advance designed for this specific useful stuffs… flowers rio de janeiro

답글 달기
comment-user-thumbnail
2025년 7월 20일

This particular is usually apparently essential and moreover outstanding truth along with for sure fair-minded and moreover admittedly useful My business is looking to find in advance designed for this specific useful stuffs… data sgp

답글 달기
comment-user-thumbnail
2025년 7월 20일

I was reading some of your content on this website and I conceive this internet site is really informative ! Keep on putting up. 토토커뮤니티

답글 달기
comment-user-thumbnail
2025년 7월 20일

I was reading some of your content on this website and I conceive this internet site is really informative ! Keep on putting up. live draw sgp

답글 달기
comment-user-thumbnail
2025년 7월 22일

This particular papers fabulous, and My spouse and i enjoy each of the perform that you have placed into this. I’m sure that you will be making a really useful place. I has been additionally pleased. Good perform! gọi video riêng tư

답글 달기
comment-user-thumbnail
2025년 7월 22일

I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here! keep up the good work... slot demo

답글 달기
comment-user-thumbnail
2025년 7월 22일

Excellent .. Amazing .. I’ll bookmark your blog and take the feeds also…I’m happy to find so many useful info here in the post, we need work out more techniques in this regard, thanks for sharing.agen togel

답글 달기
comment-user-thumbnail
2025년 7월 23일

We are really grateful for your blog post. You will find a lot of approaches after visiting your post. I was exactly searching for. Thanks for such post and please keep it up. Great work.miototo togel

답글 달기
comment-user-thumbnail
2025년 7월 23일

Nice post! This is a very nice blog that I will definitively come back to more times this year! Thanks for informative post. 다낭 골프

답글 달기
comment-user-thumbnail
2025년 7월 24일

I am happy to find this post very useful for me, as it contains lot of information. I always prefer to read the quality content and this thing I found in you post. Thanks for sharing.keytoto login

답글 달기
comment-user-thumbnail
2025년 7월 24일

I have seen some great stuff here. Worth bookmarking for revisiting. I surprise how much effort you put to create such a great informative website. Your work is truly appreciated around the clock and the globe.kiko toto

답글 달기
comment-user-thumbnail
2025년 7월 27일

This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free. 먹튀보증 ~~

답글 달기
comment-user-thumbnail
2025년 7월 27일

This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free. situs toto 176

답글 달기
comment-user-thumbnail
2025년 7월 27일

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained!베트남유흥

답글 달기
comment-user-thumbnail
2025년 7월 27일

I have read your article, it is very informative and helpful for me.I admire the valuable information you offer in your articles. Thanks for posting it.. 오피스타

답글 달기
comment-user-thumbnail
2025년 7월 28일

It is a good site post without fail. Not too many people would actually, the way you just did. I am impressed that there is so much information about this subject that has been uncovered and you’ve defeated yourself this time, with so much quality. Good Works!situs toto login link alternatif

답글 달기
comment-user-thumbnail
2025년 7월 29일

Excellent blog! I found it while surfing around on Google. Content of this page is unique as well as well researched. Appreciate it.harga toto

답글 달기
comment-user-thumbnail
2025년 7월 30일

Thanks so much for this information. I have to let you know I concur on several of the points you make here and others may require some further review, but I can see your viewpoint.
olxtoto
신용카드 현금화
hp toto
هات بت انفجار
toto togel
toto 4d

답글 달기
comment-user-thumbnail
2025년 7월 31일

All your hard work is much appreciated. Nobody can stop to admire you. Lots of appreciation.harga toto

답글 달기
comment-user-thumbnail
2025년 8월 2일

thanks for the tips and information..i really appreciate it..situs mawartoto

답글 달기
comment-user-thumbnail
2025년 8월 2일

I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often. mawartoto resmi

답글 달기
comment-user-thumbnail
2025년 8월 3일

Your work is truly appreciated round the clock and the globe. It is incredibly a comprehensive and helpful blog.seo optimization services

답글 달기
comment-user-thumbnail
2025년 8월 4일

This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free. 역삼오피

답글 달기
comment-user-thumbnail
2025년 8월 4일

This article was written by a real thinking writer. I agree many of the with the solid points made by the writer. I’ll be back. data toto macau

답글 달기
comment-user-thumbnail
2025년 8월 4일

I went to this website, and I believe that you have a plenty of excellent information, I have saved your site to my bookmarks. 오피스타 텔레그램

답글 달기
comment-user-thumbnail
2025년 8월 4일

i read a lot of stuff and i found that the way of writing to clearifing that exactly want to say was very goo
bandar slot online
bocoran macau
4D
keytoto daftar
toto slot
olxtoto 4d

답글 달기
comment-user-thumbnail
2025년 8월 4일

Thanks for the blog loaded with so many information. Stopping by your blog helped me to get what I was looking for. link situs toto

답글 달기
comment-user-thumbnail
2025년 8월 4일

Very useful info. Hope to see more posts soon!.mahkota338

답글 달기
comment-user-thumbnail
2025년 8월 4일

Thank you because you have been willing to share information with us. we will always appreciate all you have done here because I know you are very concerned with our. explicit definition

답글 달기
comment-user-thumbnail
2025년 8월 4일

Thank you because you have been willing to share information with us. we will always appreciate all you have done here because I know you are very concerned with our. 토토프레이

답글 달기
comment-user-thumbnail
2025년 8월 5일

I have bookmarked your blog, the articles are way better than other similar blogs.. thanks for a great blog! olxtoto

답글 달기
comment-user-thumbnail
2025년 8월 5일

You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. data macau I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much. koi toto Thanks so much for this information. I have to let you know I concur on several of the points you make here and others may require some further review, but I can see your viewpoint. pol88 Very good written article. It will be supportive to anyone who utilizes it, including me. Keep doing what you are doing – can’r wait to read more posts. olxtoto Awesome article! I want people to know just how good this information is in your article. It’s interesting, compelling content. Your views are much like my own concerning this subject. togel online Awesome article! I want people to know just how good this information is in your article. It’s interesting, compelling content. Your views are much like my own concerning this subject. situs toto 176

답글 달기
comment-user-thumbnail
2025년 8월 6일

I admit, I have not been on this web page in a long time... however it was another joy to see It is such an important topic and ignored by so many, even professionals. I thank you to help making people more aware of possible issues.paito sydney lotto

답글 달기
comment-user-thumbnail
2025년 8월 6일

This blog is so nice to me. I will keep on coming here again and again. Visit my link as well..hptoto

답글 달기
comment-user-thumbnail
2025년 8월 8일

This site that has articles as good as this is really worth looking at continuously.
Visit : https://mbcslot88.me/

답글 달기
comment-user-thumbnail
2025년 8월 8일

This site that has articles as good as this is really worth looking at continuously.
Visit : https://mbcslot88.me/

답글 달기
comment-user-thumbnail
2025년 8월 8일

This site that has articles as good as this is really worth looking at continuously.
Visit : https://mbcslot88.me/

답글 달기
comment-user-thumbnail
2025년 8월 9일

Very informative post! There is a lot of information here that can help any business get started with a successful social networking campaign. 오피스타

답글 달기
comment-user-thumbnail
2025년 8월 9일

Very informative post! There is a lot of information here that can help any business get started with a successful social networking campaign. Apex Trader Funding rules

답글 달기
comment-user-thumbnail
2025년 8월 9일

Very informative post! There is a lot of information here that can help any business get started with a successful social networking campaign. kingkong4d

답글 달기
comment-user-thumbnail
2025년 8월 10일

Thank you for the update, very nice site..situs slot online

답글 달기
comment-user-thumbnail
2025년 8월 12일

Hey what a brilliant post I have come across and believe me I have been searching out for this similar kind of post for past a week and hardly came across this. Thank you very much and will look for more postings from you.slot online

답글 달기
comment-user-thumbnail
2025년 8월 12일

Positive site, where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work.pestoto

답글 달기
comment-user-thumbnail
2025년 8월 12일

Your blog provided us with valuable information to work with. Each & every tips of your post are awesome. Thanks a lot for sharing. Keep blogging..pestoto

답글 달기
comment-user-thumbnail
2025년 8월 12일

I admire this article for the well-researched content and excellent wording. I got so involved in this material that I couldn’t stop reading. I am impressed with your work and skill. Thank you so much.slot gacor

답글 달기
comment-user-thumbnail
2025년 8월 12일

Cool stuff you have got and you keep update all of us.rocket languages

답글 달기
comment-user-thumbnail
2025년 8월 12일

I wanted to thank you for this excellent read!! I definitely loved every little bit of it. I have you bookmarked your site to check out the new stuff you post.togel online

답글 달기
comment-user-thumbnail
2025년 8월 16일

I would like to say that this blog really convinced me to do it! Thanks, very good post.pestoto togel

답글 달기
comment-user-thumbnail
2025년 8월 18일

Wow, What a Excellent post. I really found this to much informatics. It is what i was searching for.I would like to suggest you that please keep sharing such type of info.Thankspestoto link

답글 달기
comment-user-thumbnail
2025년 8월 21일

Just admiring your work and wondering how you managed this blog so well. It’s so remarkable that I can't afford to not go through this valuable information whenever I surf the internet!edctoto login

답글 달기
comment-user-thumbnail
2025년 8월 23일

You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers.سایت melbet

답글 달기
comment-user-thumbnail
2025년 8월 23일

Thanks for the blog loaded with so many information. Stopping by your blog helped me to get what I was looking for.سایت رومابت

답글 달기
comment-user-thumbnail
2025년 8월 23일

I read that Post and got it fine and informative.Kudu

답글 달기

I recently found many useful information in your website especially this blog page. Among the lots of comments on your articles. Thanks for sharing.slot 4d

답글 달기

Positive site, where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work.오피스타

답글 달기
comment-user-thumbnail
5일 전

I am thankful to you for sharing this plethora of useful information. I found this resource utmost beneficial for me. Thanks a lot for hard work. Situs Toto You delivered such an impressive piece to read, giving every subject enlightenment for us to gain information. Thanks for sharing such information with us due to which my several concepts have been cleared. situs toto Wonderful article. Fascinating to read. I love to read such an excellent article. Thanks! It has made my task more and extra easy. Keep rocking. slot toto Thanks for an interesting blog. What else may I get that sort of info written in such a perfect approach? I have an undertaking that I am just now operating on, and I have been on the lookout for such info. Toto xl

답글 달기

This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free.alexistogel

답글 달기
comment-user-thumbnail
1일 전

I have seen some great stuff here. Worth bookmarking for revisiting. I surprise how much effort you put to create such a great informative website. Your work is truly appreciated around the clock and the globe. basket 168

답글 달기

I just couldn't leave your website before telling you that I truly enjoyed the top quality info you present to your visitors? Will be back again frequently to check up on new posts.부산 출장마사지

답글 달기

I just couldn't leave your website before telling you that I truly enjoyed the top quality info you present to your visitors? Will be back again frequently to check up on new posts.giftcardmall

답글 달기
comment-user-thumbnail
약 15시간 전

[Cool stuff you have got and you keep update all of us.](slot toto gacor)[https://planeta-animal.org/](rtp olxtoto)[https://goodthingsbeauty.com/](olxtoto slot)[https://createwowmedia.com/](link alternatif situs toto)[https://bokehmasterskit.com/](bandar toto macau)[https://www.alessandrocolla.com/first-event-storming-experiment/](daftar togel)[https://dawgyawp.com/]

답글 달기