모델평가

모델평가의 개념

0.3보다 크거나 같으면 1

F1 score

=조화 평균

ROC와 AUC

AUC=면적

ROC

ROC 커브 그려보기

x = wine.drop(['taste', 'quality'], axis=1)
y= wine['taste']

x_train, x_test, y_train, y_test = train_test_split(x,y, test_size = 0.2, random_state = 13)

wine_tree = DecisionTreeClassifier(max_depth=2, random_state=13)
wine_tree.fit(x_train, y_train)

y_pred_tr = wine_tree.predict(x_train)
y_pred_test = wine_tree.predict(x_test)

print('taste train : ', accuracy_score(y_train, y_pred_tr))
print('taste test : ', accuracy_score(y_test, y_pred_test))

taste train : 0.7294593034442948
taste test : 0.7161538461538461

각 수치 구해보기

from sklearn.metrics import (accuracy_score, precision_score, recall_score, f1_score, roc_auc_score, roc_curve)

print('accuracy :', accuracy_score(y_test, y_pred_test))
print('recall :', recall_score(y_test, y_pred_test))
print('precision :', precision_score(y_test, y_pred_test))
print('AUC score :', roc_auc_score(y_test, y_pred_test))
print('F1 score :', f1_score(y_test, y_pred_test))

accuracy : 0.7161538461538461
recall : 0.7314702308626975
precision : 0.8026666666666666
AUC score : 0.7105988470875331
F1 score : 0.7654164017800381

%matplotlib inline

pred_proba = wine_tree.predict_proba(x_test)[:,1]
#proba 결과를 보면 [0일확률, 1일확률]로 나오는데, 거기서 1일확률만 보여주기
fpr,tpr,thresholds = roc_curve(y_test,pred_proba)

plt.figure(figsize=(10,8))
plt.plot([0,1],[0,1])
plt.plot(fpr, tpr)
plt.grid()
plt.show()

AUC