Question

Q8.   Below is output comparing the model predictions to the actual outcomes. How did the model perform? Calculate the percent correctly predicted, the true positive rate and the false positive rate. Also below is a histogram of the predicted probabilities of our observations

> predicted.prob.logit <- predict(logit.mod, dat, type="response")

> predicted.success.logit <- ifelse(predicted.prob.logit >.5, 1, 0)

> table(predicted.success.logit, dat$switch, dnn=c("PREDICTED", "ACTUAL"))

         ACTUAL

PREDICTED    0    1

        0  515  364

        1  768 1373

Q9. Describe how the ROC curve is created, in reference to Q8.   The Area Under the Curve (AUC) for the logit model as 0.596. What does the AUC indicate?

Q10. Suppose the researcher believes that the importance of distance as a predictor of likelihood of switching wells increases for households with higher existing arsenic levels. How can you update your model to test this?

Answer 1

Q8)

True positive rate = Observations predicted 1 correctly/total actual number of observations with 1

= TP/(TP+FN)

= 1373/(1373+364)

=0.7904433

Same way True negative rate = TN/(TN+FP)

=515/(515+768)

=0.401403

%CORRECT PREDICTED = (1373+515)/(1373+515+768+364)

=0.6251656

AUC - ROC curve is a performance measurement for classification problem at various thresholds settings. ROC is a probability curve and AUC represents degree or measure of separability. It tells how much model is capable of distinguishing between classes. Higher the AUC, better the model is at predicting 0s as 0s and 1s as 1s. By analogy, Higher the AUC, better the model is at distinguishing between patients with disease and no disease.

Hope the above answer has helped you in understanding the problem. Please upvote the ans if it has really helped you. Good Luck!!