-
Notifications
You must be signed in to change notification settings - Fork 231
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HW 5 #97
base: master
Are you sure you want to change the base?
HW 5 #97
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
--- | ||
title: "Assignment 5 - Decision Trees" | ||
author: "Allison Teevan" | ||
date: "November 13, 2019" | ||
output: html_document | ||
--- | ||
For this assignment we will be using data from the Assistments Intelligent Tutoring system. This system gives students hints based on how they perform on math problems. | ||
|
||
#Install & call libraries | ||
```{r} | ||
#install.packages("party") | ||
#install.packages("rpart") | ||
library(rpart) | ||
library(party) | ||
``` | ||
|
||
## Part I | ||
#read in data, "intelligent_tutor.csv" | ||
```{r} | ||
D1 <- read.table("intelligent_tutor.csv", sep = ",", header = TRUE) | ||
``` | ||
|
||
##Classification Tree | ||
First we will build a classification tree to predict which students ask a teacher for help, which start a new session, or which give up, based on whether or not the student completed a session (D1$complete) and whether or not they asked for hints (D1$hint.y). | ||
```{r} | ||
|
||
c.tree <- rpart(action ~ hint.y + complete, method="class", data=D1) #Notice the standard R notion for a formula X ~ Y | ||
|
||
#Look at the error of this tree | ||
printcp(c.tree) | ||
|
||
#Plot the tree | ||
post(c.tree, file = "tree.ps", title = "Session Completion Action: 1 - Ask teacher, 2 - Start new session, 3 - Give up") | ||
|
||
``` | ||
## Part II | ||
|
||
#Regression Tree | ||
We want to see if we can build a decision tree to help teachers decide which students to follow up with, based on students' performance in Assistments. We will create three groups ("teacher should intervene", "teacher should monitor student progress" and "no action") based on students' previous use of the system and how many hints they use. To do this we will be building a decision tree using the "party" package. The party package builds decision trees based on a set of statistical stopping rules. | ||
|
||
#Visualize our outcome variable "score" | ||
```{r} | ||
hist(D1$score) | ||
|
||
``` | ||
|
||
#Create a categorical outcome variable based on student score to advise the teacher using an "ifelse" statement | ||
```{r} | ||
D1$advice <- ifelse(D1$score <=0.4, "intervene", ifelse(D1$score > 0.4 & D1$score <=0.8, "monitor", "no action")) | ||
``` | ||
|
||
#Build a decision tree that predicts "advice" based on how many problems students have answered before, the percentage of those problems they got correct and how many hints they required | ||
```{r} | ||
tree2 <- ctree(factor(advice) ~ prior_prob_count + prior_percent_correct + hints, D1) | ||
``` | ||
|
||
#Plot tree | ||
```{r} | ||
plot(tree2) | ||
``` | ||
|
||
Please interpret the tree, which two behaviors do you think the teacher should most closely pay attention to? | ||
#number of hints they take and prior percent correct because they had high levels of "intervene" | ||
|
||
#Test Tree | ||
Upload the data "intelligent_tutor_new.csv". This is a data set of a different sample of students doing the same problems in the same system. We can use the tree we built for the previous data set to try to predict the "advice" we should give the teacher about these new students. | ||
|
||
```{r} | ||
#Upload new data | ||
|
||
D2 <- read.csv("intelligent_tutor_new.csv", header = TRUE) | ||
|
||
#Generate predicted advice using the predict() command for new students based on tree generated from old students | ||
D2$prediction <- predict(tree2, D2) | ||
|
||
``` | ||
## Part III | ||
Compare the predicted advice with the actual advice that these students recieved. What is the difference between the observed and predicted results? | ||
```{r} | ||
#everyone got a score of 1, so they should be predicted to get "no action." | ||
|
||
D2$correct_predict <- ifelse(D2$prediction == "no action", 1, 0) | ||
|
||
mean(D2$correct_predict) | ||
Comment on lines
+82
to
+84
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can also create a table or a confusion matrix. |
||
|
||
#58% accuracy. | ||
#42% are incorrectly predicted. The current model is not great, it definitely could be better. | ||
|
||
#The current model predicted 58% "no action", but the actual value is 100% no action. The difference between observed and predicted results is 42%. | ||
|
||
``` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Overall, great job but try to specify whether the model overfits or whether it is generalizable. |
||
|
||
### To Submit Your Assignment | ||
|
||
Please submit your assignment by first "knitting" your RMarkdown document into an html file and then commit, push and pull request both the RMarkdown file and the html file. | ||
|
Large diffs are not rendered by default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Elaborate on it.