diff --git a/Assignment 5.Rmd b/Assignment 5.Rmd index 8838dc9..0354900 100644 --- a/Assignment 5.Rmd +++ b/Assignment 5.Rmd @@ -8,15 +8,16 @@ For this assignment we will be using data from the Assistments Intelligent Tutor #Install & call libraries ```{r} -install.packages("party", "rpart") +#install.packages("party", "rpart", "rpart.plot") library(rpart) +library(rpart.plot) library(party) ``` ## Part I ```{r} -D1 <- +D1 <- read.csv("intelligent_tutor.csv") ``` ##Classification Tree @@ -31,6 +32,11 @@ printcp(c.tree) #Plot the tree post(c.tree, file = "tree.ps", title = "Session Completion Action: 1 - Ask teacher, 2 - Start new session, 3 - Give up") +plot(c.tree, compress = TRUE) +text(c.tree, use.n = TRUE) + +rpart.plot(c.tree) + ``` ## Part II @@ -40,25 +46,33 @@ We want to see if we can build a decision tree to help teachers decide which stu #Visualize our outcome variable "score" ```{r} - +hist(D1$score) ``` #Create a categorical outcome variable based on student score to advise the teacher using an "ifelse" statement ```{r} -D1$advice <- +D1$advice <- ifelse(D1$score < 0.3, "Intervene", ifelse(D1$score < 0.6, "Monitor Progress", "No Action")) ``` #Build a decision tree that predicts "advice" based on how many problems students have answered before, the percentage of those problems they got correct and how many hints they required ```{r} -score_ctree <- +score_ctree <- rpart(advice ~ prior_prob_count + prior_percent_correct + hints, method="class", data=D1) ``` #Plot tree ```{r} +post(score_ctree, file = "score_tree.ps", title = "Session Completion Advice: 1 - Intervene, 2 - Monitor Progress, 3 - No Action") + +plot(score_ctree, compress = TRUE) +text(score_ctree, use.n = TRUE) +rpart.plot(score_ctree) ``` Please interpret the tree, which two behaviors do you think the teacher should most closely pay attemtion to? +1. hints >= 58 +2. prior_percent_corect < 0.6 + #Test Tree Upload the data "intelligent_tutor_new.csv". This is a data set of a differnt sample of students doing the same problems in the same system. We can use the tree we built for the previous data set to try to predict the "advice" we should give the teacher about these new students. @@ -66,16 +80,35 @@ Upload the data "intelligent_tutor_new.csv". This is a data set of a differnt sa ```{r} #Upload new data -D2 <- +D2 <- read.csv("intelligent_tutor_new.csv") #Generate predicted advice using the predict() command for new students based on tree generated from old students -D2$prediction <- - +D2$prediction <- predict(score_ctree,D2, "class") ``` ## Part III Compare the predicted advice with the actual advice that these students recieved. What is the difference between the observed and predicted results? +The following results show that the predicted "no action" is less than the one observed, the "monitor progress" number is more than the observed number, and there is no student need to be intervened. I feel that the predicted one is more centralized since the middle choice - "monitor progress" is more than the observed one. +```{r} + +# Observed +# No Action: 34% + 39% = 73% +# Monitor Progress: 6% + 19% = 25% +# Intervene: 2% + +# Predicted + +# No Action +sum(D2$prediction == "No Action") / nrow(D2) # 64.5% +# Monitor Progress +sum(D2$prediction == "Monitor Progress") / nrow(D2) # 34.5% +# Intervene +sum(D2$prediction == "Intervene") / nrow(D2) # 0.01% + +``` + + ### To Submit Your Assignment Please submit your assignment by first "knitting" your RMarkdown document into an html file and then commit, push and pull request both the RMarkdown file and the html file. diff --git a/Assignment-5.html b/Assignment-5.html new file mode 100644 index 0000000..2a06b0d --- /dev/null +++ b/Assignment-5.html @@ -0,0 +1,522 @@ + + + + + + + + + + + + + + + + +Assignment 5 - Decision Trees + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + +

For this assignment we will be using data from the Assistments Intelligent Tutoring system. This system gives students hints based on how they perform on math problems.

+

#Install & call libraries

+
#install.packages("party", "rpart", "rpart.plot")
+
+library(rpart)
+library(rpart.plot)
+library(party)
+
## Loading required package: grid
+
## Loading required package: mvtnorm
+
## Loading required package: modeltools
+
## Loading required package: stats4
+
## Loading required package: strucchange
+
## Loading required package: zoo
+
## 
+## Attaching package: 'zoo'
+
## The following objects are masked from 'package:base':
+## 
+##     as.Date, as.Date.numeric
+
## Loading required package: sandwich
+
+

Part I

+
D1 <- read.csv("intelligent_tutor.csv")
+

##Classification Tree First we will build a classification tree to predict which students ask a teacher for help, which start a new session, or which give up, based on whether or not the student completed a session (D1\(complete) and whether or not they asked for hints (D1\)hint.y).

+
c.tree <- rpart(action ~ hint.y + complete, method="class", data=D1) #Notice the standard R notion for a formula X ~ Y
+
+#Look at the error of this tree
+printcp(c.tree)
+
## 
+## Classification tree:
+## rpart(formula = action ~ hint.y + complete, data = D1, method = "class")
+## 
+## Variables actually used in tree construction:
+## [1] complete hint.y  
+## 
+## Root node error: 250/378 = 0.66138
+## 
+## n= 378 
+## 
+##      CP nsplit rel error xerror     xstd
+## 1 0.052      0     1.000  1.088 0.034934
+## 2 0.012      1     0.948  1.048 0.035867
+## 3 0.010      2     0.936  0.996 0.036873
+
#Plot the tree
+post(c.tree, file = "tree.ps", title = "Session Completion Action: 1 - Ask teacher, 2 - Start new session, 3 - Give up")
+
+plot(c.tree, compress = TRUE)
+text(c.tree, use.n = TRUE)
+

+
rpart.plot(c.tree)
+

## Part II

+

#Regression Tree

+

We want to see if we can build a decision tree to help teachers decide which students to follow up with, based on students’ performance in Assistments. We will create three groups (“teacher should intervene”, “teacher should monitor student progress” and “no action”) based on students’ previous use of the system and how many hints they use. To do this we will be building a decision tree using the “party” package. The party package builds decision trees based on a set of statistical stopping rules.

+

#Visualize our outcome variable “score”

+
hist(D1$score)
+

+

#Create a categorical outcome variable based on student score to advise the teacher using an “ifelse” statement

+
D1$advice <- ifelse(D1$score < 0.3, "Intervene", ifelse(D1$score < 0.6, "Monitor Progress", "No Action"))
+

#Build a decision tree that predicts “advice” based on how many problems students have answered before, the percentage of those problems they got correct and how many hints they required

+
score_ctree <- rpart(advice ~ prior_prob_count + prior_percent_correct + hints, method="class", data=D1)
+

#Plot tree

+
post(score_ctree, file = "score_tree.ps", title = "Session Completion Advice: 1 - Intervene, 2 - Monitor Progress, 3 - No Action")
+
+plot(score_ctree, compress = TRUE)
+text(score_ctree, use.n = TRUE)
+

+
rpart.plot(score_ctree)
+

+

Please interpret the tree, which two behaviors do you think the teacher should most closely pay attemtion to? 1. hints >= 58 2. prior_percent_corect < 0.6

+

#Test Tree Upload the data “intelligent_tutor_new.csv”. This is a data set of a differnt sample of students doing the same problems in the same system. We can use the tree we built for the previous data set to try to predict the “advice” we should give the teacher about these new students.

+
#Upload new data
+
+D2 <- read.csv("intelligent_tutor_new.csv")
+
+#Generate predicted advice using the predict() command for new students based on tree generated from old students
+
+D2$prediction <- predict(score_ctree,D2, "class")
+
+
+

Part III

+

Compare the predicted advice with the actual advice that these students recieved. What is the difference between the observed and predicted results?

+

The following results show that the predicted “no action” is less than the one observed, the “monitor progress” number is more than the observed number, and there is no student need to be intervened. I feel that the predicted one is more centralized since the middle choice - “monitor progress” is more than the observed one.

+
# Observed
+# No Action: 34% + 39% = 73%
+# Monitor Progress: 6% + 19% = 25%
+# Intervene: 2%
+
+# Predicted
+
+# No Action
+sum(D2$prediction == "No Action") / nrow(D2) # 64.5%
+
## [1] 0.645
+
# Monitor Progress
+sum(D2$prediction == "Monitor Progress") / nrow(D2) # 34.5%
+
## [1] 0.345
+
# Intervene
+sum(D2$prediction == "Intervene") / nrow(D2) # 0.01%
+
## [1] 0.01
+
+

To Submit Your Assignment

+

Please submit your assignment by first “knitting” your RMarkdown document into an html file and then commit, push and pull request both the RMarkdown file and the html file.

+
+
+ + + + +
+ + + + + + + + + + + + + + + diff --git a/assignment5.Rproj b/assignment5.Rproj new file mode 100644 index 0000000..8e3c2eb --- /dev/null +++ b/assignment5.Rproj @@ -0,0 +1,13 @@ +Version: 1.0 + +RestoreWorkspace: Default +SaveWorkspace: Default +AlwaysSaveHistory: Default + +EnableCodeIndexing: Yes +UseSpacesForTab: Yes +NumSpacesForTab: 2 +Encoding: UTF-8 + +RnwWeave: Sweave +LaTeX: pdfLaTeX diff --git a/score_tree.ps b/score_tree.ps new file mode 100644 index 0000000..9450d5d Binary files /dev/null and b/score_tree.ps differ diff --git a/tree.ps b/tree.ps new file mode 100644 index 0000000..2fd2b6f Binary files /dev/null and b/tree.ps differ