Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HW 5 #97

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

HW 5 #97

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions Assignment 5 AT.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
title: "Assignment 5 - Decision Trees"
author: "Allison Teevan"
date: "November 13, 2019"
output: html_document
---
For this assignment we will be using data from the Assistments Intelligent Tutoring system. This system gives students hints based on how they perform on math problems.

#Install & call libraries
```{r}
#install.packages("party")
#install.packages("rpart")
library(rpart)
library(party)
```

## Part I
#read in data, "intelligent_tutor.csv"
```{r}
D1 <- read.table("intelligent_tutor.csv", sep = ",", header = TRUE)
```

##Classification Tree
First we will build a classification tree to predict which students ask a teacher for help, which start a new session, or which give up, based on whether or not the student completed a session (D1$complete) and whether or not they asked for hints (D1$hint.y).
```{r}

c.tree <- rpart(action ~ hint.y + complete, method="class", data=D1) #Notice the standard R notion for a formula X ~ Y

#Look at the error of this tree
printcp(c.tree)

#Plot the tree
post(c.tree, file = "tree.ps", title = "Session Completion Action: 1 - Ask teacher, 2 - Start new session, 3 - Give up")

```
## Part II

#Regression Tree
We want to see if we can build a decision tree to help teachers decide which students to follow up with, based on students' performance in Assistments. We will create three groups ("teacher should intervene", "teacher should monitor student progress" and "no action") based on students' previous use of the system and how many hints they use. To do this we will be building a decision tree using the "party" package. The party package builds decision trees based on a set of statistical stopping rules.

#Visualize our outcome variable "score"
```{r}
hist(D1$score)

```

#Create a categorical outcome variable based on student score to advise the teacher using an "ifelse" statement
```{r}
D1$advice <- ifelse(D1$score <=0.4, "intervene", ifelse(D1$score > 0.4 & D1$score <=0.8, "monitor", "no action"))
```

#Build a decision tree that predicts "advice" based on how many problems students have answered before, the percentage of those problems they got correct and how many hints they required
```{r}
tree2 <- ctree(factor(advice) ~ prior_prob_count + prior_percent_correct + hints, D1)
```

#Plot tree
```{r}
plot(tree2)
```

Please interpret the tree, which two behaviors do you think the teacher should most closely pay attention to?
#number of hints they take and prior percent correct because they had high levels of "intervene"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elaborate on it.


#Test Tree
Upload the data "intelligent_tutor_new.csv". This is a data set of a different sample of students doing the same problems in the same system. We can use the tree we built for the previous data set to try to predict the "advice" we should give the teacher about these new students.

```{r}
#Upload new data

D2 <- read.csv("intelligent_tutor_new.csv", header = TRUE)

#Generate predicted advice using the predict() command for new students based on tree generated from old students
D2$prediction <- predict(tree2, D2)

```
## Part III
Compare the predicted advice with the actual advice that these students recieved. What is the difference between the observed and predicted results?
```{r}
#everyone got a score of 1, so they should be predicted to get "no action."

D2$correct_predict <- ifelse(D2$prediction == "no action", 1, 0)

mean(D2$correct_predict)
Comment on lines +82 to +84

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can also create a table or a confusion matrix.


#58% accuracy.
#42% are incorrectly predicted. The current model is not great, it definitely could be better.

#The current model predicted 58% "no action", but the actual value is 100% no action. The difference between observed and predicted results is 42%.

```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, great job but try to specify whether the model overfits or whether it is generalizable.


### To Submit Your Assignment

Please submit your assignment by first "knitting" your RMarkdown document into an html file and then commit, push and pull request both the RMarkdown file and the html file.

341 changes: 341 additions & 0 deletions Assignment-5-AT.html

Large diffs are not rendered by default.