RA-KT: Cognition and Metacognition Tracing

---

RA-KT (Reflection Assistant-Knowledge Tracing) is a proof of concept tool that has two primary objectives:

1. Train Learners to develop metacognition while they try to solve cognitive (assignments, quizzes etc)   loads.
2. Give personalized performance metrics to the educators/education providers, to better understand      learner’s metacognitive and cognitive performances. 

Metacognition is an individual’s ability to reflect on their own knowledge (cognition). Training learners for metacognition in a cognitive setting using traditional metacognition training methods, is particularly difficult, as it may introduce added learning load to the learner. For example, training for being more aware of one's own knowledge while trying to learn skills from curriculum could be burdensome. But the RA component of RA-KT (Reflection Assistant (Gamma 2004)) simplifies this issue.

RA, also known as Reflection Assistant is a tool developed by Gamma which prompts students to solve questions in separate phase, and in-turn monitors their awareness and outlook for each question solved by student.  The different phases of RA include: planning phase (student writes down their plan for the solution), confidence rating phase (student rates their confidence for solving the question), problem solving phase (student solves the problem), The RA way of solving problems was incorporated in this project, excluding the planning phase. 

KT, also known as Knowledge Tracing is a way to measure one's skill mastery for a given topic. The KT component of this project tracks the student's skill mastery probability using Bayesian Knowledge Tracing (BKT) method.

•  Student rates their confidence before trying to solve assignment in RA-KT.
• Student then enters the problem solving phase and provides solution.
• After assignment is submitted, a performance evaluation is done per question.
• Evaluation includes:Awareness, Outlook and skill mastery percent per question

• The pyBKT library was used to train a BKT model on historical student data.
• Student grades can be provided in fractions (for example: 5/10).
• For BKT usage, the grades were classified into correct (grade>=70%) and incorrect (grade<70%) classes. These thresholds can be adjusted as per the institution requirements.
• The grades provided are converted into binary grades (0, 1) by setting institution specific grade thresholds.
• Original BKT model was used

• For KMA and KMB, the grades were classified into partially correct (grade>=55% and grade<70%), correct (grade>=70% ) and incorrect (grade<50%).
• The confidence rating and grade from each question is used to calculate KMA and KMB for each solution.