Tracing mastery over time without guessing what your data can’t support.

Knowledge tracing models a learner’s evolving state of mastery from a sequence of task attempts. In the canonical setup, each row records who attempted what, when, and whether it was correct. The model uses the history up to time t to estimate latent mastery and predict performance at time t + 1.

That sounds abstract, but the practical questions are concrete:

• Is this learner now likely to have mastered concept C01?
• Which skills are still unstable after repeated practice?
• Did groups differ in their estimated mastery trajectories over time?
• Which next item should the system recommend?
• Which students need support before the unit quiz rather than after it?

The field did not stop at one model. It evolved in layers.

Family	What it assumes	Why people still use it	Main cost
BKT	Binary latent mastery state with learn / guess / slip / forget parameters.	Interpretability, pedagogical plausibility, clean per-skill parameters, easy communication.	Rigid assumptions; limited feature richness.
AFM / PFA / LKT	Logistic response model with practice features and sometimes richer covariates.	Strong baseline, easy covariate extension, transparent coefficients.	Less “stateful” than classic latent-state framing.
DKT	RNN learns mastery dynamics directly from sequences.	Flexible sequential representation; often stronger raw prediction.	Lower interpretability, more preprocessing, more tuning.
Memory / attention KT	Sequence structure, recency, and item relationships need more expressive architectures.	Current benchmark culture in EDM/AIED/LAK.	Harder to explain to education audiences.

Your practical situation	Best first choice	Why
You need interpretable mastery per skill for teachers or reviewers.	BKT	Parameters map to learn / slip / guess / forget language people understand.
You want coefficients for opportunities, time, hints, or durations.	LKT / AFM / PFA	Feature-based logistic framing is easier to extend and report.
You have very long logs, many items, and prediction performance is the main goal.	DKT or pyKT models	Deep sequence models can capture richer dependencies.
You need a transparent baseline before trying a transformer-style KT model.	BKT + logistic KT	Strong baseline discipline prevents “black-box first” analysis.
You only have a pretest and posttest.	Not KT	You do not have enough sequential evidence.

The biggest KT failure mode is not the model. It is the dataframe. If the event order, skill mapping, or correctness coding is wrong, everything downstream is wrong.

Universal event-log schema

Column	Type	Why it matters
user_id	string	Required to separate each learner’s sequence.
order_id	integer	Required to guarantee within-student temporal order.
timestamp	datetime	Useful for checking or reconstructing order; important for spacing / delay features.
question_id	string	Needed for item-level analytics and many deep KT pipelines.
skill_name	string	Needed for KC-level BKT or logistic KT.
concept_id	string/int	Often the categorical input for DKT-style concept-level modeling.
correct	0/1	The minimum required supervised signal.
attempt_no	integer	Useful for opportunity counts, curves, and debugging.
group	factor	Needed if you will compare conditions after tracing.

Package-specific minimums

Model / package	Minimum columns	Notes
pyBKT / R BKT	order_id, user_id, skill_name, correct	correct must be coded as response status; pyBKT docs allow -1, 0, 1.
pyKT DKT family	user_id, ordered item/concept sequence, response sequence	In practice you also need train/valid/test splits and integer-coded IDs.
LKT	Anon.Student.Id, Outcome, KC..Default.	The package sample data uses CORRECT/INCORRECT strings, not 0/1.

One toy dataframe, three uses

The bundled file example_data/toy_kt_long.csv is intentionally wide enough to support both interpretable and deep KT workflows. It includes skill_name for BKT and question_id/concept_id for DKT-style preprocessing.

# first rows of the bundled toy file
order_id,user_id,skill_name,question_id,concept_id,correct,timestamp,attempt_no,group
1,S01,Fractions,Q01,C01,0,2026-01-10T09:00:00,1,control
2,S01,Fractions,Q02,C01,1,2026-01-10T09:02:00,2,control
3,S01,Decimals,Q03,C02,0,2026-01-10T09:05:00,1,control

The guide is backed by runnable scripts under tests/, not just pasted code fragments.

Package	Status in the tested environment	Notes
pyBKT 1.4.1	Fit succeeded	Required a serial workaround for EM_fit.run on Windows. Also pinned scikit-learn==1.5.2.
pykt-toolkit 0.0.38	Import + DKT forward pass succeeded	Minimal test produced output shape (2, 5, 8).
torch 2.11.0+cpu	Worked	Sufficient for smoke testing and small examples.

# tests/test_pybkt_python.py
from pyBKT.models import Model
df = pd.read_csv("example_data/toy_kt_long.csv")
model = Model(seed=42, num_fits=1)
model.fit(data=df)
preds = model.predict(data=df)

Package	Status in the tested environment	Notes
BKT 0.1.0	Fit + predict succeeded	Used parallel = FALSE, num_fits = 1 on the toy long-format CSV.
LKT 1.7.0	LKT() succeeded on sample data	The safest first entry point is the package sample / vignette format.

# tests/test_bkt_r.R
library(BKT)
df <- read.csv("example_data/toy_kt_long.csv")
fit_model <- fit(bkt(seed = 42, parallel = FALSE, num_fits = 1), data = df)
preds <- predict_bkt(fit_model, data = df)

# tests/test_lkt_r.R
library(LKT)
data(samplelkt)
lkt_model <- LKT(
  data = samplelkt,
  interc = FALSE,
  components = c("Anon.Student.Id", "KC..Default.", "KC..Default."),
  features = c("intercept", "intercept", "lineafm")
)

Most KT papers underuse visualization. If all you show is AUC, your reader cannot tell whether the model produced an educationally sensible story.

1. Mastery trajectory by opportunity

Plot mean predicted mastery against practice opportunity count, separated by concept and optionally by group. This is the most direct answer to “are students stabilizing?”

2. Predicted vs observed correctness

Calibration-style plots matter because a model can rank students well while still misestimating absolute success probabilities.

3. Heatmap of student × concept risk

Use the most recent mastery estimate per student-concept cell. This is the easiest operational view for instructors or intervention designers.

4. Opportunity curve by group

When you compare conditions, do not only compare final mastery. Compare how quickly mastery appears over repeated attempts.

These are not full statistical models. They are teaching plots designed for beginners who need intuition before code. Change the sliders and watch how a BKT-style mastery curve or a group mastery comparison changes.

Lab 1. What do BKT parameters actually do?

Lab 2. How do group mastery curves tell a story?

Symptom	What it means	Fix
pyBKT import or fit breaks with newer scikit-learn	Version mismatch in utility code paths.	Pin scikit-learn==1.5.2 in the tested environment or verify the package against your own dependency stack.
pyBKT fit fails on Windows at multiprocessing / Pipe / access	pyBKT.fit.EM_fit.run was fragile under the tested setup.	Use a serial workaround like the one in tests/test_pybkt_python.py, or verify whether your own environment reproduces the issue first.
pyBKT returns strange results after tiny toy fits	BKT is being fit on a very small dataset with strong simplifying assumptions.	Treat tiny toy results as smoke tests only, not substantive evidence.
LKT fails on your own CSV even though sample data works	Your columns or coding do not match the expected package idiom.	Replicate samplelkt first: Anon.Student.Id, Outcome, KC..Default., with CORRECT/INCORRECT.
Deep KT preprocessing explodes	Your IDs are not integer-encoded or sequences are not split correctly.	Create stable mappings for concept/item IDs and make the split logic explicit.
Group differences are uninterpretable	You traced mastery but skipped visualization and second-stage modeling.	Plot opportunity curves and then test group differences on mastery summaries.

Question	Short answer	Practical rule
How much data do I need?	There is no single magic number.	You need enough repeated, ordered, skill-linked attempts per learner and per skill to estimate change, not just level. If you only have 2-3 attempts per skill for most students, KT will usually be fragile.
Can I do KT with one pretest and one posttest?	No.	That supports growth or outcome analysis, not tracing. KT needs event-level sequences.
What is a minimally usable KT dataset?	A long event log with one row per attempt.	At minimum: user_id, order_id or timestamp, skill_name or concept tag, and correct.
Do I need timestamps?	Not always, but they help.	Plain BKT can run with stable ordering only. If you want spacing, delay, forgetting, or stealth behavior claims, reliable timestamps become much more important.
Can I run KT with only one skill?	Yes, if that skill has enough repeated opportunities.	One-skill KT can be reasonable in a narrow tutor or unit. It is just less informative than multi-skill tracing.
What if my concept tags are messy?	Clean that before modeling.	Bad KC labels break interpretation faster than most model-choice mistakes.
How many students do I need?	Enough to stabilize patterns, but the key unit is still attempts.	Do not think only in student count. A dataset with many students but almost no repeated opportunities is still weak for KT.
Can I compare treatment and control groups?	Yes.	Use KT to derive mastery trajectories, then compare those trajectories or summaries with second-stage models.
Should I start with DKT?	Usually no.	Start with BKT or logistic KT, then benchmark deep models on the same split if prediction is the goal.
Can KT measure motivation or metacognition?	Not directly.	KT estimates latent proficiency from traces. Motivation and metacognition need their own measures or a broader learner model.
What should I report in a paper?	More than AUC.	Report preprocessing, sequence definition, skill mapping, baseline model, diagnostics, and at least one interpretable trajectory plot.
When is Bayesian network modeling worth the extra complexity?	When one latent skill is not enough.	If you need prerequisites, multiple hidden states, or stealth evidence from behaviors, move beyond vanilla BKT.

• Did you verify one row equals one attempt, not one student summary?
• Did you sort sequences explicitly and document the ordering key?
• Did you confirm skill tags with a human-readable sample?
• Did you fit at least one transparent baseline before deep KT?
• Did you visualize mastery or calibration, not just AUC?
• Did you keep interpretation at the level the model actually supports?
• Did you report package versions and preprocessing assumptions?
• Did you separate smoke-test success from substantive validation?

This module sits above KT in the model family tree. BKT is a specialized dynamic Bayesian model; broader Bayesian networks let you represent prerequisites, multiple latent skills, stealth evidence, and richer learner-model integration in one probabilistic graph.

Variant	Educational use	What it adds
Static BN	Diagnostic assessment	Multiple linked proficiencies, misconceptions, and observed evidence in one graph.
Dynamic BN	Learning progression / stealth assessment	Hidden states evolve across time slices using behavior or performance evidence.
BKT	Per-skill mastery tracing	Interpretable special case of dynamic Bayesian modeling.

Where BN is especially useful in education

• Diagnostic assessment: infer several skills at once rather than one total score.
• Prerequisite modeling: encode that concept B depends on concept A, and concept C depends on B.
• Stealth assessment: update hidden proficiency from gameplay or process behavior without interrupting the activity with constant quizzes.
• Learner model integration: combine knowledge, persistence, strategy use, and help-seeking in the same student model.

Canonical KT / BKT / DKT references

• Corbett, A. T., & Anderson, J. R. Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. Original DOI
• Piech, C. et al. Deep Knowledge Tracing. NeurIPS 2015. Paper page · PDF
• Abdelrahman, G., Wang, Q., & Nunes, B. Knowledge Tracing: A Survey. ACM Computing Surveys DOI · arXiv preprint

CMU / Cognitive Tutor lineage

• CMU DataLab concept page: key concepts, KCs, learning curves, BKT, AFM
• CMU DataLab resource page: CTAT, DataShop, Carnegie Learning, and related tools
• CMU Simon Initiative CTAT page: Cognitive Tutor Authoring Tools
• CMU impact/history page: Cognitive Tutors and large-scale deployment context
• CMU / LearnLab background note: cognitive tutor overview, model tracing, and knowledge tracing
• CMU BKT extension example: Yudelson, Koedinger, & Gordon, Individualized Bayesian Knowledge Tracing Models

Python guides

• pyBKT GitHub: CAHLR/pyBKT
• pyBKT practical article: Bulut et al. (2023), An Introduction to Bayesian Knowledge Tracing with pyBKT. DOI · Article page
• pyKT toolkit site: pykt.org · docs: documentation · code: GitHub
• pyKT benchmark paper: Liu, Z., Liu, Q., Chen, J., Huang, S., Tang, J., & Luo, W. (2022). pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models. NeurIPS Datasets and Benchmarks. NeurIPS page · arXiv

R guides

• CRAN BKT docs: rdrr package page · manual
• LKT package docs are installed locally in this project under tests/r_lib/LKT/doc/. The sample-data workflow in Basic_Operations.Rmd and Examples.Rmd is the most reliable entry point.

Files bundled with this guide

• example_data/toy_kt_long.csv — general toy KT event log
• example_data/toy_lkt_minimal.csv — minimal LKT-shaped CSV
• tests/test_pybkt_python.py · tests/test_pykt_dkt_forward.py
• tests/test_bkt_r.R · tests/test_lkt_r.R
• tests/results/pybkt_result.json · tests/results/pykt_dkt_forward.json · tests/results/bkt_r_result.json · tests/results/lkt_r_result.json

Additional validation checks worth doing

• Check whether your knowledge component mapping is theoretically defensible, not just convenient for export.
• Check whether one row truly equals one opportunity; mixed granularity breaks learning curves and KT.
• Check whether help, hints, or partial credit were collapsed in a way that distorts the correctness signal.
• Check whether train / validation / test splits leak future information for the same learner or sequence.
• Check whether very sparse skills should be merged, excluded, or analyzed descriptively instead of traced.
• Check whether your main claim needs learning curves or AFM-style baselines in addition to KT, especially in the CMU / DataShop tradition.
• Check whether reported gains are prediction gains only or actually support an educational interpretation.