Large language models, education
Marvel AI Engine
The reasoning layer behind a teaching assistant that works out where a student is stuck and adapts instead of answering.
- Role
- Machine Learning Engineer
- Organisation
- Marvel AI
- Period
- 2023
- Status
- Shipped
- Domain
- Large language models, education
01
Overview
Marvel AI is a teaching assistant platform. The engine is the part that holds a conversation with a student, forms a view of what they do and do not understand, and adjusts what it asks next.
The design goal was a patient tutor rather than an answer service. A system that gives a correct answer immediately is easy to build and teaches very little.
02
Problem
A general purpose model asked to help with homework will simply do the homework. It has no representation of the student, no memory of what they got wrong ten minutes ago, and no reason to withhold an answer.
Tutoring needs the opposite behaviour. The system has to locate the specific misconception, then choose a question that tests it, and it has to hold that understanding across a long conversation without losing the thread.
03
Solution
The engine maintains an explicit model of the student separate from the conversation: which concepts have been demonstrated, which are shaky, and which misconception best explains the errors seen so far.
That state drives a decision layer that chooses between explaining, questioning, and working through an example, and it constrains generation so the assistant does not hand over an answer the student is close to reaching. Long conversations are handled by summarising into that state rather than by growing the prompt indefinitely.
04
Technical architecture
- 01
Session state
A typed record of concept level mastery, open misconceptions, and conversation summary, persisted per student and updated after every turn.
- 02
Diagnosis
Student responses are classified against the concept graph to identify which specific misunderstanding is most consistent with the errors observed.
- 03
Pedagogical policy
A decision layer selects the next move, explain, question, or worked example, from the diagnosis rather than from the last message alone.
- 04
Generation
Prompt construction assembles state, curriculum context, and the selected move, with guardrails preventing direct answer disclosure.
- 05
Serving
A FastAPI service with streaming responses, session persistence, and per turn logging for later analysis of where students actually get stuck.
05
Technologies
Language models
- Prompt orchestration
- Structured output
- Streaming inference
Backend
- Python
- FastAPI
- PostgreSQL
- Redis
Modelling
- Response classification
- Concept graph
- Session state
Operations
- Docker
- Structured logging
- Prompt versioning
06
Challenges
Keeping context without unbounded prompts
A tutoring session runs long. Appending history until the window fills degrades quality and cost at the same time, so state had to be compressed into an explicit student model that carries the meaning without the transcript.
A helpful model is a bad tutor
The default behaviour of every model available is to be maximally helpful, which here means giving away the answer. Preventing that reliably took explicit policy in the decision layer rather than instructions in a prompt.
Evaluating teaching is not evaluating text
There is no reference answer for a good tutoring turn. Evaluation moved to whether the diagnosis matched the misconception a reviewer identified, which is measurable in a way that response quality alone is not.
07
Outcomes
- Conversations that hold their thread across long sessions without prompt growth
- Misconception level diagnosis rather than turn by turn answer matching
- Per turn logging that surfaced where the curriculum, not the student, was the problem
08
Key learnings
- Putting state outside the prompt is what makes a long conversation tractable. The prompt is a rendering of state, not the memory itself.
- When a model default behaviour is wrong for the product, constrain the system around it rather than asking it to behave differently.
- Defining the evaluation target precisely, diagnosis accuracy instead of response quality, made the whole project measurable.