Known limitations

The limit: The AI scores answers against your rubric and is sometimes wrong — especially on partially-correct answers and unconventional-but-valid reasoning.

What we do about it: The professor always controls the grade: every AI score arrives with the transcript, a written rationale, and a confidence level; you approve, adjust, or override, low-confidence and failed scoring runs are flagged for review instead of auto-finalized, and every override is logged and used to evaluate our scoring.

The limit: Strong accents, noisy rooms, low-quality microphones, and dense technical vocabulary (chemical nomenclature, non-English names, formulas spoken aloud) can still be mis-transcribed.

What we do about it: We moved to a transcription model specifically chosen for fewer hallucinations on noisy audio, the original recording is kept so you can click any transcript line and hear exactly what the student said, and AI-cited quotes are verified verbatim against the transcript — unverifiable evidence flags the response for human review rather than counting toward a score.

The limit: The AI persona in roleplay scenarios follows the goals and assertiveness you set, but it can be more patient and more coherent than a real patient, customer, or negotiator — and may occasionally respond in ways a real person wouldn't.

What we do about it: You author the persona, its goals, and how hard it pushes back; you see the full conversation transcript with per-criterion evidence excerpts; and roleplay is positioned as structured practice and assessment of the student's side of the conversation, not a claim of human-equivalent realism.

The limit: An oral exam inherently requires speaking and hearing, and while parts of the interface have screen-reader and keyboard support, we have not yet completed a full WCAG audit — students who can't complete a spoken assessment currently need an instructor-arranged alternative.

What we do about it: Every student-facing session includes a visible accommodation path ("contact your instructor"), recording consent is explicit with an opt-out, and a structured accessibility audit is on the roadmap before broad institutional rollout.

The limit: Question delivery, the AI examiner, transcription, and scoring are built and tested for English; other languages are unsupported and untested today.

What we do about it: We say so up front rather than letting a Spanish-language oral exam silently degrade — multilingual support is planned and will ship per-language only after transcription and scoring quality are verified in that language.

The limit: LMS integration (single sign-on launch, assignment linking, grade passback) works with Canvas via LTI 1.3 — Moodle, Blackboard, and D2L are not integrated yet.

What we do about it: QPAssess also works standalone with a share link and CSV export, so non-Canvas courses can run assessments today; our LTI implementation follows the 1.3 standard, which is what the other major LMSs accept, so additional platforms are certification work rather than rebuilds.

The limit: Oral assessment means we necessarily record student speech (and webcam/screen only when an instructor enables it, behind an explicit consent screen).

What we do about it: Recordings and transcripts are stored privately, accessible only to the course's educator through expiring signed links, are never sold, and are not used to train AI models.