What “validated” means for CANRA OSCE practice
Current status: Not clinically or psychometrically validated
BoardQBank’s 24 original cases and 11-criterion practice rubric are educational simulations. The handbook does not disclose a full examiner marking sheet, an official pass mark, or detailed anchoring for each station. No exam pass probability is provided.
Clinical content review
- Two licensed, experienced naturopathic clinical educators independently review every case for patient safety, differential diagnosis, procedure contraindications and provincial scope.
- Resolve disagreements, log correction rationale and version every case.
- Confirm that cited guidelines genuinely support the specific claims; mark reference gaps explicitly.
- Test at least 5–10 learners per case for clarity and unexpected interpretations before wider deployment.
Simulation realism
- Ask a standardized-patient actor or peer to run the scripted interaction when hands-on demonstration matters.
- Test time transitions at 02:00, 17:00, 21:00 and 25:00 during a full live session.
- Use the observer rating mode and document actions actually observed, not inferred from typed text.
- For acupuncture, practice needling only on suitable artificial pads with supervision and applicable training; for manipulation, no thrust is performed in simulation.
Psychometric study and rater calibration
- Recruit a representative consented pilot cohort; give standardized rater training and blinded case scoring.
- Collect sufficiently large independent assessments, assess inter-rater agreement and rater severity.
- Calculate domain score distributions, missingness, item-total association, and generalizability measures if design supports them.
- Compare formative ratings against external assessment and examine differential performance; investigate negative item discrimination and floor/ceiling effects.
- Determine any standard through a documented defensible standard-setting procedure—do not borrow an unofficial fixed percentage.
Optional administrator summary reports only show descriptive data. Self-assessment and self-declared observer ratings are not independent evidence of psychometric validity.
Privacy and operations
Student free-text entries should only describe fictional stations. Optional account sync stores rubric ratings, safety flags, brief evidence and a learning goal in the same private SQLite store used for student accounts, outside the public root. Browser-only history remains local. Administrators must establish consent, retention, deletion, backup and breach procedures before collecting identifiable performance data.