Writing exam questions that meet strict NASBA compliance standards is a grind. It used to take our Subject Matter Experts (SMEs) weeks of manual drafting to create assessments. I decided to automate that bottleneck by building NASbot—a custom AI agent that turns course content into compliant exam questions in seconds.
By shifting our experts from "drafting from scratch" to "reviewing and refining," we didn't just speed up the process—we completely transformed how we build courses.
The Highlights
The Role: Primary Architect & Creator.
The Mission: Use AI to kill the assessment bottleneck without breaking regulatory rules.
The Result: Cut the question-drafting cycle from weeks to hours while maintaining 100% compliance.
The Strategy
SMEs are brilliant at strategy, but their time is often wasted on the repetitive task of drafting multiple-choice questions. NASbot changes the math:
Smart Extraction: The agent pulls from PowerPoints and transcripts to create 10 final exam questions and 3 review questions per course.
Built-in Compliance: I "taught" the AI our internal style guides and NASBA standards, so the output isn't just fast—it’s actually usable.
The Human Element: This wasn't about replacing SMEs; it was about giving them their time back. They still do the final spot-checks, ensuring the quality stays high while the AI handles the heavy lifting.
The Technical Build
I didn't just point a prompt at a chatbot; I built a governed system:
The Architecture: Designed a dual-input pipeline that reads both slide decks and transcripts.
Bloom’s Taxonomy: Integrated cognitive level mapping to ensure questions actually test the right level of learning.
Seamless Output: The AI delivers data in formats ready to drop straight into Articulate Rise or Storyline, complete with distractor rationales (the "why" behind the wrong answers).
Security & Governance: Built-in strict controls to ensure the AI only uses approved documents and never touches sensitive personal info (PII).
The Payoff
Massive Time Savings: We moved from a weeks-long manual process to a system that generates a full draft in an afternoon.
Consistency at Scale: No more "compliance drift." Every question follows the same rigorous standards every time.
Future-Proof: The success of the NASbot pilot has become the blueprint for how we’ll use "Agentic AI" across the rest of our learning media.
The Real Takeaway
The biggest lesson here? Governance is what turns a "cool AI tool" into a "trustworthy system." By being deliberate about a controlled rollout and setting strict rules for where the AI gets its info, I built something the firm actually trusts to handle high-stakes regulatory content.
