Delaware Pilot — Technical
← Back to landing Architecture Models Datasets Evaluation Privacy & Controls

Delaware Pilot Technical Page

This page is meant to be readable. It describes how the pilot can run end-to-end: what we train, what data “looks like,” how updates move, and how we evaluate quality and safety—without moving raw institutional data between partners.

Architecture Candidate models Phase 1 use cases Dataset templates + examples Evaluation + benchmarks Privacy controls Governance

Architecture (pilot-friendly)

Local training + shared updates; PATHS adds structure + provenance.

Local (each HEI)

PATHS Engine ingests approved sources, structures them into a knowledge graph, and prepares training/eval data locally. Training runs inside each institution’s boundary.

Local data control PATHS graph Local evaluation

Federation layer

A coordinating server aggregates encrypted model updates and distributes the improved model back to partners. No raw records are centralized.

Secure updates Aggregation Repeatable rounds
Sequence (one training round) ⌄
  1. Coordinator sends current model weights (or adapter weights) to each institution.
  2. Each institution trains locally for N steps on its own prepared dataset.
  3. Each institution sends back model updates (or adapter deltas), not raw data.
  4. Coordinator aggregates updates and produces a new global model.
  5. Global model is redistributed to institutions for the next round or deployment.

Candidate model strategy (pilot constraints)

Right-sized models + adapter-based training (where feasible).

Pilot-friendly default: start with a smaller/medium open model and use adapter-based fine-tuning (e.g., LoRA/PEFT) where possible. This reduces compute requirements and makes federation more practical.
Option A: Small/medium model + adapters (recommended) ⌄
  • Train adapters for “case-study reasoning + rubric feedback” style.
  • Use PATHS-backed retrieval for citations and grounding (instead of baking everything into weights).
  • Share adapter updates across institutions through federated rounds.

Candidate families (pick based on institutional constraints): open-weight 7B–13B class models, or locally-hostable instruction-tuned variants.

Option B: Retrieval-heavy + minimal tuning ⌄
  • Keep the base model fixed; focus on strong retrieval, citations, and templates.
  • Federate on evaluation, prompt templates, and dataset generation—then decide later if tuning is needed.
Option C: Larger model (only if partners have capacity) ⌄

Larger models can help reasoning, but increase compute and privacy risk surface area. If explored, do it after Phase 1 success, with explicit governance and resourcing.

Phase 1 use cases: Case studies

What the model actually does for teaching & learning.

Generate case studies (faculty-controlled)

Create scenarios aligned to learning objectives, with constraints (level, domain, length, required concepts).

Rubric-aligned feedback

Evaluate responses against criteria, explain gaps, and suggest revisions without “giving the answer away.”

Grounded responses with citations

When used with retrieval, answers reference approved sources (policy, readings, slides) and show provenance.

Equity checks + consistency

Test parity across partner institutions with shared benchmarks and “stress tests” for bias and drift.

Dataset templates (what “data” looks like)

Examples are synthetic-format templates — adapt locally.

Important: The pilot can be run using faculty-reviewed synthetic data, de-identified data, and/or institution-approved content. Each HEI controls what is included locally.
Dataset A: Case-study generation (JSONL) ⌄

One record per case prompt/target. Use it for fine-tuning or evaluation.

{
  "case_id": "BUS-ETH-001",
  "domain": "Business Ethics",
  "level": "Undergraduate",
  "learning_objectives": [
    "Identify stakeholder conflicts",
    "Apply an ethical framework to justify a decision"
  ],
  "required_concepts": ["stakeholders", "duty-based ethics", "conflict of interest"],
  "constraints": {
    "length_tokens": 550,
    "tone": "realistic, academic",
    "include_data_table": true
  },
  "prompt": "Generate a case study about ...",
  "target_case_study": "A mid-sized nonprofit receives ..."
}
Dataset B: Rubric-aligned feedback (JSONL) ⌄

Evaluates a student response against a rubric. You can also use this format for model comparison.

{
  "case_id": "NURS-CLIN-014",
  "rubric": [
    {"criterion":"Clinical reasoning", "levels":["insufficient","developing","proficient","advanced"]},
    {"criterion":"Safety & escalation", "levels":["insufficient","developing","proficient","advanced"]},
    {"criterion":"Clarity", "levels":["insufficient","developing","proficient","advanced"]}
  ],
  "student_response": "I would prioritize ...",
  "target_feedback": {
    "scores": {
      "Clinical reasoning": "developing",
      "Safety & escalation": "proficient",
      "Clarity": "developing"
    },
    "strengths": ["Good prioritization of symptoms", "Mentions escalation"],
    "gaps": ["Missing justification using vitals trend", "No reference to protocol"],
    "next_steps": ["Add a brief SBAR structure", "Cite the guideline section on ..."],
    "guardrail_note": "Do not provide definitive medical advice; keep educational framing."
  }
}
Dataset C: Grounding + citations (RAG-compatible) ⌄

Pairs a question with approved source snippets and a citation-required answer template.

{
  "question": "According to our policy, what are the steps for ...?",
  "approved_sources": [
    {
      "source_id": "POLICY-IT-ACCESS-2025",
      "title": "Access Management Policy",
      "excerpt": "Section 4.2 states ...",
      "citation": "Access Management Policy, §4.2"
    }
  ],
  "target_answer": "Step 1 ... (cite: Access Management Policy, §4.2)"
}
PATHS graph export (example fields)

For institutions using PATHS graphs, a common pattern is exporting graph-derived features locally:

{
  "node_type": "Concept",
  "node_id": "concept:operant-conditioning",
  "labels": ["Psychology", "Learning"],
  "neighbors": [
    {"rel":"RELATES_TO","node":"concept:reinforcement"},
    {"rel":"CONTRASTS_WITH","node":"concept:classical-conditioning"}
  ],
  "metadata": {"source":"PSY101_slides_week3.pdf","verified":true}
}

Evaluation + benchmarks

What we measure so “better” is real, not vibes.

Core evaluation dimensions ⌄
  • Rubric alignment: does feedback match instructor criteria?
  • Grounding: does it cite approved sources when required?
  • Helpfulness: faculty/student ratings (or structured review)
  • Safety: avoids prohibited advice and sensitive outputs
  • Equity checks: parity tests on shared benchmark sets
Benchmark pack (shared across institutions) ⌄

A shared benchmark pack can be built from faculty-reviewed synthetic cases and common rubric templates. Each institution can add local “holdout” tests without sharing them.

Privacy controls (what we do + what we don’t do)

Designed for real higher-ed constraints.

Controls we expect to use ⌄
  • Local-only training data handling (approved sources only)
  • Role-based access controls on training and deployment
  • Logging that supports auditing without storing sensitive content
  • Evaluation gates before any broader deployment
What we explicitly avoid in this pilot ⌄
  • Centralizing raw student records across institutions
  • “Black box” deployment with no provenance or evaluation
  • Unbounded use cases (Phase 1 is intentionally narrow)

Governance

How we decide what’s acceptable, and who decides.

Lightweight governance model (pilot scale) ⌄
  • Working group with representation from: faculty, IT/security, library, student success, leadership
  • Pre-agreed benchmarks and acceptance thresholds
  • Documented model changes per round (model card style notes)
Next step: If you want, we can add a “Partner Packet” download area (PDF) + a public-safe list of participating HEIs once approved.
← Back to landing Email: ai-blue@udel.edu
© University of Delaware — PATHS Engine
Pilot landing • Federated learning • Contact