Applied AI Prototypes

Note on Scope: The initiatives detailed below represent rapid architectural prototypes and hackathon explorations designed to evaluate practical AI workflows, rather than long-running production systems. They focus on evaluating system boundaries, output determinism, and responsible AI guardrails.

Overview

As generative AI and large language models (LLMs) matured, evaluating how these capabilities integrate securely and reliably into enterprise systems became paramount. Rather than deploying black-box AI directly into workflows, these hackathon prototypes explored architectural patterns for structured information extraction, human-in-the-loop validation, and traceable AI decision-making.

The prototypes focused on two high-utility enterprise domains:

1. Enterprise Meeting Action Tracking: Extracting assignees, deadlines, and committed deliverables from multi-party conversation transcripts.
2. Citizen-Service Inquiry Triage: Classifying multi-lingual public service requests, detecting urgency, and preparing verifiable response templates for municipal caseworkers.

Problem Statement

Unconstrained LLM outputs introduce critical risks in business settings:

1. Non-Deterministic Outputs: Free-form prose responses cannot be reliably processed by downstream enterprise databases or automated schedulers.
2. Hallucination & Misattribution: In multi-party conversations, LLMs may assign obligations to participants who did not commit to them.
3. Lack of Auditability: Automated decisions without source reference citations undermine organizational compliance and trust.
4. Data Privacy Boundaries: Unfiltered prompts risk leaking proprietary meeting context or confidential citizen identifiers.

My Role & Design Focus

• Architectural Guardrails: Formulated end-to-end integration designs prioritizing deterministic outputs, privacy boundaries, and validation checkpoints.
• Structured Schema Design: Authored strict JSON schemas with field-level constraints, fallback structures, and confidence thresholds.
• Human-in-the-Loop Workflow: Designed review interfaces requiring user sign-off before AI-generated tasks could create calendar events or database records.
• Evaluation & Prompt Engineering: Crafted few-shot prompt templates and system directives enforcing attribution traceability back to transcript line ranges.

System Architecture: Deterministic AI Pipeline

Both prototypes adhered to a structured, multi-tier processing pipeline separating raw LLM inference from enterprise state changes:

┌─────────────────────────────────────────────────────────────┐
│                 Input Source (Transcript / Inquiry)         │
└──────────────────────────────┬──────────────────────────────┘
                               │ Sanitization & PII Redaction
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                Prompt Orchestration Layer                   │
│   - System Prompt with Few-Shot Grounding Examples          │
│   - Schema Constraints (Function Calling / JSON Mode)       │
└──────────────────────────────┬──────────────────────────────┘
                               │ Structured API Request
                               ▼
┌─────────────────────────────────────────────────────────────┐
│                       LLM Inference                         │
└──────────────────────────────┬──────────────────────────────┘
                               │ Raw Structured Payload
                               ▼
┌─────────────────────────────────────────────────────────────┐
│            Validation & Traceability Engine                 │
│   - JSON Schema Verification & Type Check                   │
│   - Source Span Verification (Grounding Reference Check)    │
│   - Confidence Score Calculation                            │
└──────────────────────────────┬──────────────────────────────┘
                               │ Validated Candidate Items
                               ▼
┌─────────────────────────────────────────────────────────────┐
│               Human-in-the-Loop Review UI                   │
│   [Associate verifies, adjusts, or rejects candidates]      │
└──────────────────────────────┬──────────────────────────────┘
                               │ Confirmed Actions Only
                               ▼
┌─────────────────────────────────────────────────────────────┐
│              Downstream Enterprise Systems                  │
│       (Task Trackers / Databases / Notification Hub)        │
└─────────────────────────────────────────────────────────────┘

Prototype Implementations

Prototype 1: Enterprise Meeting Action Tracker

• Mechanism: Processed meeting audio transcripts to detect explicit commitments (“I will prepare the deployment diagram by Thursday”).
• Attribution & Spans: The model was constrained to output the speaker name, committed task, target date, and the verbatim transcript quote.
• Confidence Gating: Items below an 80% confidence threshold were flagged as “Tentative Ambiguities” for user verification.
• Outcome: Produced a clean summary card allowing the meeting chair to check off verified tasks with one click to populate project boards.

Prototype 2: Citizen-Service Inquiry Assistant

• Mechanism: Analyzed incoming public service queries regarding local infrastructure, permits, and municipal services.
• Categorization: Extracted primary issue classification, geo-location hints, and urgency tiers (Emergency, Standard, Informational).
• Draft Assistance: Drafted courteous, policy-aligned preliminary response templates containing official links, leaving final dispatch strictly to human municipal staff.

Engineering Challenges & Design Decisions

Challenge 1: Ensuring Reliable JSON Structure

• Problem: Early LLMs occasionally returned Markdown wrappers (json ... ) or malformed commas, breaking automated JSON parsers.
• Solution: Enforced API-level schema enforcement (JSON mode) coupled with resilient client-side parsers that stripped formatting artifacts and validated against strict schemas before memory instantiation.

Challenge 2: Eliminating Unverified Automations

• Problem: Allowing AI to autonomously assign project tasks or send emails risked unintended communication and false alarms.
• Solution: Implemented a strict zero-autonomous-execution rule. AI outputs were strictly treated as “draft recommendations” pending explicit human approval.

Key Learnings

1. AI is an Assistant, Not an Autonomous Agent in Enterprise Workflows: Introducing a human-in-the-loop review step transforms AI from a risky liability into a high-leverage productivity multiplier.
2. Schema Enforcement is Crucial: Constraining LLMs to strict output schemas is essential for integrating AI capabilities with existing enterprise software stacks.
3. Traceability Creates Trust: Users are far more confident accepting AI recommendations when the model highlights the exact source sentences that justify its output.