/* John Thomas · Fairfax, VA */

9 years of federal SIEM, → aimed at AI security.

Principal Splunk consultant translating proven detection-engineering and compliance discipline into the AI/LLM security domain. Eight projects, every one with a threat model, a measurable detector, and a SOC-shaped output — built on the Anthropic SDK and mapped to MITRE ATLAS, OWASP LLM Top 10, and NIST AI RMF.

CISM Splunk Enterprise Architect Splunk Core Consultant SOAR Automation Developer Cribl Stream + Edge Admin Qmulos Q-Compliance

Why this portfolio exists

AI Security Engineer postings ask for two skill stacks that almost never live in the same person: detection-engineering muscle from a mature SOC, and current LLM/agent security knowledge. I have the first from nine years of building Splunk-based SIEMs at federal agencies (IRS, federal health, federal oversight) and Fortune-500 banks. This portfolio is me showing the second.

Detection engineering, not demos

Each project has inputs, a detector or guardrail, and a measurable output — precision/recall/F1 on a held-out set, or a clear pass/fail signal. Project 1 ships F1 = 1.000 on the included eval set with CI enforcing an 0.85 regression floor.

Splunk + AI, deliberately

Verdicts are CIM-aligned JSON events ready for Splunk HEC. Project 4 generates ATLAS-mapped Splunk ES correlation searches end-to-end. The portfolio reads as content for a SOC team that has just been handed AI workloads.

Federal-fluent

Built by someone who already does ATO evidence packages on Splunk via Qmulos. NIST 800-53 control IDs, FedRAMP awareness, and a NIST AI RMF gap analyzer (Project 8) that consumes the other seven projects as evidence sources.

Reproducible, offline by default

Every project's heuristic / deterministic path runs without an API key. Reviewers can clone, pytest -q, and read JSON output in under five minutes. The Anthropic SDK path is opt-in.

The eight projects

Each project is self-contained — its own README, requirements, tests, and threat model — so any single one can be lifted into a public repo as a job-application artifact. Project 1 is a working MVP; the rest are spec-quality and scheduled into a two-week sprint.

01 / mvp working mvp · F1 = 1.000

LLM Prompt Injection Detection Pipeline

Hybrid heuristic + Claude-judge detector with optional NVIDIA NeMo Guardrails third layer. Verdicts forward to Splunk ES as notable events. Ships with eval set, pytest, CI, asciicast demo, and a 2,400-word engineering write-up.

AML.T0051 AML.T0054 Splunk HEC Anthropic SDK NVIDIA NeMo Guardrails
02 / mvp working mvp

AI Red-Team Automation Framework

Automated probing of LLM applications against the OWASP LLM Top 10, with findings auto-mapped to NIST AI RMF subcategories. Ships with 20 starter probes, three-layer scorer, demo target, pytest suite, and a 3,300-word engineering write-up.

OWASP LLM 2025 AI RMF ATLAS
03 / mvp working mvp

RAG Pipeline Security Auditor

Static corpus scanner — finds indirect-injection, PII, credentials, and sensitivity markers in RAG corpora before they reach a production LLM. Reuses Project 1's detector. Ships with 4 scanners, 6 demo docs, AI RMF-aligned audit, pytest, and an engineering write-up.

LLM01 / LLM02 AI RMF CUI / FOUO / PHI
04 / mvp working mvp

SIEM Correlation Rule Generator

One canonical YAML rule → four platform-native outputs (Splunk ES correlation searches, Sigma rules, Cortex XSIAM XQL, NVIDIA Morpheus pipeline configs). 8 starter rules covering Projects 1–3 event shapes, auto-generated AI-RMF coverage map.

ATLAS Splunk ES Sigma · XSIAM NVIDIA Morpheus
05 / mvp working mvp

LLM Output Compliance Monitor

Runtime output guardrail with PII / secrets / classification / policy filters plus optional NVIDIA NeMo Guardrails fifth filter. Every decision is appended to a hash-chained, tamper-evident audit log — the FedRAMP-flavored integrity-evidence pattern.

FedRAMP Hash chain AI RMF NVIDIA NeMo Guardrails
06 / mvp working mvp

Model Supply Chain Risk Scanner

Pickle-opcode disassembly without execution + provenance + risky-deps scanning of model artifacts. Emits a CycloneDX 1.5 SBOM fragment for downstream supply-chain tooling.

LLM03 CycloneDX 1.5 Pickle scan
07 / mvp working mvp

Adaptive Threat Intel Summarizer

STIX 2.1 → cited analyst briefs + Splunk notables. Citation verifier rejects hallucinated references — every claim Claude writes is bound to a real bundle object ID before reaching the SOC.

STIX 2.1 Citation-verified Splunk notable
08 / capstone working mvp

NIST AI RMF Compliance Gap Analyzer

Reads outputs from Projects 1–7, maps every finding to its AI RMF subcategory, produces continuous-monitoring evidence + gap report. The Qmulos Q-Compliance / Q-Audit pattern, applied to AI governance.

AI RMF 1.0 Continuous evidence Capstone
09 / agent working mvp

Autonomous SOC Analyst Agent

Agentic tool-use loop over Splunk notables. Six analyst tools, deterministic stub LLM client for offline runs, 8-case eval suite at 100% pass rate. The agent's input layer is defended by Project 1 — closed-loop AI-security defense-in-depth.

Agent Tool use Eval suite

What I'm bringing in the door

Nine years of doing the unglamorous work that makes detections actually fire — data onboarding, CIM, search performance, ATO evidence — applied to AI workloads.

Splunk + SIEM

  • Splunk Enterprise & Splunk Cloud (FedRAMP)
  • Enterprise Security, ITSI, SOAR, UBA
  • CIM TAs, data model acceleration, tstats
  • Detection content + correlation searches
  • Cribl Stream / Edge admin

AI Security

  • Prompt-injection & jailbreak detection
  • MITRE ATLAS, OWASP LLM Top 10 (2025)
  • NIST AI RMF 1.0 control mapping
  • RAG pipeline auditing & output guardrails
  • Anthropic / Claude SDK, Python

Federal & compliance

  • 9+ yrs federal SIEM & detection engineering
  • NIST 800-53, RMF, FedRAMP, CMMC
  • Qmulos Q-Compliance / Q-Audit
  • ATO evidence collection & ConMon
  • ISACA CISM

Talk to me

Open to AI Security Engineer / Detection Engineering roles, federal or commercial. Strong preference for teams shipping detection content for AI workloads (XSIAM, ES, or comparable).