Open to senior QE and AI quality roles
I build tests that can actually fail.
Ten years hardening UI, API, and data pipelines for federal programs. Now doing the same for production AI systems.
Most automated suites pass without proving anything.
The failure nobody reports is the test that goes green without checking anything. An assertion that cannot fail is not a test, it is a receipt. On a recent enterprise Salesforce suite I audited every assertion for falsifiability, deleted the ones that could not go red, and replaced them with DOM-grounded checks. Defect detection went up. The pass rate went down first, which is the correct direction.
Generated tests make this worse, not better. A model writes an assertion that reads plausibly and compiles cleanly, and nobody checks whether it can ever fail. I use AI heavily for authoring, migration, and triage. I do not let it decide what counts as verified. That line is where a lot of teams are about to get hurt, and it is the reason I spend as much time deleting tests as writing them.
AI systems quality
Validating production chatbots across five risk axes
A federal agency is putting assistants in front of the public. Each axis fails differently, so each one needs its own tool. Knowing which tool catches what is most of the job.
Does the answer survive a check against its own source?
Faithfulness and hallucination scoring against the retrieved context, answer relevancy, and custom judge criteria for domain correctness the generic metrics miss. Graded per response, tracked per release, gated in CI.
MLflow 3 LLM judges
Who does the answer serve worse?
Bias and toxicity scoring across demographic and topical prompt sets, run as a release gate rather than a one-time audit.
Can it be talked out of its instructions?
Prompt injection, jailbreak and DAN patterns, encoding attacks, and training-data leakage probes fired at the deployed endpoint.
What happens under sustained attack?
Multi-turn adversarial campaigns with converters and scorers, pushing past single-shot probes into real red-team pressure.
Did this week's prompt change break last week's answers?
Tracing over prompts, retrievals, and tool calls with versioned evaluation runs, so quality is a trend line instead of an anecdote.
AI in the test framework
Built a validation framework an agent can operate — and drew the line it cannot cross
Testing AI systems is one half. The other is letting AI drive the test framework without letting it decide what counts as verified. A 4,000-line validation package covering 235+ table contracts on a federal medallion platform, reshaped so an agent can run it, triage it, and author against it.
What is AI actually good at here?
Translation and volume. Five Lightning automation frameworks in fourteen months, Gherkin and Cypress migrated to Playwright TypeScript, 200+ cases moved off Selenium — coverage up 50% and a 97% pass rate held through cutover. The model drafts; every generated assertion still gets audited for whether it can go red. A test that cannot fail is a receipt, and a model writes those fluently.
Can a check run outside the notebook?
Every check was a Spark DataFrame function, so nothing ran without a live cluster. Compiling each one to parameterised SQL means the same library runs from a laptop, from CI, or from an agent — and the emitted SQL is inspectable, which is what makes a dry run possible and a failure explicable to something that has to reason about it.
What should an agent be allowed to call?
Ten verbs, not a SQL console: lint a contract, compile it, run it, fetch failures, compare two runs, trend a table's health. Read-only by default — writes need server configuration, not a tool argument. Small surfaces get chosen correctly, and every tool description is prompt tokens on every single request.
What must never become a tool?
Whether a warning is acceptable this quarter. Whether three orphaned reference codes are a source-data problem or a transformation bug. Signing off a promotion gate. That reasoning lives in runbooks built from real root-cause write-ups, where it can be read and argued with — not compiled into a tool that returns a verdict with nothing behind it.
Whose text ends up in the model's context?
Failed checks capture real values, and PII is in scope on this pipeline — so sampling is per-column, default deny, with a hash and a count where a value is not allowed out. Those values also originate in ingested source files, which makes validation results an unusually direct carrier for indirect prompt injection. They come back in structured fields, never interpolated into prose, and the tool description says plainly that they are data, not instructions.
One data contract, asserted at three layers
UI, API, and data tests usually assert three different versions of the truth, then drift apart quietly. This chain gives all three the same source.
Generated, not hand written
The scenario drives the Databricks query and the result becomes the contract. The expectation comes from the data, not from someone's memory of what the data should look like.
Asserted twice
The same contract validates the service response and the rendered screen. When the API and the UI disagree, the run tells you which one moved.
Readable when it breaks
Failures point at the contract field that broke rather than at a stack trace, so a business reviewer can read the result without opening the framework.
What I work in
Test automation
Data and ETL validation
AI systems quality
Delivery and gates
Ten years, mostly on systems that cannot quietly break
Client names are withheld under engagement agreements. Metrics are real.
Lead Test Automation Engineer & Quality Architect
- Built AI validation program for public-facing assistants, covering accuracy, hallucination, safety, security, and bias with DeepEval, MLflow 3, garak, and PyRIT.
- Architected YAML data contract framework: Databricks source-of-truth auto-propagates across service APIs and UI tests with single Allure report.
- Built medallion validation framework from zero requirements: 235+ tables across Bronze/Silver/Gold layers, 40+ validation types per table, all YAML-driven. Gating CI/CD for 90%+ coverage.
- Delivered 5 Lightning automation frameworks in 14 months with AI-augmented test migration (Gherkin/Cypress → Playwright TS). Grew coverage 50% (70→100 cases), maintained 97% pass rate through cutover, trained 3 teams who still expand independently.
- Modernized legacy Selenium + TestNG to Playwright with Cucumber BDD in Java 17: 23 feature files, 54 scenarios, 354 step definitions across 10 applications.
- Cut ETL validation cycle from 5 days to 6 hours by automating 90%+ of scenarios on PySpark + EMR, 16M+ rows validated daily.
Senior SDET
- Rebuilt Selenium and Cucumber framework, taking pass rate from 30% to 97% and cutting runtime by over 80%.
- Expanded automated coverage by 125% while building GitLab CI pipelines with Docker and Allure reporting.
- Led JMeter performance testing, resolving critical API latency and timeout failures.
- Ran Section 508 accessibility testing and remediation support.
QA Manager, Automation Lead, Senior Automation Engineer
- Designed full-stack automation for UI, API, and database validation across regulatory workflows, mentored engineering teams.
- Served as performance architect during surge periods, modeling load and stress for traffic spikes above 250%.
- Led independent verification and validation for reporting systems on Jenkins pipelines.
- Built ETL validation with FitNesse, DbFit, and PostgreSQL.
Senior Test Automation Engineer
- Built a hybrid Selenium framework for cross-browser and cross-platform execution with FitNesse and Xebium on Jenkins, plus TestNG, Maven, and JMeter folded into regression.
Built outside work
ForgeQA
Full-stack AI-native test automation platform: React dashboard with role-based views (viewer, qa-engineer, a11y-tester, admin), framework migration wizard (Gherkin/Cypress/Playwright TS), AI debug agent, Playwright MCP for interactive debugging, Salesforce Lightning helpers (SmartLocator, SfPage), typed test data factories, Allure reporting, Jenkins integration, encrypted token management. Built for federal compliance (Section 508, WCAG 2.1 AA, PII-safe).
Vision Vault Systems
Case studies covering federal QA automation, Databricks medallion validation, AI-augmented Salesforce migration, and big data ETL validation. Six deployments with real metrics and anonymized clients. The site's QA maturity assessment is a production Claude agent I built end to end: an adaptive five-phase interview that scores a team and returns a costed report — grounded in one versioned knowledge file so it cannot invent capabilities, a strict JSON output contract, a hard turn cap, and a scripted fallback so the page still completes with the model switched off.
Prime Custom OS
Designed and built the whole product — a build, not a testing project. The business operating system a Northern Virginia remodeling company runs on daily: leads, CRM, projects, estimates and change orders, scheduling, invoicing, customer portal, analytics. 315 API endpoints over 19k lines of Python, a 22k-line React front end, 108 database migrations, nine CI workflows for staged deploys, backups, and uptime. Solo, from schema to deploy, in production with a team using it every day.
EnChaptered
An AI storybook generator for children aged two to eight. Built and shipped solo, from design through API to deploy.
Need someone to own quality on a data or AI platform?
Based in Chantilly, Virginia. Open to senior SDET, quality architect, and AI quality roles, remote or hybrid.