QA Engineer

Sr. QA Engineer

HerculesAI is an AI-powered productivity company transforming how professionals work. We build intelligent tools that automate complex tasks, streamline workflows, and help teams focus on meaningful, high-impact work. Headquartered in California with global teams across North America and Europe, HerculesAI combines cutting-edge AI with a fast-paced startup environment where innovation, ownership, and collaboration drive everything we do.

As a Sr. QA Engineer at HerculesAI, you'll lead the charge in embedding quality throughout our entire software development lifecycle. You'll own end-to-end product quality—from defining success criteria and automation strategies to managing releases and ensuring post-deployment stability. You'll work closely with Engineering, Product, AI, and DevOps teams to ensure every release meets our standards for reliability, performance, and user impact—making quality a shared responsibility across the organization.

What you'll do:

1. Lead end-to-end product quality and integrate QA across the SDLC (shift-left, CI/CD quality gates, test evidence as part of "definition of done").
2. Own Release Management: plan releases, cut release candidates, manage freeze windows, lead go/no-go, coordinate phased rollouts (flags/canary), and execute rollback plans.
3. Design and maintain automation for UI, API, component, and E2E tests in partnership with all Engineering teams; establish non-functional baselines (performance, security, accessibility, resilience).
4. Design, own, and evolve performance and stress testing (load, capacity, scalability): define SLIs/SLOs, create repeatable perf/stress suites, profile bottlenecks, and gate releases via CI/CD and PRV.
5. Translate business initiatives into clear acceptance criteria and measurable Success Criteria Docs (KPIs, telemetry, rollout/rollback triggers).
6. Partner with Engineering, Product, AI, and DevOps to ensure observability, PRV (post-release verification), incident retrospectives, and continuous improvement of quality KPIs.

Qualifications:

• 6–10+ years in QA/Software Engineering, including ownership of release management and large-scale test automation.
• Required: Proficiency in Python (test harnesses, AI/LLM eval tooling, CI utilities) and ReactJS (TypeScript preferred) for UI testability reviews and building test fixtures/mocks.
• Hands-on with modern delivery stacks (microservices, containers/K8s, CI/CD) and test frameworks (Playwright/Cypress, PyTest/JUnit/TestNG, k6/JMeter for performance).
• Demonstrated experience validating AI/LLM features (prompt testing, guardrails/red-teaming, offline/online eval alignment).
• Strong systems thinking and risk-based testing; fluency in telemetry/observability (logs, metrics, traces) and security/a11y/performance gates, including capacity planning and perf profiling.
• Excellent communication skills for turning ambiguous requirements into unambiguous, testable criteria and leading cross-functional quality reviews.

Success Metric Examples:

• Shift-Left & SDLC Health: % stories with testable criteria at grooming ↑; pre-merge test pass rate ↑; escaped defects from unit/component layers ↓; lead time for changes ↓.
• Release Quality: change failure rate ↓; rollback/hotfix frequency ↓; PRV (post-release verification) pass rate ↑; incident MTTR ↓.
• Automation & Coverage: automated regression coverage ↑; flaky test rate ↓; time to fix flaky tests ↓; security/a11y/perf gate pass rate ↑.
• Performance & Resilience: p95/p99 latency ↓; throughput ↑; saturation/error budget burn ↓; successful load/stress/soak test pass rate ↑; resiliency checks (retry/backoff, timeouts) pass rate ↑.
• Observability & Evidence: % launches with telemetry + dashboards + alert thresholds = 100%; quality evidence attached to releases ↑; data-driven retros with action closure rate ↑.