Senior QA Analyst, AI & Threat Analytics - Remote Eligible

Senior QA Analyst AI & Threat Analytics

Keeper is hiring a talented Senior QA Analyst to join our AI & Threat Analytics team. This can be a 100% remote position from select locations with an opportunity to work a hybrid schedule for candidates based in the El Dorado Hills, CA or Chicago, IL metro area.

Keeper's cybersecurity software is trusted by millions of people and thousands of organizations globally. Keeper is published in 21 languages and is sold in over 120 countries. Join one of the fastest-growing cybersecurity companies and help ensure the quality, safety and reliability of Keeper's next-generation AI features and threat analytics capabilities.

Keeper Security is transforming cybersecurity for organizations around the world with next-generation privileged access management. Keeper's zero-trust and zero-knowledge cybersecurity solutions are FedRAMP and StateRAMP Authorized, FIPS 140-2 validated, as well as SOC 2 and ISO 27001 certified. Keeper deploys in minutes, not months, and seamlessly integrates with any tech stack to prevent breaches, reduce help desk costs and ensure compliance. Trusted by thousands of organizations to protect every user on every device, Keeper is the industry leader for best-in-class password management, secrets management, privileged access, secure remote access and encrypted messaging.

The Senior QA Analyst AI & Threat Analytics will lead testing efforts for AI-driven features and machine learning models across Keeper's platform, including LLM-based summarization, autofill and classification systems, and behavior analysis pipelines. You will design and execute test strategies that validate model accuracy, reliability, guardrails, data quality, and integration with production systems. This role requires hands-on Python automation expertise, strong analytical and evaluation skills, and the ability to test nondeterministic systems with rigor and consistency. You will collaborate closely with ML engineers, security teams and product engineering to ensure AI features meet Keeper's quality, safety and performance standards.

Responsibilities

• Design and execute test plans for AI and ML-driven features, including classification, summarization, anomaly detection and autofill systems
• Develop Python-based automation and evaluation scripts to validate model performance, reliability and output quality
• Test LLM-based features for accuracy, guardrail effectiveness, hallucination risk and behavior under edge cases
• Validate data pipelines, input/output transformations and model integration points across Keeper's platform
• Conduct regression testing on ML models and AI workflows to ensure reliability across releases
• Test enterprise-scale behavior analysis features driven by data from ARAM, PAM and session recordings
• Validate security, privacy and zero-trust requirements for AI model interactions and data handling
• Collaborate with ML engineers and cross-functional teams to refine model behavior, identify defects and improve output quality
• Analyze logs, metrics and model outputs to detect anomalies, performance issues or silent failures