Senior QA Engineer (CRO) - Remote Eligible

Senior QA Engineer

Winona is one of the leading telemedicine companies providing HRT for women in menopause. We've built all of our technology in house including an EHR, patient care admin system, patient portal, website, marketing technology etc. Winona has two compounding pharmacies and a team of in house physicians providing world-class care via a fully vertically integrated business model.

We are seeking a highly skilled Senior QA Engineer with a focus on Conversion Rate Optimization (CRO) and web experimentation. In this role, you will play a key role in ensuring the quality, reliability, and accuracy of web-based experiments that directly impact business growth. This is a hands-on, senior-level role where you will be responsible for validating A/B tests, ensuring accurate tracking, debugging experiment issues, and collaborating closely with developers, product managers, marketers, and analysts. The ideal candidate has deep QA expertise, CRO testing experience, and a passion for uncovering insights that drive better user experiences and higher conversions.

Seniority Level: Senior Level

Responsibilities

• Own end-to-end QA for CRO experiments across multiple platforms and devices.
• Validate experiment setups in Optimizely (and similar platforms) including targeting rules, traffic allocation, and variant logic.
• Verify analytics and tracking (GA4, Adobe Analytics, Mixpanel, Segment, Amplitude, etc.) to ensure accurate data collection.
• Debug front-end issues (selectors, dataLayer, JavaScript, HTML/CSS) that impact experiment performance.
• Execute QA across browsers and devices using tools like BrowserStack.
• Identify risks such as flicker, layout shifts, accessibility issues, or personalization conflicts.
• Proactively identify potential conflicts and risks that could disrupt core user flows or break critical site functionality.
• Document QA findings, edge cases, and validation results with clarity.
• Collaborate cross-functionally with developers, marketers, product managers, and analysts.
• Contribute to QA process improvements, including automation opportunities.

Qualifications

• 5+ years of experience in QA, with at least 3 years focused on CRO testing or web experimentation.
• Hands-on experience with Optimizely or similar A/B testing platforms (VWO, Google Optimize, etc.).
• Strong knowledge of validating tracking with GA4, Adobe Analytics, Amplitude, Mixpanel, or Segment.
• Proficiency with browser dev tools for debugging JS, HTML, and CSS.
• Familiarity with tools like BrowserStack and Postman.
• Experience working cross-functionally with developers, product managers, marketers, and analysts.
• Ability to build and implement experiment variants directly in testing tools or in code.
• Understanding of responsive design, accessibility, and performance optimization.
• Ability to debug broken selectors, dataLayer issues, console errors, and personalization conflicts.
• Experience validating custom events, GTM/Optimizely event tracking, and ensuring correct data flow.
• Knowledge of A/B test risks such as sample ratio mismatch (SRM) and traffic allocation errors.
• Strong understanding of asynchronous scripts, multi-variant testing, and testing under real traffic conditions.
• A personal computer with stable internet capable of efficiently running these development tools and applications.
• Available to work 3–4 hours during EST to collaborate with the US team.

What's in it for you?

As a full-time member of our team, you'll enjoy:

• Work wherever you choose.
• Unlimited Paid Time off.
• Paid non-working holidays per country of residence.
• Pro-rated 13th-month bonus in select regions.
• Referral bonuses.
• Financial support for online courses.
• Fun and casual work environment.
• Employee engagement activities and virtual gatherings.
• We are a diverse, global team!

Equal Employment Opportunity

Winona provides equal employment opportunities to all qualified individuals without regard to race, color, religion, sex, gender identity, sexual orientation, pregnancy, age, national origin, physical or mental disability, military or veteran status, genetic information, or any other protected classification. Equal employment opportunity extends to all aspects of employment including, but not limited to, recruitment, hiring, training, promotion, demotion, transfer, leaves of absence, and termination. Winona takes allegations of discrimination, harassment, and retaliation seriously and will promptly investigate such behaviors.