We are looking for a hands-on QA Automation & AI Lead to build the next generation of AI-driven quality engineering for our web applications.
This is not a traditional QA Lead role. We are looking for an engineer who deeply understands QA, is strong in automation, and has practical experience building and testing LLM-powered products.
What You’ll Do
Build AI-driven frontend test automation, primarily using Playwright + LLMs, to test real user journeys.
Design frameworks where AI can help generate test cases, execute tests, validate outcomes, and analyze failures.
Build evaluation frameworks for LLM products, including prompt evaluation, hallucination testing, regression testing, and output quality.
Make intelligent decisions about which model to use where, balancing accuracy, latency, context size, and cost.
Optimize token consumption for frontend automation instead of sending entire pages/context to expensive models unnecessarily.
Identify where to use deterministic automation vs. LLM-based automation.
Explore AI-assisted approaches to make manual and exploratory testing faster and more effective.
Remain highly hands-on while providing technical direction and guiding engineers implementing the automation platform.
What We’re Looking For
Strong software engineering and test automation experience.
Hands-on experience with LLMs and AI-powered applications.
Strong experience with Playwright or similar browser automation frameworks.
Experience with prompt/LLM evaluation, hallucination testing, and non-deterministic systems.
Good understanding of token optimization, context management, model selection, latency, and LLM costs.
Strong understanding of QA principles, test strategy, regression testing, and edge cases.
Ability to prototype quickly and turn experiments into scalable automation frameworks.
Strong technical leadership skills while remaining an individual contributor.
What Success Looks Like
You will help us move from traditional QA toward AI-native quality engineering—where AI assists in creating and validating test scenarios, frontend journeys are intelligently automated, LLM products are continuously evaluated, and model/token usage is optimized for both quality and cost.
We’re looking for someone who thinks beyond “How do I automate this test?” and asks “What is the smartest combination of deterministic automation and AI to test this product?”
Gross salary 14,000 - 21,000 PLN
