We are launching a major architectural upgrade of our B2B cloud management SaaS and want to avoid building a bloated manual QA department. Instead, we are looking for an Autonomous QA Systems Architect to engineer an AI-first quality pipeline where LLM agents handle test creation, execution, and maintenance, leaving human engineers in a supervisory role.
What You Will Build
Architect a clean, modular test automation framework (Playwright with Python or TypeScript) covering UI, APIs, and service integrations, paired with machine-readable test documentation.
Wire the test suite into Azure Pipelines with automated release gates. Provision isolated test environments and host agent workers using Azure Container Apps or Azure Functions.
Build LLM-driven workflows (via Azure OpenAI or similar) integrated directly with Azure Boards and Azure Repos. When a developer updates a ticket or opens a PR, your agents should propose test cases, generate scripts, triage failures, and submit self-healing fixes via Pull Requests.
Implement strict human-in-the-loop approval policies, secret management, and telemetry via Application Insights so AI-driven changes remain predictable and auditable. Hand over a documented, production-ready autonomous QA setup along with prompt templates and runbooks for our core dev team.
Your Toolkit
Deep expertise in building scalable test automation from zero (Playwright/Cypress, Python/TS, REST APIs).
Proven hands-on work with LLMs in software engineering (agent orchestration, RAG/repo-context injection, automated test generation).
Strong DevOps mindset within the Microsoft Azure ecosystem (ADO Pipelines, Repos, Boards, serverless/containers, IAM, and cloud monitoring).
