Job Description
At NiCE, we don’t limit our challenges. We challenge our limits. Always. We’re ambitious. We’re game changers. And we play to win. We set the highest standards and execute beyond them. And if you’re like us, we can offer you the ultimate career opportunity that will light a fire within you.
So, what’s the role all about?
As AI Quality Architect, you are the person who makes quality native to the agentic SDLC. You design the systems, standards, and intelligence layers that ensure every stage of an AI-accelerated pipeline — from requirement ingestion to autonomous deployment — is observable, trustworthy, and continuously improving. You don’t retrofit testing onto AI workflows; you architect quality into them from the ground up.
How will you make an impact?
Agentic Quality Architecture
- Design the end-to-end quality architecture for agentic SDLC pipelines — spanning requirement analysis, code generation, test creation, execution, triage, and release gates
- Define how quality agents are orchestrated: which decisions they own autonomously, which require human-in-the-loop checkpoints, and how confidence thresholds govern both
- Architect multi-agent quality workflows: requirement validation agents, test generation agents, failure triage agents, and regression analysis agents working in coordinated pipelines
- Establish trust and verification models for agent-produced artifacts — test code, assertions, coverage reports, and defect analyses must all be auditable and traceable
- Own the architectural patterns for quality feedback loops between agents: how a deployment agent learns from a triage agent’s findings, and how that signal improves future generation
AI-Native Test Engineering Platform
- Design and own the LLM-powered test generation platform — from natural language requirement ingestion to executable, maintainable test output
- Architect the evaluation harness that continuously measures test generation quality: coverage delta, false-positive rates, assertion accuracy, and maintenance burden over time
- Build the self-healing test infrastructure layer — agents that detect broken selectors, drifted APIs, or changed behaviors and propose or apply fixes autonomously
- Define the prompt engineering standards, context injection patterns, and RAG architectures that ground test generation agents in real codebase context
- Architect test artifact governance: versioning, ownership attribution (human vs. agent), rollback capability, and confidence scoring for every generated artifact
Quality Gates in Autonomous Pipelines
- Design intelligent, adaptive quality gates that operate at the speed of agentic CI/CD — gates that reason about risk, not just pass/fail thresholds
- Build risk-scoring models that dynamically adjust gate strictness based on change scope, code origin (human vs. AI-generated), historical failure patterns, and deployment context
- Architect the observability layer for agentic pipelines: what signals indicate a pipeline agent is making poor quality decisions, and how are those signals surfaced in real time
- Define the integration patterns between quality gates and orchestration platforms (LangChain, LlamaIndex, custom agent frameworks) used across the engineering org
- Establish rollback and circuit-breaker patterns for autonomous deployments triggered by quality signal degradation
AI Model & Agent Validation
- Build behavioral testing frameworks for validating AI agents and LLM-powered features in production — testing non-deterministic outputs with statistical rigor
- Design evaluation benchmarks for internal AI tooling: measuring agent task completion accuracy, hallucination rates, context retention, and decision quality over time
- Architect drift detection systems that identify when agent behavior changes between model versions, prompt updates, or context window shifts
- Define adversarial and edge-case testing methodologies for AI features: prompt injection resistance, boundary condition handling, and graceful degradation under distribution shift
- Partner with ML platform and data science teams to establish quality acceptance criteria for every model and agent promoted to production
Have you got what it takes?
- 12+ years in software engineering with strong depth across both development and quality engineering
- 4+ years as a hands-on principal architect or distinguished engineer with cross-org technical scope
- Demonstrated experience designing quality infrastructure used at scale — 50+ engineers, high-velocity pipelines, enterprise SLAs
- Direct production experience building or operating systems that incorporate LLMs or AI agents — not evaluations, but shipped systems
- Background in large-scale CI/CD architecture and the performance engineering domain
- Enterprise SaaS or platform engineering background; familiarity with regulated, high-uptime environments strongly preferred
Agentic AI & LLM Proficiency
- Deep, hands-on understanding of agentic AI patterns: tool use, multi-agent orchestration, planning loops, memory architectures, and human-in-the-loop design
- Expert-level prompt engineering including chain-of-thought, few-shot, RAG, and structured output techniques applied to code and test generation
- Experience designing evaluation harnesses for non-deterministic AI systems — statistical confidence, behavioral consistency, and regression detection
- Familiarity with agent frameworks (LangChain, LlamaIndex, AutoGen, CrewAI, or equivalents) and their tradeoffs in production quality pipelines
Technical Depth
Expert in Python and TypeScript; proficient in at least one compiled language (Java, Go, C#)
Deep knowledge of modern test frameworks across UI, API, and contract layers: Playwright, Stryker, pytest, REST Assured, Pact
Strong CI/CD architecture skills: Jenkins, GitHub Actions, pipeline-as-code, artifact management
Proficient with cloud-native infrastructure: Docker, Kubernetes, AWS, serverless execution environments
Leadership & Influence
- Proven ability to drive technical consensus and architectural adoption across multiple teams without direct authority
- Exceptional written communication — your ADRs, RFCs, and design docs set the standard others follow
- Effective technical mentor to senior and staff engineers; you expand the thinking of those around you
- Comfortable and credible presenting to VP and C-level stakeholders; you make complex architecture legible to executives
What’s in it for you?
Join an ever-growing, market disrupting, global company where the teams – comprised of the best of the best – work in a fast-paced, collaborative, and creative environment! As the market leader, every day at NICE is a chance to learn and grow, and there are endless internal career opportunities across multiple roles, disciplines, domains, and locations. If you are passionate, innovative, and excited to constantly raise the bar, you may just be our next NiCEr!
Enjoy NiCE-FLEX!
At NiCE, we work according to the NiCE-FLEX hybrid model, which enables maximum flexibility: 2 days working from the office and 3 days of remote work, each week. Naturally, office days focus on face-to-face meetings, where teamwork and collaborative thinking generate innovation, new ideas, and a vibrant, interactive atmosphere.
Requisition ID: 10833 Reporting into: SeniorDirector, Engineering, CX
Role Type: Individual Contributor
#LI-Hybrid
About NiCE
NICE Ltd. (NASDAQ: NICE) software products are used by 25,000+ global businesses, including 85 of the Fortune 100 corporations, to deliver extraordinary customer experiences, fight financial crime and ensure public safety. Every day, NiCE software manages more than 120 million customer interactions and monitors 3+ billion financial transactions.
Known as an innovation powerhouse that excels in AI, cloud and digital, NiCE is consistently recognized as the market leader in its domains, with over 8,500 employees across 30+ countries.
NiCE is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, age, sex, marital status, ancestry, neurotype, physical or mental disability, veteran status, gender identity, sexual orientation or any other category protected by law.
