Position Overview
An organization is seeking a Software Developer in Test to support quality engineering for a core technology platform. This role will focus on backend, API, automation, AI/ML-enabled application testing, and CI/CD-integrated quality practices. The position requires a hands-on test engineer with experience improving automation frameworks, partnering with engineering teams, and supporting feature-based testing across distributed systems.
Mandatory Requirements:
- 10+ years of QA engineering delivery experience, with a strong focus on backend and API testing.
- 6+ years of experience delivering within Agile SDLC teams, ideally in CI/CD environments.
- Experience testing AI/ML or LLM-powered systems.
- Experience with model validation, prompt testing, AI behavior regression testing, and evaluation of non-deterministic outputs.
- Familiarity with AI testing methods and metrics, including precision, recall, drift detection, bias assessment, and offline versus online evaluation.
- Working knowledge of AI-enabled development tools or platforms, such as AI coding assistants, test generation tools, or internal AI platforms.
- Strong experience testing REST APIs, backend microservices, and distributed systems.
- Solid understanding of CI/CD tooling and pipelines, especially GitLab CI/CD.
- Experience with Python, Behave, PyTest, Postman or REST testing tools, JMeter, or similar performance testing tools.
- Experience with GitLab, Git, Jenkins, and Docker.
Key Responsibilities
- Design, build, and maintain automated testing suites for backend services and API layers.
- Develop testing approaches for AI/ML models and AI-enabled internal applications, including validation of model responses, prompt performance, repeatability, and release-over-release regression.
- Create automated evaluation frameworks for AI systems, covering accuracy, consistency, hallucination detection, bias, and model drift.
- Incorporate AI evaluation and automated testing processes into CI/CD pipelines.
- Work with AI/ML engineers, platform teams, and governance stakeholders to support responsible AI practices, including traceability, test coverage, and production readiness.
- Integrate automated test suites into GitLab CI/CD pipelines and deployment workflows.
- Enhance automation frameworks to support changing application architecture, including blockchain-related components where applicable.
- Collaborate with software engineers, product managers, and DevOps teams to promote quality and testability across distributed systems.
- Manage defect tracking workflows using tools such as JIRA, ensuring issues are clearly documented, communicated, and traceable.
- Define and report QA metrics related to test automation development, test execution, defect trends, and overall release quality.
- Build and support performance and load testing for critical backend services and smart contract interactions when relevant.
- Perform functional, API, automation, performance, and integration testing as needed.
- Contribute to ongoing improvements in QA processes, tools, standards, and Agile testing practices.
Required Qualifications
- 10+ years of experience in QA engineering delivery, with emphasis on backend systems or API testing.
- 6+ years of experience working within Agile SDLC teams, preferably with CI/CD delivery models.
- Experience testing AI/ML or LLM-based systems, including model output validation, prompt testing, regression testing, and non-deterministic response evaluation.
- Knowledge of AI testing metrics and evaluation techniques, including precision and recall, drift detection, bias assessment, and offline versus online evaluation methods.
- Understanding of AI-enabled software development tools and their impact on SDLC processes, testing practices, and quality governance.
- Strong hands-on experience with REST API testing, backend microservices, and distributed systems.
- Practical knowledge of CI/CD tools and pipeline integration, particularly GitLab CI/CD.
- Experience using Python, Behave, PyTest, Postman or comparable REST testing tools, JMeter, or similar performance testing platforms.
- Experience with GitLab, Git, Jenkins, and Docker.
- Ability to work closely with developers, product managers, DevOps engineers, AI/ML teams, and governance stakeholders.
- Strong understanding of test automation frameworks, defect management, and Agile QA delivery.
Preferred Qualifications
- Experience working with blockchain technologies, including smart contracts, distributed ledgers, or blockchain node interactions.
- AI integration experience.
- Exposure to banking, financial services, or other regulated environments.
- Prior experience enhancing existing automation frameworks.
- Experience supporting quality governance for AI-enabled applications.
Why Join This Team / Organization Summary
This role offers the opportunity to support quality engineering for complex backend, API, AI-enabled, and distributed technology systems. The position is suited for a senior QA automation professional who can combine hands-on test development, AI testing practices, CI/CD integration, and cross-functional collaboration.