AI Software Engineer in Test
GreenSky Administrative Services LLC10 days ago
Alpharetta, GA, USAMid Level
Base Salary
$115k - $145k/yr
Responsibilities
- Use AI tools to generate, modify, refactor, debug, and maintain automated tests and repeatable quality-engineering workflows.
- Design, develop, and maintain scalable automated test frameworks for web, mobile, backend, API, data, and AI-enabled applications.
- Create functional, regression, integration, end-to-end, API, performance, security, and data-validation test suites.
- Develop validation strategies, evaluation datasets, prompt test suites, golden answer sets, and scoring rubrics for generative AI, LLM, RAG, conversational AI, and agentic systems.
- Validate AI outputs for accuracy, relevance, completeness, hallucination risk, toxicity, policy alignment, bias, fairness, explainability, traceability, and business-rule compliance.
- Integrate automated tests into CI/CD pipelines and establish quality gates for build validation, coverage, security checks, and release readiness.
- Collaborate with engineering, QA, product, architecture, security, data science, DevOps, and platform teams on test strategy, quality standards, and responsible AI practices.
- Translate requirements and acceptance criteria into test plans, automation scenarios, validation criteria, and traceability across defects and releases.
- Advocate for clean, maintainable, observable, testable, and secure code through reviews and earlier lifecycle testability practices.
Requirements
- Bachelor’s or master’s degree in Computer Science, Software Engineering, Information Systems, Data Science, Artificial Intelligence, or a related technical discipline.
- At least 2 years of hands-on experience in software development, test automation, or quality engineering.
- Experience building and maintaining automated test frameworks.
- Experience with AI development lifecycle or delivery models, including Scrum and Kanban.
- Experience working in fast-paced delivery environments with short turnaround times.
- 2 to 4+ years of database experience, preferably with SQL.
- Experience with AI-assisted coding tools such as Claude Code, Kiro, Copilot, or similar tools is preferred.
- Strong software engineering, quality engineering, data validation, and practical AI-tool adoption skills.
Benefits
- Hybrid work in Alpharetta, Georgia, with 3 days per week onsite.
- Base salary ranges from $115,000 to $145,000 per year, with eligibility for an annual bonus.
- Medical, dental, vision, disability, and life insurance.
- 401(k) retirement benefits.
- Paid time off, paid holidays, and paid personal/sick time.