8 hours ago
Remote, Spain +8 moreSenior
Responsibilities
- Review coding tasks derived from real GitHub issues and pull requests.
- Assess whether problem statements and success criteria are clear, complete, technically solvable, and representative.
- Evaluate unit tests for correctness, coverage, robustness, and appropriate implementation flexibility.
- Identify flaky tests, missing dependencies, version conflicts, ambiguous requirements, and environment-specific issues.
- Determine whether tasks can be reliably reproduced across environments and assess their real-world difficulty.
- Provide clear recommendations to accept, improve, or exclude tasks.
Requirements
- At least 3 years of professional software engineering experience.
- Strong experience working with large, multi-file codebases and maintaining production software.
- Experience reviewing pull requests and debugging complex issues.
- Strong understanding of unit testing, test coverage, test validation, and reproducibility.
- Experience with dependency management, environment setup, and Git/GitHub development workflows.
- Ability to analyze complex technical problems and provide clear written feedback.
- Preferred qualifications include open-source contributions, SWE-Bench experience, familiarity with major Python open-source projects, Docker, pip, conda, dependency pinning, test fixtures, test isolation, property-based testing, technical assessment design, and AI/ML evaluation.
Benefits
- Remote work arrangement.
- Part-time, project-based consulting engagement.