about 4 hours ago
Toronto, CanadaMid Level
H1B Sponsor
Responsibilities
- Evaluate the model's ability to respond to coding requests and workflows.
- Assess agent trajectories and model capabilities for code generation and debugging.
- Prompt models to complete complex coding tasks and review the accuracy of responses.
- Label, proofread, and improve software engineering-related outputs.
- Report quality and performance trends related to model behavior and project assignments.
Requirements
- 3-5 years of relevant industry experience in software engineering.
- Proficient knowledge of Python and familiarity with Java, JavaScript, Go, and SQL.
- Prior experience designing, building, and deploying APIs is preferred.
- Experience with code agents or evaluating agent trajectories is a plus.
Benefits
- Remote work within Canada.
- Part-time independent contractor position with flexible hours.
- Performance incentives included.