Applied AI Engineer Intern - Advancing Agent Quality
Brightstar.AI5 days ago
Miami, FL, USAIntern
H1B sponsor
Responsibilities
- Design and scale realistic agent environments and task suites for board deliberation, research synthesis, diligence, and multi-step tool use.
- Develop and calibrate LLM judges against human expert evaluations and benchmark agents against frontier models.
- Build trajectory-analysis and diagnostic tooling to identify agent failures and feed improvements into prompts, harnesses, and architecture.
- Enable agents to generate verifiers, automated test cases, checklists, and ground-truth sets.
- Harvest interaction trajectories into datasets and reward signals for model fine-tuning and post-training.
- Contribute to the AI Twin Board and extend its evaluation methodology to additional AI initiatives.
- Participate in Think Tank reviews, analyze current AI research, and explain technical developments to business and investment audiences.
Requirements
- Currently enrolled in a Masters or PhD program, or serving as a postdoctoral or early-career academic researcher, in AI, machine learning, computer science, computational linguistics, or a related field.
- At least 1 year of research, project, or professional experience with LLMs or agents.
- At least 2 years of experience with machine learning and deep learning.
- Experience with software engineering and cloud-based development.
- Experience with agent builders such as OpenSDK or OpenAI Agent Builder.
- Ability to communicate complex technical concepts clearly to non-technical executive audiences.
- Preferred qualifications include knowledge of supervised and reinforcement learning, hands-on agent evaluation systems, multi-agent orchestration, retrieval-augmented generation, tool-use reliability, and research or publications related to LLMs, reasoning systems, AI safety, or advanced AI impacts.
Benefits
- Full-time internship or opportunity for a recently graduated Masters or PhD researcher.
- External research affiliate and Think Tank contributor role conducted in coordination with university professors and research leaders.
- South Florida location is preferred.
- Work is performed under faculty direction or academic research supervision when applicable.
Categories
About Brightstar.AI
Brightstar.AI is a Miami-based boutique consulting firm that designs and implements AI, data, and automation programs for large enterprises and private-equity portfolio companies. It partners with executives to translate business challenges into applied machine-learning, data platform, and process-automation interventions, then supports execution and change management. Founded in 2025 and privately held, it focuses on technology-enabled value creation delivered through consulting engagements and transformation projects.