Compensation: Up to $250,000 + Equity + Benefits
Location: Mountain View, CA or Waterloo, ON (Hybrid)
DeepRec.ai have been tasked to hire an experienced Data Scientist to lead the development of systems that support model evaluation, ranking infrastructure, and data-informed product features. This is a senior-level position suited to someone who combines deep technical capability with a strong understanding of product and user experience.
You’ll be responsible for designing experiments, building evaluation frameworks, and extracting insights from behavioural signals and system-level data. The work will involve close collaboration with engineering, product, and research teams, with real impact on product direction and model quality.
Responsibilities:
- Design and run experiments to measure model performance using user interaction and human feedback.
- Develop robust methodologies for sampling, scoring, bias correction, and metric definition.
- Collaborate with engineering to build scalable data pipelines and deploy evaluation logic.
- Investigate product usage data to uncover insights that inform product and research roadmaps.
- Build dashboards, tracking tools, and diagnostics to monitor core business and performance metrics.
- Contribute to system design through code contributions, schema proposals, and infrastructure decisions.
- 10+ years of experience in data science, applied machine learning, or related technical roles.
- Deep understanding of experimental design, causal inference, and applied statistics.
- Proficient in Python and key scientific libraries (NumPy, pandas, scikit-learn, statsmodels).
- Experience working on ranking systems, embedding-based models, or preference learning techniques.
- Familiarity with data warehousing and query tools (e.g. BigQuery, SQL).
- Ability to communicate technical ideas clearly and translate them into product-relevant insights.
- Previous work on evaluation frameworks for LLMs or RLHF pipelines.
- Exposure to consumer-scale AI systems, recommender products, or human-in-the-loop workflows.
- Knowledge of experimentation platforms or A/B testing infrastructure.
