LLM / GenAI Engineer — New York, NY

About The Role

The role builds and ships production LLM systems that handle real user traffic: retrieval-augmented generation pipelines, agentic workflows, fine-tuning jobs, and evaluation infrastructure. This is hands-on engineering work, not prompt tinkering — latency budgets, cost controls, and hallucination rates are the metrics that matter.

The team sits at the center of the product, working directly with applied scientists, backend engineers, and product leadership to turn frontier model capabilities into reliable, revenue-generating features. Decisions made here shape how the entire organization uses 27.

Key Responsibilities

• Design, build, and maintain production RAG pipelines using LangChain, LlamaIndex, or custom retrieval architectures with hybrid search and re-ranking

• Optimize vector database performance (Pinecone, Weaviate, pgvector) for low-latency semantic retrieval across large-scale document corpora

• Build agentic systems with tool use, multi-step reasoning, and guardrails — including orchestration, state management, and failure recovery

• Develop and run systematic evaluation frameworks: golden datasets, LLM-as-judge pipelines, regression suites, and online A/B tests tied to product metrics

• Fine-tune open-weight and frontier models using LoRA/QLoRA and supervised instruction tuning on domain-specific datasets

• Instrument LLM applications for observability — tracing, token cost tracking, drift detection — and drive improvements in quality, latency, and cost per request

• Contribute to architecture reviews and establish engineering standards for shipping 27 features safely to production

What We Are Looking For

• 3–7 years of software engineering experience, including 2+ years building LLM/GenAI systems in production

• Deep hands-on experience with at least one LLM orchestration framework (LangChain, LlamaIndex, Haystack) and the judgment to know when not to use one

• Production experience with vector databases, embedding models, and semantic search at scale

• Strong Python engineering skills: async programming, API design, testing, and cloud infrastructure (AWS, GCP, or Azure)

• Experience fine-tuning or evaluating LLMs (LoRA/QLoRA, instruction tuning, RLHF concepts) and a rigorous approach to measuring model quality

• MS or PhD in Computer Science, ML, or related field, or equivalent industry experience

• Bonus: experience serving models with vLLM or TensorRT-LLM, GPU cost optimization, contributions to open-source LLM tooling, or building eval infrastructure for agentic systems