LLM / GenAI Engineer — Evlo AI, Chicago, IL

About The Role

The role builds the LLM-powered systems at the core of the product: retrieval pipelines, agentic workflows, and evaluation infrastructure that determine whether AI features actually work in production.

This is a hands-on engineering position on a small, senior team — expect to ship code daily, own systems end-to-end, and make architecture decisions with real production consequences.

Key Responsibilities

• Design and implement RAG pipelines end-to-end: chunking strategies, hybrid retrieval, reranking, and context optimization using LangChain, LlamaIndex, or custom architectures

• Build and scale vector database integrations (Pinecone, Weaviate, pgvector) handling millions of documents with sub-second retrieval latency

• Develop agentic workflows using tool-calling, function execution, and multi-step reasoning frameworks — deployed and monitored in production

• Own LLM evaluation infrastructure: golden datasets, LLM-as-judge pipelines, regression test suites, and automated quality gates in CI/CD

• Run parameter-efficient fine-tuning (LoRA, QLoRA) and structured output optimization on domain-specific datasets using PyTorch and Hugging Face

• Integrate LLM applications with backend services via FastAPI, streaming responses, and robust async patterns

• Instrument observability for GenAI systems: tracing (LangSmith/Langfuse), token cost tracking, latency dashboards, and hallucination detection alerts

What We Are Looking For

• 3–7 years of software engineering experience, including at least 2 years building LLM/GenAI systems in production

• Deep proficiency in Python with experience in async programming, API design, and production-grade software practices

• Hands-on experience with at least one vector database and embedding model stack in a real deployment

• Practical understanding of LLM internals: tokenization, context windows, prompt engineering tradeoffs, and inference optimization (vLLM, quantization)

• Experience deploying GenAI features on AWS, GCP, or Azure, including Docker-based containerization and CI/CD pipelines

• Bachelor's degree in Computer Science, Engineering, or equivalent practical experience

• Bonus: experience fine-tuning open-source models (Llama, Mistral, Qwen), prior work on agentic systems, contributions to open-source LLM tooling, or a relevant ML publication