Forward Deployment Engineer — San Francisco Bay Area

Job Title: AI – Sr. Application Engineer / Forward Deployment Engineer

Location: Bay area, CA

Duration: FTE

Range : $200k to $225k Per Annum

• Will work on the intelligence layer for multiple programs — owns all model quality, RAG accuracy, prompt engineering, and AI safety across applications

• Socratic tutor persona, adaptive learning recommendation engine, multi-modal AI (text and voice), RAG evaluation framework, and feedback loop into retrieval

• 6-LLM call chain orchestration (NeMoGuardrails → intent classification → query rewriting → RAG → synthesis), , and compatibility check logic

• Production-grade AI quality from launch — this is not a research or prototyping role; accuracy thresholds, latency requirements, and safety guardrails must pass InfoSec adversarial testing before Release 1

Required Skills

Experience

• 10+ years of relevant professional experience in software engineering, application development, or AI/ML engineering.

• 4+ years of software engineering with at least 2 years focused on LLM application development in production. Demonstrated experience delivering and maintaining LLM-powered applications in production environments, with measurable user adoption, accuracy, and quality metrics.

• Has shipped an LLM-powered feature or product to production where real users depend on the accuracy and the engineer owns the quality metrics

• Has owned an AI safety or guardrails implementation for a customer-facing product — not just added an off-the-shelf filter; designed and tested the safety layer

• Has built RAG evaluation pipelines and used them to make go/no-go release decisions — accuracy gating is part of the workflow.

• Has profiled and optimized a multi-step LLM call chain for latency

Must Have

LLM Application Development

• LLM prompt engineering — system prompts, few-shot examples, chain-of-thought, instruction following · Expert

• Multi-step LLM chain orchestration — LangChain, LlamaIndex, or custom orchestration · Expert ·

• Multi-turn conversation design — context window management, conversation summarization, session memory · Advanced ·

• Streaming LLM response handling — token-by-token streaming, partial response rendering · Advanced ·

• Model selection and benchmarking — matching model size to task; balancing latency, cost, and accuracy · Advanced ·

RAG Pipeline Design & Quality

• RAG pipeline design — chunking strategy, embedding model selection, retrieval configuration · Expert ·

• Vector similarity search tuning — index parameters, similarity thresholds, retrieval depth · Advanced ·

• Reranking — cross-encoder rerankers, relevance scoring · Advanced ·

• RAG evaluation frameworks — RAGAS, TruLens, or equivalent; automated eval pipelines · Advanced ·

• Hybrid search — combining dense vector retrieval with BM25 or keyword search · Proficient ·

AI Safety & Guardrails

• Prompt injection detection and mitigation · Advanced ·

• Jailbreak testing and red-teaming LLM systems · Advanced

• Content safety classifier integration · Advanced

• Hallucination detection and mitigation strategies · Advanced

• Topical control — enforcing scope boundaries on LLM responses · Advanced ·

Evaluation & Production Quality

• Automated evaluation pipeline design — test set curation, metric selection, regression detection · Advanced ·

• A/B evaluation methodology for prompt and model changes · Proficient

• Latency profiling for LLM call chains — identifying bottlenecks across multi-step pipelines · Proficient ·

• Feedback loop design — user signal collection, signal-to-retrieval-weight integration · Proficient ·

• Production model monitoring — accuracy drift detection, quality degradation alerting · Proficient ·

Development

• Python — ML/AI application development, async programming · Expert

• API design for AI services — streaming endpoints, error handling, timeout management · Advanced

• Embedding model operations — model selection, batch embedding, index updates · Advanced ·

Nice to Have

• Adaptive learning systems or personalization engine experience

• Knowledge graph integration with RAG

• Multi-agent orchestration patterns

• ServiceNow API integration

• Prior experience building AI products on NVIDIA infrastructure