AI/ML Engineer (Remote) — Chapel Hill, NC

About the Role

Our client, a growing technology consultancy focused on helping banks build and deploy secure, production-grade AI solutions, is seeking a Lead AI/ML Engineer to build the intelligence layer of an AI platform designed specifically for the banking industry.

This is a hands-on engineering role focused on production LLM applications, RAG, retrieval, model serving, tool orchestration, and evaluation. The focus is on developing AI systems that deliver accurate, verifiable, and auditable answers in highly regulated environments—not simply building impressive demonstrations. You will work closely with data engineering, platform engineering, and product teams to develop secure AI systems for banking environments, including self-hosted and on-premises deployments.

Key Responsibilities

• Design and build production-grade RAG and retrieval systems that provide verifiable answers supported by queries, sources, and citations.

• Develop retrieval and verification workflows across structured and unstructured banking data.

• Deploy and operate self-hosted open-weight LLMs and embedding models, including within individual bank environments.

• Build and evolve standardized RAG architectures and engineering patterns.

• Design and implement an MCP and tooling layer that enables AI systems to securely interact with governed data, metrics, documents, and customer information.

• Build tool and function-calling workflows with strong auditing, access controls, and clearly defined boundaries around AI system access.

• Develop and own an AI evaluation framework covering golden-question regression testing, retrieval quality, groundedness, answer accuracy, and appropriate "I don't know" behavior.

• Establish evaluation as a formal release gate for production AI systems.

• Implement LLM guardrails, including PII redaction, prompt-injection defenses, context controls, and query and row limits.

• Partner with data engineering to ensure AI systems use governed semantic-layer metrics instead of generating unreliable ad hoc SQL.

• Work with platform and security engineering teams to deploy reliable AI infrastructure in controlled environments.

• Monitor and optimize latency, inference costs, retrieval quality, and model performance across different models and deployment configurations.

• Document model architecture, evaluation methodology, security controls, and guardrails for customer security reviews and audits.

• Provide technical leadership while remaining hands-on with implementation and production troubleshooting.

Required Qualifications

• 6+ years of software or ML engineering experience, including at least 2–3 years building and shipping production LLM, RAG, NLP, or generative AI systems.

• Proven experience taking AI systems from prototype to production and delivering solutions used by real users.

• Strong Python skills and solid software engineering fundamentals.

• Hands-on experience building production RAG and retrieval pipelines.

• Strong understanding of LLM evaluation, including groundedness, retrieval quality, regression testing, and hallucination reduction.

• Experience implementing AI guardrails and security controls.

• Experience deploying or operating self-hosted and open-weight models.

• Ability to evaluate trade-offs between model quality, latency, infrastructure, and cost.

• Strong understanding of prompt engineering, tool calling, structured outputs, and LLM application architecture.

• An engineering mindset focused on accuracy and verifiability rather than simply building compelling AI demonstrations.

• Ability to work cross-functionally with data, platform, security, and product teams.

Technology Environment

• Languages: Python

• Model Serving: vLLM, Ollama

• Infrastructure: NVIDIA GPUs, CUDA, NVIDIA AI Enterprise

• RAG and Retrieval: LlamaIndex, Haystack, Qdrant, pgvector

• Embeddings: Modern embedding models and vector search

• AI Orchestration: MCP, tool/function calling

• Structured AI: Text-to-SQL, semantic layers, Cube, dbt MetricFlow

• Evaluation: LLM evaluation frameworks, groundedness testing, retrieval evaluation

• AI Security: PII redaction, prompt-injection defense, access controls

Experience with every technology listed is not required. The team is particularly interested in engineers who understand the architecture and engineering principles behind production AI systems and can quickly adapt to new tools.

Why This Role?

• Build AI systems that move beyond prototypes and are deployed for use by banking professionals.

• Take ownership of AI architecture, with meaningful influence over RAG, model serving, evaluation, tooling, and guardrails.

• Work with self-hosted AI infrastructure in environments where security, privacy, and deployment constraints make traditional cloud-only approaches impractical.

• Solve complex reliability challenges by making AI systems accurate, grounded, observable, and resistant to failure.

• Work at the intersection of LLM engineering, structured data, semantic layers, and intelligent retrieval.

• Address high-impact challenges in banking, where incorrect answers can have significant business and regulatory consequences.

• Establish engineering standards and best practices for production AI systems while remaining hands-on.

Preferred Qualifications

• Experience building AI systems in banking, financial services, healthcare, cybersecurity, or another high-stakes or regulated environment.

• Experience deploying AI systems in on-premises or highly controlled environments.

• Experience with fine-tuning, adapters, or retrieval-quality optimization.

• Familiarity with banking, financial services, or financial data.

• Experience with AI security, governance, or compliance frameworks.

• Experience working with GPUs, CUDA, and production inference infrastructure.

Ideal Candidate Profile

This role is suited to an engineer who has progressed beyond LLM experimentation and has experience operating AI systems in production.

Relevant backgrounds may include Lead AI Engineer, Senior ML Engineer, LLM Engineer, Applied AI Engineer, or Staff AI Engineer, with hands-on experience building RAG systems, deploying models, implementing evaluation frameworks, and addressing the practical challenges of production AI.

The strongest candidates are focused on one central question: "How do we know the AI is actually right?"

If you are interested in building reliable AI systems where accuracy, security, and verifiability matter as much as model performance, this role offers the opportunity to contribute to that work.