LLM / GenAI Engineer

About The Role

The role sits at the core of an AI product team building production LLM systems - retrieval-augmented generation pipelines, agentic workflows, and fine-tuned models deployed to enterprise-scale workloads. This is hands-on engineering: latency budgets, cost-per-token, hallucination mitigation, and evaluation harnesses that catch regressions before clients do.

The impact is direct. GenAI features shipped by this team are customer-facing and revenue-linked, and the role works alongside backend engineers, applied scientists, and product leadership to take LLM prototypes from demo to dependable production systems.

Key Responsibilities

• Design, build, and maintain production RAG pipelines - chunking strategies, hybrid retrieval (dense + BM25), reranking, and context assembly - using LangChain, LlamaIndex, or custom architectures
• Integrate and tune vector databases (Pinecone, Weaviate, pgvector) for semantic search at scale, including embedding model selection and reindexing strategies
• Build agentic workflows with tool calling, function orchestration, and guardrails using frameworks like LangGraph or OpenAI/Anthropic tool APIs
• Develop systematic LLM evaluation frameworks: golden datasets, LLM-as-judge pipelines, regression suites, and hallucination/faithfulness metrics
• Run parameter-efficient fine-tuning (LoRA, QLoRA) and prompt optimization on domain-specific datasets; manage training runs on GPU infrastructure
• Optimize inference cost and latency through quantization, caching layers, batching, and model routing across OpenAI, Anthropic, and open-weight models (vLLM, Llama)
• Instrument production LLM systems with tracing, observability, and alerting (LangSmith, OpenTelemetry, custom dashboards); own on-call quality for AI services

What We Are Looking For

• 3–6 years of software engineering experience, with at least 2 years building and shipping LLM/GenAI features to production
• Strong Python engineering skills: async programming, API design (FastAPI or equivalent), testing, and clean service architecture
• Hands-on production experience with RAG systems and vector databases - not just prototypes
• Working knowledge of transformer architecture, tokenization, embeddings, and the practical tradeoffs of open-weight vs. API-hosted models
• Experience deploying and operating services in AWS, GCP, or Azure, including Docker, CI/CD, and GPU-based inference
• Bachelor's degree in Computer Science, Engineering, or related quantitative field, or equivalent practical experience
• Bonus: experience fine-tuning models (LoRA/QLoRA) with Hugging Face + PyTorch; contributions to open-source LLM tooling; MLOps experience (Kubernetes, Ray, MLflow); SQL and data pipeline experience

Show more

Show less

Job industries: Software Development