Dev.Pro is hiring a Senior LLM Systems Engineer to architect, implement, and optimize multi-node LangGraph pipelines for a civic intelligence startup building a data-backed Trust Score for elected officials. This role focuses on graph-based orchestration, data integration, evaluation pipelines, and latency optimization.
What You'll Do
- Build multi-node LangGraph workflows with dynamic routing, shared state, and conditional edges.
- Define and manage the graph schema with nodes as active processors and edges as routing logic.
- Implement LLM-driven intent classification to route user input across nodes or mark it out-of-scope.
- Manage state synchronization so nodes only update relevant data.
- Connect nodes to multiple external JSON-returning endpoints and normalize data structures.
- Design intermediate transformation layers to unify inconsistent API responses.
- Use mock endpoints and seamlessly transition them to live APIs without breaking architecture.
- Handle asynchronous and parallel API calls to minimize latency.
- Implement prompt and query-level guardrails to block unsafe inputs and prevent data leakage.
- Design state-aware routing logic that enforces isolation and privacy constraints between nodes.
- Build internal validators for payloads, ensuring all node inputs/outputs conform to strict schemas.
- Develop synthetic JSON-based ground-truth datasets to test system output deterministically.
- Build automated evaluation scripts to calculate F1, precision, recall, and exact-match scores.
- Compare multiple LLMs (Claude, GPT, etc.) by node and metric to determine best performers.
- Automate regression testing for new model versions or prompt updates.
- Use async execution and parallel node evaluation to reduce latency.
- Stream partial responses to improve perceived performance.
- Profile system components to locate and fix slow or sequential bottlenecks.
- Implement caching or smart pre-fetching for frequently used data sources.
- Build logging, tracing, and metric dashboards for every node and edge in the graph.
- Define error-handling strategies for malformed API responses or timeouts.
- Maintain test coverage across orchestration logic, node isolation, and evaluation functions.
- Implement CI/CD hooks to automatically re-evaluate the system before deployment.
What We're Looking For
- Advanced Python skills including async I/O, FastAPI, type hinting, logging, and pytest.
- Expert-level LangGraph orchestration knowledge of state machines, conditional edges, and node composition.
- Deep experience with LLM APIs (OpenAI, Anthropic, etc.) including structured prompting, JSON mode, and token usage optimization.
- Data Engineering background in JSON schema normalization, API integration, and validation layers.
- Proven ability to build Evaluation Systems with F1, precision/recall metrics, dataset design, and automated scoring.
- Expertise in Asynchronous & Parallel Processing using asyncio, concurrent futures, and non-blocking execution.
- Experience implementing Security / Guardrails such as prompt validation, regex filters, payload sanitization, and sandboxing.
- DevOps / Tooling proficiency with Docker, CI/CD, logging, and observability (Grafana, Prometheus, OpenTelemetry).
Nice to Have
- Experience with LangChain, LlamaIndex, or orchestration frameworks beyond LangGraph.
- Familiarity with vector databases (pgvector, Pinecone, Weaviate).
- Background in deterministic or safety-critical LLM applications (finance, legal, etc.).
- Comfort with multi-model evaluation pipelines and A/B testing at node level.
Technical Stack
- Python, FastAPI, LangGraph
- OpenAI, Anthropic
- Docker, Grafana, Prometheus, OpenTelemetry
Work Mode
This role supports a global work mode.
Dev.Pro operates on values of democracy, human rights, and state sovereignty, and offers challenging projects with world-leading clients, a modern technology stack, and career opportunities.
