A leading company is seeking a skilled Prompt Engineer (LLM Systems, Evals & Safety) to join its team. This role focuses on designing high-quality prompts, system instructions, and evaluation frameworks to ensure large language model (LLM) features are accurate, safe, and cost-effective, while driving continuous improvement and performance optimization.

Key Responsibilities of the Prompt Engineer Role

• Author, refactor, and chain prompts (system/tool/policy) for varied tasks
• Create offline/online evaluation harnesses (rubrics, golden sets, metrics)
• Build prompt libraries with versioning, A/B testing, and telemetry
• Reduce hallucinations via verification, constrained decoding, and tool use
• Implement safety: jailbreak/prompt-injection tests, content policy checks, PII handling
• Partner with engineers to integrate prompts into production features

Requirements of the Prompt Engineer Role

• Demonstrated prompt design across multiple task types and models
• Experience building eval datasets and automated scoring (e.g., accuracy, faithfulness, utility, cost/latency)
• Familiarity with retrieval-augmented generation concepts and tool/function calling
• Strong scripting (Python/TypeScript) for data prep, evals, and analysis
• Clear writing; ability to translate business goals into measurable prompt specs

Nice to Haves

• Experience with LangChain/LLM orchestration, vector stores, and rerankers
• Knowledge of safety tooling and red-teaming techniques
• Experiment platforms (feature flags, A/B tests), analytics

We refresh listings regularly, but some roles close early on the source platform.