The Conversational AI & Prompt Engineer will lead the development of intuitive, reliable interactions between users and an AI assistant. By refining prompt design, dialogue flows, and evaluation systems, you'll ensure responses are accurate, contextually appropriate, and aligned with user needs. Reporting to the Senior Director of Growth & AI, this position operates in a fully remote environment across the U.S.
Key Responsibilities
- Review conversation logs and user feedback to detect breakdowns, misinterpretations, and security risks in AI responses
- Partner with product leadership to define and monitor performance metrics such as resolution rate, containment, and satisfaction
- Refine prompt structures and model configurations to balance accuracy, speed, and cost in live environments
- Collaborate with engineering teams to enhance natural language understanding and intent classification
- Develop evaluation systems using golden datasets, automated scoring methods, and rubric-based analysis
- Implement safeguards to minimize hallucinations, prevent prompt injection, and maintain compliance
- Design end-to-end dialogue paths, including fallback strategies and handoffs to human support
- Continuously update system instructions for large language models to ensure brand-aligned tone and factual consistency
- Optimize structured output formats, function routing, and tool integration to support downstream processes
Qualifications
- Minimum of 5 years of professional experience, including at least 2 years focused on conversational AI, applied LLM systems, or NLP in production settings
- Proven expertise in crafting and tuning prompts for models like GPT, Gemini, or comparable systems, including structured outputs and function calling
- Hands-on experience with RAG pipeline design—chunking strategies, embedding models, and retrieval evaluation
- Experience building test frameworks for prompts, including A/B testing, regression checks, and LLM-as-judge setups
- Proficient in Python or TypeScript, with a track record of integrating LLM APIs into scalable applications
- Strong analytical skills to interpret conversation logs and drive measurable improvements
- Systems-oriented mindset with user empathy and the ability to convert business requirements into robust AI behaviors
Preferred Experience
- Work with agentic architectures or autonomous AI systems, such as those seen in Decagon, Agentforce, Fin, or Sierra
Technology Environment
Gemini, OpenAI, Python, TypeScript, LLM APIs, RAG systems, function calling, structured outputs, NLU, intent recognition, evaluation frameworks, LLM-as-judge, A/B testing, regression testing
Compensation & Benefits
This role offers competitive salaries, annual bonuses, and equity in the form of RSUs. Benefits include medical, dental, vision, disability, and life insurance, along with 401k matching, pet insurance, and an employee assistance program. The company provides 10 paid holidays and supports parental and military leave. Paid parking is available for in-office roles where applicable.
Work Environment
This is a fully remote position open to candidates across the United States, from Florida to Oregon and everywhere in between. The team emphasizes connection through regular All Hands meetings, virtual social events, hackathons, and learning sessions.
Company Values
The organization prioritizes empathy, inclusion, and user-centered design. Employees come from diverse backgrounds in race, ethnicity, gender, sexual orientation, age, and geography, and the culture actively supports those differences. The belief is that happy teams create better customer experiences, and inclusion strengthens both product and workplace.
Equal Opportunity
The company is committed to diversity, equity, and inclusion. All voices are valued regardless of race, ethnicity, gender, sexual orientation, age, location, or background. The organization thrives when differences are embraced, ensuring representation across teams and customer solutions.
