Senior Applied AI Engineer – Prompting & Evaluation

About Klipboard

Klipboard provides specialist software, services and support to deliver fully integrated trading and business management solutions to companies in the distributive trade worldwide. With expertise in ERP/SaaS solutions, the company serves wholesalers, distributors, merchants and retailers from small traders to multinational enterprises. Klipboard has offices in the UK, Ireland, The Netherlands, South Africa, Kenya and North America.

Our mission is to design and deliver high performance, integrated ERP solutions that enable our distributive trade customers to source effectively, stock efficiently, sell profitably and service competitively.

Klipboard is a global, growing business that embraces AI and emerging technologies to enhance customer outcomes, collaboration, and continuous improvement.

Work Environment

Klipboard operates a flexible hybrid work policy where employees spend three days in the office and two days working from home. This approach promotes a balanced work environment that combines office collaboration with the comfort and convenience of remote work.

The Role

This is a high-impact senior engineering role for someone who wants to shape how a large engineering organisation uses AI in real products and real development work. You will set patterns, create evaluation standards, coach engineers and help Klipboard turn AI from individual experimentation into repeatable, scalable engineering practice.

The role is two jobs in one: the craft of getting reliable, measurable results from large language models through prompt engineering, context design and evaluation; and making everyone around you better at it. With hundreds of engineers across the UK and South Africa, the difference between an organisation where a few people prompt well and one where everyone does is enormous. Your success is measured less by what you personally produce and more by how quickly the engineers around you improve.

Role Accountabilities

• Practise the craft at a high level – designing, building and refining prompts, context strategies and agentic workflows for AI features in Klipboard products.
• Grow the engineers around you – coaching, pairing, workshops and building shared assets that spread good practice across hundreds of engineers.
• Set standards and keep the organisation honest – defining what good looks like for prompting, evaluation and context engineering, and maintaining those standards as tools change.

Key Activities and Contributions

• Design, build and refine prompts, context strategies and agentic workflows for AI features in Klipboard products, where a confidently wrong price, part match or stock answer is worse than no answer.
• Build evaluation into everything – test prompts against real business cases at scale, measure quality honestly, and treat "it looked fine when I tried it" as the start of the work, not the end.
• Work across models and providers, choosing pragmatically on quality, cost and latency rather than loyalty, and keeping up as models, tools and providers change quickly.
• Get deep into the domain detail with product teams and subject matter experts, because a prompt is only as good as the understanding of the business problem behind it.
• Coach engineers across teams to get dramatically more from AI coding tools and LLMs, through pairing, reviews, workshops and patient one-to-one help.
• Build and maintain shared prompt patterns, reusable templates, evaluation harnesses and internal guides, so good practice spreads as assets rather than folklore.
• Run an internal community of practice where engineers share what is working, what failed and why, and make it somewhere people actually want to show up.
• Help define what good looks like for prompting, context engineering and evaluation across R&D, and keep those standards current as tools and models change.
• Share measurable results with engineering leadership, including where AI tooling is genuinely paying off and where it is not.

Core Responsibilities

• Deliver AI features in Klipboard products with proper evaluation and measurable quality standards.
• Embed prompt engineering and evaluation best practice across the R&D organisation.
• Be the sceptical voice when something demos well but will not hold up in production; be the encouraging one when a genuinely good idea needs backing.
• Maintain high standards for safety, accuracy and data handling in everything that ships – customers run their businesses on the answers our software gives them.
• Build and sustain a living library of prompt patterns, templates and guides.
• Meet engineers where they are, including those working in long-established codebases, where getting real value from AI tools is some of the highest-leverage work in the company.

Customer Experience

• Ensure AI features in Klipboard products give accurate, safe and trustworthy answers – customers run their businesses on our output.
• Work with product teams and subject matter experts to understand the business problem behind each feature properly before building.
• Prevent confidently wrong answers – incorrect prices, part matches or stock levels – from reaching customers.

Systems, Tools and Technology

• Large language model APIs across multiple providers (GPT, Claude, Llama and equivalents)
• AI coding tools: GitHub Copilot, Cursor or equivalents
• Prompt engineering, context design and agentic workflow frameworks
• Evaluation harnesses, evaluation datasets and automated test suites for AI outputs
• Retrieval-augmented generation (RAG), vector search, embeddings and retrieval pipelines (desirable)
• LLM orchestration frameworks (LangChain, Semantic Kernel or equivalents – desirable)

Required Qualifications and Experience

• Strong software engineering background with several years building and shipping production software – this role needs an engineer first.
• Substantial hands-on experience with large language models: prompt design, context engineering and structured outputs, in real work rather than personal experiments alone.
• Experience building evaluations or test harnesses for LLM outputs, and using them to improve quality measurably.
• Clear evidence of growing other engineers – through mentoring, teaching, internal training, communities of practice or similar.
• Daily working fluency with AI coding tools such as GitHub Copilot, Cursor or equivalents, with a practical view of where they help and where they mislead.

Preferred Qualifications and Experience

• Retrieval-augmented generation, agentic workflows, tool use or fine-tuning in production settings.
• Experience with LLM APIs, orchestration frameworks, vector search, embeddings, retrieval pipelines, evaluation datasets or automated test suites for AI outputs.
• Experience across more than one model provider, with a feel for their different strengths.
• Domain exposure to any of Klipboard's sectors: distributive trades, rental, retail, automotive aftermarket parts or garage management.
• Experience helping teams adopt AI tooling in established or legacy codebases, not just greenfield projects.
• Public sharing of your work: internal talks given externally, writing or open-source contributions.

People, Collaboration & Culture

• Generous with knowledge and genuinely energised by other people getting better. If the instinct is to guard expertise rather than spread it, this is the wrong role.
• Rigorous and honest about results – measures, shows working, and says so when data disagrees with the enthusiasm.
• Patient with engineers at every level of AI maturity, from those who live in these tools to those who are quietly sceptical of the whole thing.
• Curious about the trades Klipboard customers work in, because the best prompts are full of domain understanding.
• Calm about the pace of change – treats model and tooling shifts as the interesting part of the job, not a threat.
• Works closely with engineering leadership, architecture, product and cross-site engineering teams across the UK and South Africa.

Key Relationships

Internal

• Engineering leadership and architecture teams
• R&D engineering teams across Newcastle, Tankersley, Hungerford, Johannesburg and Cape Town / Stellenbosch
• Product managers and product owners
• Subject matter experts in Klipboard's product verticals (distributive trades, rental, automotive)

External

• AI and LLM platform and tooling providers (as needed)

Additional Responsibilities

• Some travel between Klipboard's UK and South Africa offices as part of the coaching and community-building remit.
• Keep standards current as models, tools and providers change rapidly.
• Represent Klipboard's engineering practice externally where appropriate – talks, writing or open-source contributions.

What Success in This Role Looks Like

• Engineers across several teams can point to specific things they now do better because of coaching and knowledge-sharing.
• At least one AI feature or workflow shipped with proper evaluation behind it, and the team involved could now repeat the process without you.
• A living, used library of prompt patterns, templates and guides that did not exist before.
• The community of practice is running, well attended, and producing things teams actually reuse.
• Engineering leadership has a clearer, evidence-based picture of where AI is paying off across the organisation.

Diversity and Inclusion

As a global company, Klipboard values and respects the diversity of its workforce, aiming to empower everyone to embrace each other's differences. The company is committed to creating an inclusive workplace where diversity, equity, and inclusion are integral to company and culture.

Research shows that women and other underrepresented groups are less likely to apply for a role unless they meet every listed requirement. However, Klipboard recognises that skills and experience come in many forms, and encourages you to apply even if you don't meet every criterion. If you are passionate about this role and believe you have the right mindset and transferrable skills, Klipboard would love to hear from you.

If you require any help, adjustments and/or support during the interview and offer process please advise the TA or HR team.

Note for Recruitment Agencies

Klipboard does not accept agency speculative resumes. At present the company only accepts CVs from Agencies on its PSL who have been assigned specific positions. Please do not forward resumes to the careers site or direct to Klipboard employees as this does not constitute an introduction and Klipboard retrospectively will not be liable for any candidate ownership or fees related to unsolicited resumes.