Klu: Complete Guide & Tutorial

Introduction

Building reliable applications with Large Language Models (LLMs) is a complex challenge. Teams often struggle with managing prompt versions, evaluating outputs consistently, and maintaining visibility across different models. Klu is a platform designed specifically to solve these problems. It provides a unified workspace where product managers, engineers, and data scientists can collaborate on designing, deploying, and optimizing LLM-powered features.

Klu is not just another prompt editor. It is a complete operational platform that brings structure to the often chaotic process of working with generative AI. With built-in version control for prompts, automated evaluation workflows, and real-time observability dashboards, Klu helps teams move from experimental prototypes to production-grade AI experiences. The platform also prioritizes enterprise security, offering private infrastructure options and governance controls that meet compliance requirements.

This tutorial will guide you through everything you need to know to start using Klu effectively. Whether you are a developer looking to iterate faster or a product manager wanting to align stakeholders on quality, this guide will help you understand the core features and practical workflows.

Getting Started

Creating Your Account and Workspace

To begin using Klu, navigate to https://klu.ai/ and click the “Get Started” or “Sign Up” button. You can register using your work email or a Google account. After verifying your email, you will be prompted to create your first workspace. A workspace in Klu represents your team or project environment. Name it something descriptive, such as “Customer Support AI” or “Content Generation Pipeline.”

Understanding the Dashboard

Once you log in, you will see the main dashboard. This is your command center. The left sidebar contains the primary navigation sections:

• Prompts: Where you design, version, and manage your prompt templates.
• Experiments: For running real-time tests and comparing different configurations.
• Evaluations: To create and review automated evaluation workflows.
• Observability: Dashboards showing performance, cost, and drift tracking across models.
• Settings: For managing team members, API keys, and governance controls.

Take a few minutes to click through each section. The interface is clean and intuitive, with tooltips explaining each element.

Inviting Team Members

Klu is built for collaboration. Go to Settings > Team and click “Invite Member.” Enter their email address and assign a role (Admin, Editor, or Viewer). This allows your team to work on the same prompts, review evaluations, and share dashboards without stepping on each other’s work.

Key Features

Collaborative Prompt Design with Versioning

This is the heart of Klu. Instead of saving prompts in scattered documents or chat threads, Klu stores every prompt as a versioned entity. Each change is tracked, and you can roll back to any previous version instantly. Team members can leave comments on specific prompt versions, propose edits, and approve changes. This eliminates the confusion of “which prompt is currently in production?”

Built-in Evaluation Workflows

Evaluating LLM outputs manually is slow and inconsistent. Klu allows you to define evaluation criteria, such as accuracy, tone, or safety. You can create automated tests that run every time a prompt is updated. The platform supports both human review (where team members score outputs) and automated checks (using rule-based or AI-assisted evaluators). Results are aggregated into clear reports.

Observability Across Models and Apps

Klu connects to multiple LLM providers (OpenAI, Anthropic, Google, open-source models, etc.) and tracks every API call. You can see latency, token usage, cost per request, and error rates in real-time. The observability dashboards also detect data drift—when the model’s behavior changes over time due to updates or shifts in input patterns.

Performance, Cost, and Drift Tracking

One of the biggest challenges in production AI is controlling costs. Klu provides granular cost tracking per model, per prompt, and even per user. You can set budget alerts. Drift tracking monitors if the model’s outputs start deviating from expected patterns, which is critical for maintaining reliability in customer-facing applications.

Private Infrastructure and Governance Controls

For enterprises with strict data privacy requirements, Klu offers private deployment options. Your data never leaves your infrastructure. Governance controls allow you to set approval workflows (e.g., a prompt cannot go to production without a manager’s sign-off) and audit logs for compliance.

Real-time Experiments and Dashboards

You can run A/B tests on prompts, model parameters, or even different LLMs side-by-side. The experiment dashboard shows live results, allowing you to make data-driven decisions about which configuration performs best for your specific use case.

How to Use

Step 1: Creating Your First Prompt

Navigate to the Prompts section and click “New Prompt.” You will see a text editor with a sidebar for configuration. Give your prompt a name, such as “Customer Reply Generator.”

In the editor, write your prompt template. Klu supports variables using double curly braces. For example:

Prompt: “You are a helpful customer support agent. Reply to the following customer query in a friendly and professional tone. Customer query: {{user_query}} Keep the response under 100 words.”

On the right sidebar, you can define the variables. Add a variable called “user_query” and set its type to “string.” You can also set default values for testing.

Step 2: Versioning and Collaboration

After writing your first version, click “Save Version.” Klu will automatically assign it version 1. Now, make a small change—perhaps adjust the tone from “friendly” to “empathetic.” Save again as version 2. You can now see both versions in the version history panel. Team members can click “Add Comment” on any version to suggest improvements. Use the “Compare” button to see a diff between two versions.

Step 3: Running an Experiment

Go to the Experiments section and click “New Experiment.” Select the prompt you just created. You can now test different versions against each other. For example, set version 1 as the control and version 2 as the variant. Choose a model (e.g., GPT-4 or Claude) and define how many test runs you want (e.g., 10 queries per version).

Klu will run the experiment and show you a side-by-side comparison of outputs, latency, and cost. You can manually rate each output or use automated evaluators. Based on the results, you can promote a version to production.

Step 4: Setting Up Evaluation Workflows

In the Evaluations section, click “New Evaluation.” You have two options: Human Evaluation or Automated Evaluation.

For automated evaluation, define criteria. For example, create a rule that checks if the response contains any profanity (using a blocklist) or if it exceeds the word limit. Klu also offers AI-assisted evaluation where you can ask another LLM to rate the output on a scale of 1-5 for criteria like “helpfulness.”

For human evaluation, Klu will distribute samples to your team members. Each person scores the outputs, and Klu aggregates the scores into a report. You can set a threshold (e.g., average score of 4.0) that a prompt must pass before it can be deployed.

Step 5: Deploying and Monitoring

Once you are satisfied with a prompt version, click “Deploy.” Klu provides an API endpoint that your application can call. In the Observability dashboard, you can see every request made to that endpoint. Monitor latency, cost, and error rates. Set up alerts for when cost exceeds a daily budget or when error rates spike above 5%.

To track drift, enable the “Drift Detection” feature. Klu will analyze the distribution of inputs and outputs over time. If the model starts producing significantly different responses for similar inputs, Klu will flag it. This is invaluable for catching issues before they affect users.

Step 6: Managing Governance

For production-critical applications, go to Settings > Governance. You can enforce that only specific team roles (e.g., “Prompt Manager”) can deploy prompts to production. Enable “Approval Required” so that any deployment request sends a notification to a designated approver. All changes are logged in the audit trail.

Tips

Start with Simple Prompts

Do not try to build the perfect prompt on the first attempt. Klu’s versioning makes it easy to iterate. Start with a basic instruction, test it, and gradually add constraints, examples, and formatting guidelines. Each iteration should have a clear goal (e.g., reduce verbosity, improve accuracy).

Leverage Variables for Reusability

Variables are powerful. Instead of hardcoding context or instructions, use variables for things like user input, tone, language, or output format. This allows you to use the same prompt template for multiple scenarios. For example, a “Summarization” prompt can have variables for article_text, target_language, and max_words.

Use Experiments Before Deploying

Never deploy a prompt change without running an experiment first. Even small changes in wording can dramatically alter outputs. Experiments give you quantitative evidence that the new version is better (or at least not worse) than the current one. Always include a diverse set of test inputs that reflect real-world usage.

Set Up Automated Evaluations Early

Manual review does not scale. As soon as you have a basic prompt, create an automated evaluation for it. Even simple checks (e.g., “response must contain a greeting,” “response must not mention competitors”) catch many issues. Over time, you can add more sophisticated evaluators. This builds a safety net that prevents bad prompts from reaching users.

Monitor Drift Regularly

LLMs change over time. A model that performed perfectly last month may start giving inconsistent results today due to an upstream update or subtle shifts in user behavior. Check the drift dashboard weekly. If you see a drift alert, run an experiment to see if your prompt needs adjustment. Do not assume your prompt is stable forever.

Collaborate in the Platform, Not in Email

Resist the temptation to share prompt ideas via email or chat. Klu’s comment and versioning features are designed to keep all context in one place. When a team member leaves a comment on a specific version, that feedback is permanently attached to that version. This creates a clear history of why decisions were made.

Optimize for Cost

LLM costs can spiral quickly. Use Klu’s cost tracking to identify which prompts or models are consuming the most tokens. Consider using smaller, cheaper models for simpler tasks (e.g., classification) and reserving expensive models (like GPT-4) for complex reasoning. Experiment with different temperature and max token settings to find the sweet spot between quality and cost.

Set Up Alerts for Production

In the Observability section, configure alerts for critical metrics: high latency, increased error rates, budget thresholds, and drift detection. Klu can send notifications via email, Slack, or webhook. This ensures you are aware of issues before they escalate into user-facing problems.

Document Your Prompt Strategy

Use the description field in each prompt to document the intent, expected behavior, and known limitations. This is especially helpful for new team members or when revisiting a prompt months later. Good documentation reduces onboarding time and prevents repeated mistakes.

Klu is a powerful ally in the journey from AI prototype to production-grade application. By following this tutorial and applying these tips, you will be able to design better prompts, evaluate them rigorously, and monitor them continuously. The platform abstracts away much of the operational complexity, allowing your team to focus on what matters: delivering reliable, high-quality AI experiences to your users.