Guide: Handling Large Payloads

Intentgine enforces input limits to keep classification and resolution fast and accurate. If your input exceeds these limits, you’ll need to reduce it before sending. This guide covers practical patterns for doing that.

Where is the intent?

In most real-world text, the intent lives in a predictable place — the first or last sentence of an email, or near markers like “Action Required”, “In Summary”, “TLDR”, or “TL;DR”. You rarely need to send the full document.

1. Extract the Intent-Bearing Text

Before reaching for summarisation or chunking, try extracting the part of the text that actually carries the intent. For emails and documents, this is usually:

• First sentence — “I’d like to request a refund for order #1234”
• Last sentence — “Please cancel my subscription”
• Near markers — text following “Action Required”, “In Summary”, “TLDR”, “TL;DR”, “Next Steps”

function extractIntent(text: string): string {
  const markers = ['action required', 'in summary', 'tldr', 'tl;dr', 'next steps'];
  const lower = text.toLowerCase();

  for (const marker of markers) {
    const idx = lower.indexOf(marker);
    if (idx !== -1) {
      return text.slice(idx, idx + 500).trim();
    }
  }

  // Fall back to first + last sentence
  const sentences = text.split(/[.!?]\s+/).filter(Boolean);
  if (sentences.length <= 2) return text;
  return `${sentences[0]}. ${sentences[sentences.length - 1]}`;
}

2. Summarise Before Classifying

For long documents where the intent isn’t in a predictable location, use a cheap LLM to produce a one-sentence summary, then send that summary to Intentgine.

// Step 1: Summarise with a cheap model
const summary = await cheapLLM.complete({
  prompt: `Summarise the following in one sentence, focusing on what the sender wants:\n\n${document}`,
});

// Step 2: Classify the summary
const result = await fetch('https://api.intentgine.dev/v1/classify', {
  method: 'POST',
  headers: { 'Authorization': 'Bearer <api-key>', 'Content-Type': 'application/json' },
  body: JSON.stringify({
    data: summary,
    classification_set: 'support-routing-v1',
  }),
});

This keeps your Intentgine costs low while handling arbitrarily long input.

3. Sliding Window (Streaming / Voice-to-Text)

For real-time input like voice transcription, maintain a sliding window buffer and send it to Intentgine on an interval. When a tool is resolved, execute it and clear the buffer.

const WINDOW_SIZE = 300;
const INTERVAL_MS = 3000;

let buffer = '';

// Append new text as it arrives from transcription
function onTranscript(chunk: string) {
  buffer += ' ' + chunk;
  // Keep only the last WINDOW_SIZE characters
  if (buffer.length > WINDOW_SIZE) {
    buffer = buffer.slice(-WINDOW_SIZE);
  }
}

// Poll on an interval to avoid rate limits
setInterval(async () => {
  if (!buffer.trim()) return;

  const res = await fetch('https://api.intentgine.dev/v1/resolve', {
    method: 'POST',
    headers: { 'Authorization': 'Bearer <api-key>', 'Content-Type': 'application/json' },
    body: JSON.stringify({
      query: buffer.trim(),
      toolsets: ['voice-commands-v1'],
    }),
  });

  const data = await res.json();
  if (data.resolved?.tool) {
    await executeAction(data.resolved);
    buffer = ''; // Clear after successful intent detection
  }
}, INTERVAL_MS);

Caution

Choose your interval carefully. Too frequent and you’ll hit rate limits. Too slow and the user experience suffers. 2–5 seconds is a good starting point.

4. Constructed Queries

Instead of sending raw user input, you can construct a query that summarises the situation. This is useful when the user’s intent is spread across a long conversation or when you have metadata that helps.

// Raw situation: user has been complaining about a refund for 30 minutes
// in a chat app, total message length ~5000 chars

// Instead of sending all 5000 chars, construct a focused query:
const query = 'User complaining about refund for 30 minutes, escalating frustration';

const res = await fetch('https://api.intentgine.dev/v1/resolve', {
  method: 'POST',
  headers: { 'Authorization': 'Bearer <api-key>', 'Content-Type': 'application/json' },
  body: JSON.stringify({
    query,
    toolsets: ['escalation-v1'],
    context: 'Chat support session, 30 min duration, 5000 chars total',
  }),
});

// Might return: { resolved: { tool: "send_full_refund", parameters: { ... } }, metadata: { ... } }

Constructed queries work well when:

• The raw input is too long or noisy
• You have session metadata (duration, message count, sentiment trend)
• The intent is implicit rather than stated directly
• You’re aggregating across multiple messages

Choosing a Pattern

Pattern	Best For
Extract intent text	Emails, documents with predictable structure
Summarise first	Long unstructured documents
Sliding window	Real-time streaming (voice, live chat)
Constructed query	Multi-message conversations, metadata-rich contexts