Classification Extraction

Classification Extraction

Classification extraction allows your classification sets to automatically split compound queries into individual sub-queries and classify each one.

Overview

When enabled, the LLM can detect when a query contains multiple distinct intents and extract them into separate queries. Each extracted query is then classified using the same classification set.

Example

Input: “Turn on kitchen lights and turn off bedroom lights”

Without extraction:

{
  "classification": "",
  "extraction_needed": true,
  "confidence": 0.95
}

With extraction:

{
  "classification": "",
  "extraction_needed": true,
  "confidence": 0.95,
  "extracted": [
    {
      "query": "turn on kitchen lights",
      "classification": "kitchen",
      "confidence": 0.97
    },
    {
      "query": "turn off bedroom lights",
      "classification": "bedroom",
      "confidence": 0.96
    }
  ]
}

When to Use Extraction

Enable extraction when:

• Your queries may contain multiple distinct intents
• You need to process each intent separately
• You want to save HTTP round trips

Don’t enable extraction when:

• All queries are single-intent
• You handle multi-intent queries differently
• Cost is a primary concern

Cost Impact

Extraction increases request usage:

• Without extraction: 1 request per classification
• With extraction: 2 requests (1 classify + 1 extract with sub-classifications)

Example:

• “Turn on kitchen lights” → 1 request (extraction_needed: false)
• “Turn on kitchen and turn off bedroom” → 2 requests (extraction with 2 sub-classifications)

Benefit: Saves N-1 requests vs manual approach (which would need 1 + N requests)

How It Works

1. You enable extraction on a classification set
2. When classifying, the LLM determines if extraction is needed (returns extraction_needed flag)
3. If needed AND extraction is enabled, a second LLM call extracts and classifies all sub-queries
4. Results are returned in a single response

Enabling Extraction

Via Console

1. Go to your app’s Classification Sets
2. Create or edit a classification set
3. Check “Enable Automatic Extraction”
4. Save

Via API

// Create with extraction enabled
const response = await fetch('https://api.intentgine.dev/v1/classification-sets', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    signature: 'my-classifier-v1',
    name: 'My Classifier',
    classes: [
      { label: 'class1', description: 'First class' },
      { label: 'class2', description: 'Second class' }
    ],
    enable_extraction: true
  })
});

Response Format

When extraction is performed, the response includes an extracted array:

interface ClassificationResult {
  input: string;
  classification: string;
  extraction_needed: boolean;
  confidence: number;
  extracted?: Array<{
    query: string;
    classification: string;
    confidence: number;
  }>;
}

Response Fields

• classification (string): One of the defined classes, or empty string if no match
• extraction_needed (boolean): LLM’s determination of whether extraction should be performed
• extracted (array, optional): Array of extracted sub-queries with classifications
• metadata.extraction_performed (boolean): Whether extraction was actually performed
• metadata.sub_classifications_performed (number): Number of sub-queries classified

Best Practices

1. Test thoroughly: Verify extraction works for your use case
2. Monitor costs: Track request usage with extraction enabled
3. Use appropriate classes: Ensure your classes support multi-intent detection
4. Handle both cases: Code should handle results with and without extraction

Examples

Multi-Intent Detection

// Classification set
{
  "signature": "intent-type-v1",
  "classes": [
    { "label": "single-intent", "description": "One action" },
    { "label": "multi-intent", "description": "Multiple actions" }
  ],
  "enable_extraction": true
}

// Query
"Turn on kitchen lights and turn off bedroom lights"

// Response
{
  "results": [{
    "input": "turn on kitchen lights and turn off bedroom lights",
    "classification": "",
    "extraction_needed": true,
    "confidence": 0.95,
    "extracted": [
      {
        "query": "turn on kitchen lights",
        "classification": "single-intent",
        "confidence": 0.98
      },
      {
        "query": "turn off bedroom lights",
        "classification": "single-intent",
        "confidence": 0.97
      }
    ]
  }],
  "metadata": {
    "requests_used": 2,
    "extraction_performed": true,
    "sub_classifications_performed": 2,
    "source": "compute",
    "requests_remaining": 9998
  }
}

Area-Based Routing

// Classification set
{
  "signature": "area-router-v1",
  "classes": [
    { "label": "kitchen", "description": "Kitchen area" },
    { "label": "bedroom", "description": "Bedroom area" },
    { "label": "living-room", "description": "Living room area" }
  ],
  "enable_extraction": true
}

// Query
"Turn on kitchen and bedroom lights"

// Response
{
  "classification": "",
  "extraction_needed": true,
  "extracted": [
    {
      "query": "turn on kitchen lights",
      "classification": "kitchen",
      "confidence": 0.96
    },
    {
      "query": "turn on bedroom lights",
      "classification": "bedroom",
      "confidence": 0.95
    }
  ]
}

Configuring Extraction Prompt

Admins can customize the extraction prompt in the system settings:

1. Navigate to System → System Settings
2. Edit the “Classification Extraction Prompt”
3. Save changes

The default prompt guides the LLM to:

• Extract when: Different actions, devices, or areas
• Don’t extract when: Same action on multiple targets

Troubleshooting

Extraction not working

• Verify enable_extraction is true on the classification set
• Check that queries actually contain multiple intents
• Review extraction prompt configuration

Unexpected costs

• Monitor requests_used in response metadata
• Check extraction_performed flag
• Consider disabling extraction for single-intent use cases

Low confidence scores

• Review class descriptions for clarity
• Adjust extraction prompt if needed
• Test with different query phrasings