Skip to main content

​
Introduction

DSPy is a declarative framework for building modular AI software with structured code instead of brittle prompts, offering algorithms that compile AI programs into effective prompts and weights for language models across classifiers, RAG pipelines, and agent loops.

​
Integration Steps

1
2
Create a .env file in your project.
HELICONE_API_KEY=sk-helicone-...
3
Python
pip install dspy
4
Python
import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure DSPy to use Helicone AI Gateway
lm = dspy.LM(
  'gpt-4o-mini',  # or any other model from the Helicone model registry
  api_key=os.getenv('HELICONE_API_KEY'),
  api_base='https://ai-gateway.helicone.ai/'
)

dspy.configure(lm=lm)

print(lm("Hello, world!"))
5
While you’re here, why not give us a star on GitHub? It helps us a lot!

​
Complete Working Examples

​
Basic Chain of Thought

Python
import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure Helicone AI Gateway
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1'
)
dspy.configure(lm=lm)

# Define a module
qa = dspy.ChainOfThought('question -> answer')

# Run inference
response = qa(question="How many floors are in the castle David Gregory inherited?")

print('Answer:', response.answer)
print('Reasoning:', response.reasoning)

​
Custom Generation Configuration

Configure temperature, max_tokens, and other parameters:
Python
import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with custom generation parameters
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    temperature=0.9,
    max_tokens=2000
)
dspy.configure(lm=lm)

# Use with any DSPy module
predict = dspy.Predict("question -> creative_answer")
response = predict(question="Write a creative story about AI")
print(response.creative_answer)

​
Tracking with Custom Properties

Add custom properties to track and filter your requests in the Helicone dashboard:
Python
import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with custom Helicone headers
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    extra_headers={
        # Session tracking
        'Helicone-Session-Id': 'dspy-example-session',
        'Helicone-Session-Name': 'Question Answering',

        # User tracking
        'Helicone-User-Id': 'user-123',

        # Custom properties for filtering
        'Helicone-Property-Environment': 'production',
        'Helicone-Property-Module': 'chain-of-thought',
        'Helicone-Property-Version': '1.0.0'
    }
)
dspy.configure(lm=lm)

# Use normally
qa = dspy.ChainOfThought('question -> answer')
response = qa(question="What is DSPy?")
print(response.answer)

​
Helicone Prompts Integration

Use Helicone Prompts for centralized prompt management with DSPy signatures:
Python
import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with prompt parameters
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    extra_body={
        'prompt_id': 'customer-support-prompt-id',
        'version_id': 'version-uuid',
        'environment': 'production',
        'inputs': {
            'customer_name': 'Sarah',
            'issue_type': 'technical'
        }
    }
)
dspy.configure(lm=lm)
Learn more about Prompts with AI Gateway.

​
Advanced Features

​
Rate Limiting

Configure rate limits for your DSPy applications:
Python
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    extra_headers={
        'Helicone-Rate-Limit-Policy': 'basic-100'
    }
)

​
Caching

Enable intelligent caching to reduce costs:
Python
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    cache=True  # DSPy's built-in caching works with Helicone
)

​
Session Tracking for Multi-Turn Conversations

Track entire conversation flows in DSPy programs:
Python
import uuid

session_id = str(uuid.uuid4())

lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    extra_headers={
        'Helicone-Session-Id': session_id,
        'Helicone-Session-Name': 'Customer Support',
        'Helicone-Session-Path': '/support/chat'
    }
)
dspy.configure(lm=lm)

# All calls in this session will be grouped together
qa = dspy.ChainOfThought('question -> answer')

# Multiple turns
response1 = qa(question="What is your return policy?")
response2 = qa(question="How long does shipping take?")
response3 = qa(question="Do you ship internationally?")

# View the full conversation in Helicone Sessions

AI Gateway Overview

Learn about Helicone’s AI Gateway features and capabilities

Provider Routing

Configure intelligent routing and automatic failover

Model Registry

Browse all available models and providers

Prompt Management

Version and manage prompts with Helicone Prompts

Custom Properties

Add metadata to track and filter your requests

Sessions

Track multi-turn conversations and user sessions

Rate Limiting

Configure rate limits for your applications

Caching

Reduce costs and latency with intelligent caching