/cursive-py

✦ The intuitive python LLM framework

Primary LanguagePython

Logo Logo

Cursive is a universal and intuitive framework for interacting with LLMs.

highlights

✦ Extensible - You can easily hook into any part of a completion life cycle. Be it to log, cache, or modify the results.

✦ Functions - Easily describe functions that the LLM can use along with its definition, with any model (currently supporting GPT-4, GPT-3.5, Claude 2, and Claude Instant)

✦ Universal - Cursive aims to bridge as many capabilities between different models as possible. Ultimately, this means that with a single interface, you can allow your users to choose any model.

✦ Informative - Cursive comes with built-in token usage and costs calculations, as accurate as possible.

✦ Reliable - Cursive comes with automatic retry and model expanding upon exceeding context length. Which you can always configure.

quickstart

  1. Install.
poetry add cursivepy
# or
pip install cursivepy
  1. Start using.
from cursive import Cursive

cursive = Cursive()

response = cursive.ask(
    prompt='What is the meaning of life?',
)

print(response.answer)

usage

Conversation

Chaining a conversation is easy with cursive. You can pass any of the options you're used to with OpenAI's API.

res_a = cursive.ask(
    prompt='Give me a good name for a gecko.',
    model='gpt-4',
    max_tokens=16,
)

print(res_a.answer) # Zephyr

res_b = res_a.conversation.ask(
    prompt='How would you say it in Portuguese?'
)

print(res_b.answer) # Zéfiro

Streaming

Streaming is also supported, and we also keep track of the tokens for you!

result = cursive.ask(
    prompt='Count to 10',
    stream=True,
    on_token=lambda partial: print(partial['content'])
)

print(result.usage.total_tokens) # 40

Functions

You can use very easily to define and describe functions, along side with their execution code.

from cursive import cursive_function, Cursive

cursive = Cursive()

@cursive_function()
def add(a: float, b: float):
    """
    Adds two numbers.

    a: The first number.
    b: The second number.
    """
    return a + b

res = cursive.ask(
    prompt='What is the sum of 232 and 243?',
    functions=[add],
)

print(res.answer) # The sum of 232 and 243 is 475.

The functions' result will automatically be fed into the conversation and another completion will be made. If you want to prevent this, you can add pause to your function definition.

@cursive_function(pause=True)
def create_character(name: str, age: str):
    """
    Creates a character.

    name: The name of the character.
    age: The age of the character.
    """
    return {
        'name': name,
        'age': age,
    }

res = cursive.ask(
    prompt='Create a character named John who is 23 years old.',
    functions=[create_character],
)

print(res.function_result) # { name: 'John', age: 23 }

Cursive also supports passing in undecorated functions!

def add(a: float, b: float):
    return a + b

res = cursive.ask(
    prompt='What is the sum of 232 and 243?',
    functions=[add], # this is equivalent to cursive_function(pause=True)(add)
)
if res.function_result:
    print(res.function_result) # 475
else:
    print(res.answer) # Text answer in case the function is not called

Models

Cursive also supports the generation of Pydantic BaseModels.

from cursive.compat.pydantic import BaseModel, Field # Pydantic V1 API

class Character(BaseModel):
    name: str
    age: int
    skills: list[str] = Field(min_items=2)

res = cursive.ask(
    prompt='Create a character named John who is 23 years old.',
    function_call=Character,
)
res.function_result # is a Character instance with autogenerated fields

Hooks

You can hook into any part of the completion life cycle.

cursive.on('completion:after', lambda result: print(
    result.data.cost.total,
    result.data.usage.total_tokens,
))

cursive.on('completion:error', lambda result: print(
    result.error,
))

cursive.ask({
    prompt: 'Can androids dream of electric sheep?',
})

# 0.0002185
# 113

Embedding

You can create embeddings pretty easily with cursive.

embedding = cursive.embed('This should be a document.')

This will support different types of documents and integrations pretty soon.

Reliability

Cursive comes with automatic retry with backoff upon failing completions, and model expanding upon exceeding context length -- which means that it tries again with a model with a bigger context length when it fails by running out of it.

You can configure this behavior by passing the retry and expand options to Cursive constructor.

cursive = Cursive(
    max_retries=5, # 0 disables it completely
    expand={
        'enable': True,
        'defaults_to': 'gpt-3.5-turbo-16k',
        'resolve_model': {
            'gpt-3.5-turbo': 'gpt-3.5-turbo-16k',
            'gpt-4': 'claude-2',
        },
    },
)

Available Models

OpenAI models
  • gpt-3.5-turbo
  • gpt-3.5-turbo-16k
  • gpt-4
  • gpt-4-32k
  • Any other chat completion model version
Credentials

You can pass your OpenAI API key to Cursive's constructor, or set the OPENAI_API_KEY environment variable.

Anthropic models
  • claude-2
  • claude-instant-1
  • claude-instant-1.2
  • Any other model version
Credentials

You can pass your Anthropic API key to Cursive's constructor, or set the ANTHROPIC_API_KEY environment variable.

OpenRouter models

OpenRouter is a service that gives you access to leading language models in an OpenAI-compatible API, including function calling!

  • anthropic/claude-instant-1.2
  • anthropic/claude-2
  • openai/gpt-4-32k
  • google/palm-2-codechat-bison
  • nousresearch/nous-hermes-llama2-13b
  • Any model version from https://openrouter.ai/docs#models
Credentials
from cursive import Cursive

cursive = Cursive(
    openrouter={
      "api_key": "sk-or-...",
      "app_title": "Your App Name",
      "app_url": "https://appurl.com",
    }
)

cursive.ask(
    model="anthropic/claude-instant-1.2",
    prompt="What is the meaning of life?"
)
Cohere models
  • command
  • Any other model version (such as command-nightly)
Credentials

You can pass your Cohere API key to Cursive's constructor, or set the COHERE_API_KEY environment variable.

Replicate models You can prepend `replicate/` to any model name and version available on Replicate.
Example
cursive.ask(
    prompt='What is the meaning of life?',
    model='replicate/a16z-infra/llama-2-13b-chat:2a7f981751ec7fdf87b5b91ad4db53683a98082e9ff7bfd12c8cd5ea85980a52',
)
Credentials

You can pass your Replicate API key to Cursive's constructor, or set the REPLICATE_API_TOKEN environment variable.

roadmap

vendor support

  • Anthropic
  • Cohere
  • Replicate
  • OpenRouter
  • Azure OpenAI models
  • Huggingface