Quickstart

This guide gives a tour of parallem. We support many agentic features including function calling, image input, structured output, web search, and more -- all with the Batch API.

examples/full_tour.py

from pydantic import BaseModel
from dotenv import load_dotenv
from PIL import Image

import parallem as pllm


class MyModel(BaseModel):
    final_answer: str


def count_files(directory: str) -> int:
    """Counts files in a directory"""
    return 4


def tour_agent(agt: pllm.AgentContext):
    # 1. Basic LLM call
    resp1 = agt.ask_llm("Please name a power of 3.")

    # 2. Web search tool
    resp2 = agt.ask_llm(
        "In 1 sentence, what is AAPL's current price?",
        tools=[pllm.tools.WebSearchTool()],
    )

    # 3. Structured output
    resp3 = agt.ask_llm("What is the capital of France?", structured_output=MyModel)

    # 4. Image input. NOTE: Adjust image as needed.
    img = Image.open("tests/data/images/Nokota_Horses_cropped.jpg")
    img.thumbnail((100, 100))  # Downsample
    resp4 = agt.ask_llm("What animal is this?", img)

    # 5,6. Function calling.
    # Keeping track of message state can be tedious. see the MessageState abstraction.
    ls_prompt = "How many files are in ~/examples? Give the final answer in words."
    resp5 = agt.ask_llm(
        ls_prompt,
        tools=pllm.to_tool_schema([count_files]),
    )
    fc_outs = agt.ask_functions(resp5, count_files=count_files)
    resp6 = agt.ask_llm([ls_prompt, resp5, *fc_outs])

    for i, resp in enumerate([resp1, resp2, resp3, resp4, resp5, resp6]):
        if fcs := resp.resolve_function_calls():
            agt.print(fcs)
        final_answer = resp.final_answer.replace("\n", " ")
        agt.print(f"{i + 1}. {final_answer}")


if __name__ == "__main__":
    load_dotenv()

    with pllm.resume_directory(
        ".pllm/example/batch",
        provider="openai",
        strategy="sync",
        dashboard=True,
        hash_by=["llm"],
    ) as orch:
        with orch.agent() as agt:
            tour_agent(agt)

[INFO] Resuming with session_id=0
1. 243 (which is 3^5).
2. Apple Inc. (AAPL) is currently trading at $257.46 per share (latest trade: 01:15:00 UTC on March 7, 2026).
3. {"final_answer":"The capital of France is Paris."}
4. These are horses — domestic equines. The photo shows two adult horses standing in a grassy field.
FunctionCall(name=count_files, call_id=call_Xau, args={'directory': '~/examples'})
5.
6. There are four files in ~/examples.
[DASH] ↘ 2b55f032 ↘ f7824348 ↘ c9f2fcc4 ↘ f13b65b2 ↘ bc7f1641 ↘ 2883bea6

If you run the program again, the results are cached and available instantly.

Batch mode

The entire workflow can be done in batch mode simply by changing one line of code:

with pllm.resume_directory(
    ".pllm/example/batch",
    provider="openai",
    strategy="batch",  # Only change!!!
    # ...
) as orch:
    # ...

[INFO] Resuming with session_id=0
Submit 1 batch (6 calls)? (y/n/preview): y
Sent batch: batch_69b4a26290008190a08e246922784ed8
[DASH] ⇈ 69b4a262

After waiting a while, run the program again. The batch will automatically be downloaded and handled for you.

[INFO] Resuming with session_id=1
Batch batch_69b4a26290008190a08e246922784ed8 completed and stored.
1. 243 (which is 3^5).
2. Apple Inc. (AAPL) is currently trading at $257.46 per share (latest trade: 01:15:00 UTC on March 7, 2026).
3. {"final_answer":"The capital of France is Paris."}
4. These are horses — domestic equines. The photo shows two adult horses standing in a grassy field.
FunctionCall(name=count_files, call_id=call_Xau, args={'directory': '~/examples'})
5.
6. There are four files in ~/examples.
C 2b55f032 C f7824348 C c9f2fcc4 C f13b65b2 C bc7f1641 C 2883bea6

Warning

ParaLLeM saves progress by hashing. However, not all config settings are hashed. For instance, tool definitions are not hashed. If available tools change, then the hashes are still considered identical, so the old cached value is still returned. See the Persistence guide for how to control this with hash_by and salt.

Advanced Usage

• See the docs for:
  • The MessageState guide: simply a list that automatically stores documents and responses as they get added. Helps track long conversations, reducing boilerplate.
  • The Ask guide: ask_llm, ask_functions (invoking user functions).
  • The memoize guide: for caching expensive or non-deterministic blocks of code.

Further examples

A suite of examples (a "cookbook") is available under examples/*.