Parallelization Idioms

There are many ways to parallelize code, so parallem tries to be accomodating.

Sync Idiom

The synchronous idiom is simplest to write. Declare agents in a for loop.

import parallem as pllm
from dotenv import load_dotenv


def power_of_n_agent(agt: pllm.AgentContext, n: int):
    return agt.ask_llm(f"Name a power of {n}").final_answer


load_dotenv()
with pllm.resume_directory(
    ".pllm/simplest",
    strategy="sync",  # or batch
) as orch:
    for i in range(2, 6):
        with orch.agent(i) as agt:
            print(power_of_n_agent(agt, i))

It is effective with sync and batch. Batch parallelization is effective because of ParaLLeM's interrupt semantics.

However, it is not effective if ran asynchronously, because power-of-2 agent must finish before power-of-3 agent can begin.

That is a limitation of python: you need await, async, and asyncio.run to allow async calls.

Async Idiom

The async idiom uses asyncio, involves async/await and async, and works seamlessly with other async functions.

It is effective with sync, concurrent, and batch.

import parallem as pllm
from dotenv import load_dotenv


async def power_of_n_agent(agt: pllm.AgentContext, n: int):
    return await agt.ask_llm(f"Name a power of {n}")


load_dotenv()
with pllm.resume_directory(
    ".pllm/example/async/powers",
    strategy="batch",  # or concurrent, batch
    dashboard=True,
) as orch:
    agts = []
    for i in range(2, 6):
        # Instantiate the agent.
        agts.append(orch.create_agent(power_of_n_agent, i))

    # Run agents. Similar to async.gather
    out = orch.run_agents(agts)
    print(out)

However, the async idiom is trickier to write.

Note

orch.run_agents is similar to asyncio.run(asyncio.gather(agts)). However, orch.run_agents is recommended when using batch mode, because orch.run_agents properly handles parallem's interrupt semantics.

Pseudo-Async Idiom

You can use the concurrent strategy even with synchronous functions to achieve parallelization. However, you must ensure that one agent does not block the other.

This idiom is less efficient than true async. In true async, await yields control between tasks, but here that is not possible.

import parallem as pllm
from dotenv import load_dotenv


def power_of_n_agent(agt: pllm.AgentContext, n: int):
    return agt.ask_llm(f"Name a power of {n}")


load_dotenv()
with pllm.resume_directory(
    ".pllm/simplest",
    strategy="concurrent",
) as orch:
    collector: list[pllm.LLMResponse] = []
    for i in range(2, 6):
        with orch.agent(i) as agt:
            collector.append(power_of_n_agent(agt, i))

    out = orch.resolve_all(collector)
    agt.print(out)