Conversations | Scrapybara Docs

Understanding multi-turn conversations

Multi-turn conversations in Scrapybara enable your agents to maintain context and state across multiple interactions. The Act SDK provides a structured way to manage these conversations through its message architecture.

Message architecture

The Act SDK uses a structured message system with three primary message types and five different part types. Understanding these components is crucial for building sophisticated multi-turn agents.

Message types

# Message types
class UserMessage:
    role: str = "user"  # Always "user"
    content: List[Union[TextPart, ImagePart]]  # What the user sends
class AssistantMessage:
    role: str = "assistant"  # Always "assistant"
    content: List[Union[TextPart, ToolCallPart, ReasoningPart]]  # The agent's response
    response_id: Optional[str] = None  # Unique identifier for the response
class ToolMessage:
    role: str = "tool"  # Always "tool"
    content: List[ToolResultPart]  # Results from tool operations
Message = Union[UserMessage, AssistantMessage, ToolMessage]

Message part types

Each message type contains various “parts” that serve different purposes:

# Message part types
class TextPart:
    type: str = "text"  # Always "text"
    text: str  # Plain text content
class ImagePart:
    type: str = "image"  # Always "image"
    image: str  # Base64 encoded image or URL
    mime_type: Optional[str] = None  # e.g., "image/png", "image/jpeg"
class ToolCallPart:
    type: str = "tool-call"  # Always "tool-call"
    id: Optional[str] = None  # Unique identifier for the tool call
    tool_call_id: str  # ID matching the tool result
    tool_name: str  # Name of the tool being called
    args: dict[str, Any]  # Arguments passed to the tool
class ToolResultPart:
    type: str = "tool-result"  # Always "tool-result"
    tool_call_id: str  # ID matching the original tool call
    tool_name: str  # Name of the tool that was called
    result: Any  # Result returned by the tool
    is_error: Optional[bool] = False  # Whether the tool execution resulted in an error
class ReasoningPart:
    type: str = "reasoning"  # Always "reasoning"
    id: Optional[str] = None  # Unique identifier for the reasoning part
    reasoning: str  # The agent's internal reasoning
    signature: Optional[str] = None  # Cryptographic signature for verification
    instructions: Optional[str] = None  # Additional context about the reasoning

Building multi-turn conversations

Instead of providing a single prompt, you can pass a complete message history using the messages parameter. This allows you to maintain the full conversation context. The Act SDK returns a messages field in the response that contains the complete conversation history. You can reuse this directly in your next act call.

Python

TypeScript

from scrapybara import Scrapybara
from scrapybara.anthropic import Anthropic
from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT
from scrapybara.tools import BashTool, ComputerTool, EditTool
client = Scrapybara()
instance = client.start_ubuntu()
# Initial conversation
response = client.act(
    model=Anthropic(),
    tools=[
        BashTool(instance),
        ComputerTool(instance),
        EditTool(instance),
    ],
    on_step=lambda step: print(step.text),
    system=UBUNTU_SYSTEM_PROMPT,
    prompt="Create a file called hello.py that prints 'Hello, World!'",
)
print('--------------------------------')
# Continue the conversation with the previous messages
follow_up_response = client.act(
    model=Anthropic(),
    tools=[
        BashTool(instance),
        ComputerTool(instance),
        EditTool(instance),
    ],
    on_step=lambda step: print(step.text),
    system=UBUNTU_SYSTEM_PROMPT,
    messages=response.messages + [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Now modify the file to accept a name as a command line argument and print 'Hello, {name}!'"
                }
            ]
        }
    ]
)
instance.stop()

Including screenshots in messages

Screenshots are a powerful way to provide visual context to your agent. You can include them in user messages using the ImagePart type.

Python

TypeScript

from scrapybara import Scrapybara
from scrapybara.anthropic import Anthropic
from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT
client = Scrapybara()
instance = client.start_ubuntu()
# Take a screenshot
screenshot = instance.screenshot().base_64_image
# Send the screenshot to the agent
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "What do you see in this screenshot? Describe the desktop environment."
            },
            {
                "type": "image",
                "image": 'data:image/png;base64,' + screenshot,
                "mime_type": "image/png"
            }
        ]
    }
]
response = client.act(
    model=Anthropic(),
    system=UBUNTU_SYSTEM_PROMPT,
    messages=messages,
)
print(response.text)
instance.stop()

Working with tools and reasoning

The Act SDK captures both tool calls and agent reasoning in its message architecture. Here’s how you can access and work with this information:

Examining tool calls and results

Python

TypeScript

from scrapybara import Scrapybara
from scrapybara.anthropic import Anthropic
from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT
from scrapybara.tools import BashTool
client = Scrapybara()
instance = client.start_ubuntu()
response = client.act(
    model=Anthropic(),
    tools=[BashTool(instance)],
    system=UBUNTU_SYSTEM_PROMPT,
    prompt="Show me the current directory structure",
)
# Analyze the conversation steps
for message in response.messages:
    if message.role == "assistant":
        for part in message.content:
            if part.type == "tool-call":
                print(f"Tool called: {part.tool_name}")
                print(f"Arguments: {part.args}")
    elif message.role == "tool":
        for part in message.content:
            print(f"Tool result from {part.tool_name}: {part.result}")
instance.stop()

Accessing agent reasoning

Python

TypeScript

from scrapybara import Scrapybara
from scrapybara.anthropic import Anthropic
from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT
from scrapybara.tools import BashTool, ComputerTool
client = Scrapybara()
instance = client.start_ubuntu()
response = client.act(
    model=Anthropic(name="claude-3-7-sonnet-20250219-thinking"),
    tools=[
        BashTool(instance),
        ComputerTool(instance),
    ],
    system=UBUNTU_SYSTEM_PROMPT,
    prompt="Open Firefox and navigate to scrapybara.com",
)
# Extract reasoning parts from assistant messages
for message in response.messages:
    if message.role == "assistant":
        for part in message.content:
            if part.type == "reasoning":
                print("Agent reasoning:")
                print(part.reasoning)
# Or access reasoning directly from steps
for step in response.steps:
    if step.reasoning_parts:
        print(f"Step reasoning: {step.reasoning_parts}")
instance.stop()

Best practices for multi-turn conversations

Maintain message history: Always use the returned messages from each call to maintain conversation context.
Clear instructions: Provide clear, specific instructions in each new user message.
Handle context length: For very long conversations, consider summarizing or truncating older messages to avoid exceeding model context limits.
Include visual context: Use screenshots when appropriate to provide additional context to the agent.
Monitor token usage: Track token usage through the usage field to prevent exceeding quotas or limits.
Process message parts: Parse and handle different message parts appropriately based on their type.

Simple multi-turn example

Here’s an interactive Read-Eval-Print Loop (REPL) implementation that allows you to have ongoing conversations with your agent:

Python

TypeScript

from scrapybara import Scrapybara
from scrapybara.anthropic import Anthropic
from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT
from scrapybara.tools import BashTool, ComputerTool, EditTool
def agent_repl():
    client = Scrapybara()
    instance = client.start_ubuntu()
    tools = [BashTool(instance), ComputerTool(instance), EditTool(instance)]
    messages = []
    print("Scrapybara REPL started. Type 'exit' to quit")
    try:
        while True:
            # Get user input
            user_input = input("\n> ")
            # Exit command
            if user_input.lower() == 'exit':
                break
            # Regular text command
            messages.append({
                "role": "user",
                "content": [{"type": "text", "text": user_input}]
            })
            # Process with agent
            print("Processing...")
            response = client.act(
                model=Anthropic(),
                tools=tools,
                system=UBUNTU_SYSTEM_PROMPT,
                on_step=lambda step: print(step.text),
                messages=messages
            )
            # Update conversation history
            messages = response.messages
    finally:
        instance.stop()
        print("Session ended.")
if __name__ == "__main__":
    agent_repl()