Overview

Act SDK

The Act SDK is a unified SDK for building computer use agents with Python and TypeScript. It provides a simple interface for executing looping agentic actions with support for many models and tools. Build production-ready computer use agents with pre-built tools to connect to Scrapybara instances.

How it works

act initiates an interaction loop that continues until the agent achieves its objective. Each iteration of the loop is called a step, which consists of the agent’s text response, the agent’s tool calls, and the results of those tool calls. The loop terminates when the agent returns a message without invoking any tools, and returns messages, steps, text, output (if schema is provided), and usage after the agent’s execution.

1
response = client.act(
2
    model=Anthropic(),
3
    tools=[
4
        BashTool(instance),
5
        ComputerTool(instance),
6
        EditTool(instance),
7
        BrowserTool(instance),
8
    ],
9
    system=SYSTEM_PROMPT,
10
    prompt="Go to the top link on Hacker News",
11
    on_step=lambda step: print(step.text),
12
)
13
messages = response.messages
14
steps = response.steps
15
text = response.text
16
usage = response.usage

An act call consists of 3 core components:

Model

The model specifies the base LLM for the agent. At each step, the model examines the previous messages, the current state of the computer, and uses tools to take action. Currently, the SDK supports Anthropic models, with more providers coming soon.

1
from scrapybara.anthropic import Anthropic
2
3
model = Anthropic()

Tools

Tools are functions that enable agents to interact with the computer. Each tool is defined by a name, description, and how it can be executed with parameters and an execution function. A tool can take in a Scrapybara instance to interact with it directly. Learn more about pre-built tools and how to define custom tools here.

1
from scrapybara import Scrapybara
2
from scrapybara.tools import BashTool, ComputerTool, EditTool, BrowserTool
3
4
client = Scrapybara()
5
instance = client.start()
6
7
tools = [
8
    BashTool(instance),
9
    ComputerTool(instance),
10
    EditTool(instance),
11
    BrowserTool(instance),
12
]

Prompt

The prompt is split into two parts, the system prompt and a user prompt. system defines the general behavior of the agent, such as its capabilities and constraints. You can use our provided SYSTEM_PROMPT to get started, or define your own. prompt should denote the agent’s current objective. Alternatively, you can provide messages instead of prompt to start the agent with a history of messages. act conveniently returns messages after the agent’s execution, so you can reuse it in another act call.

1
from scrapybara.prompts import SYSTEM_PROMPT
2
3
system = SYSTEM_PROMPT
4
prompt = "Go to the top link on Hacker News"

Structured output

Use the schema parameter to define a desired structured output. The response’s output field will contain the typed data returned by the model. This is particularly useful when scraping or collecting structured data from websites.

Under the hood, we pass in a StructuredOutputTool to enforce and parse the schema.

1
from pydantic import BaseModel
2
from typing import List
3
4
class HNSchema(BaseModel):
5
    class Post(BaseModel):
6
        title: str
7
        url: str 
8
        points: int
9
    
10
    posts: List[Post]
11
12
response = client.act(
13
    model=Anthropic(),
14
    tools=[
15
        BashTool(instance),
16
        ComputerTool(instance),
17
        EditTool(instance),
18
        BrowserTool(instance),
19
    ],
20
    schema=HNSchema,
21
    system=SYSTEM_PROMPT,
22
    prompt="Get the top 10 posts on Hacker News",
23
)
24
25
posts = response.output.posts

Agent credits

Consume agent credits or bring your own API key. Without an API key, each step consumes 1 agent credit. With your own API key, model charges are billed directly to your provider API key.

Full example

Here is how you can build a hybrid computer use agent that can take action graphically, control the browser programmatically, and output structured data.

1
from scrapybara import Scrapybara
2
from scrapybara.anthropic import Anthropic
3
from scrapybara.prompts import SYSTEM_PROMPT
4
from scrapybara.tools import BashTool, ComputerTool, EditTool, BrowserTool
5
from pydantic import BaseModel
6
from typing import List
7
8
client = Scrapybara()
9
instance = client.start()
10
instance.browser.start()
11
12
class HNSchema(BaseModel):
13
    class Post(BaseModel):
14
        title: str
15
        url: str 
16
        points: int
17
    
18
    posts: List[Post]
19
20
response = client.act(
21
    model=Anthropic(),
22
    tools=[
23
        BashTool(instance),
24
        ComputerTool(instance),
25
        EditTool(instance),
26
        BrowserTool(instance),
27
    ],
28
    system=SYSTEM_PROMPT,
29
    prompt="Get the top 10 posts on Hacker News",
30
    schema=HNSchema,
31
    on_step=lambda step: print(step.text),
32
)
33
34
posts = response.output.posts
35
print(posts)
36
37
instance.browser.stop()
38
instance.stop()