Simon Willison on Nostr: Anthropic released a fascinating new capability today called "Computer Use" - a mode ...
Anthropic released a fascinating new capability today called "Computer Use" - a mode of their Claude 3.5 Sonnet model where it can do things like accept screenshots of a remotely operated computer and send back commands to click on specific coordinates, enter text etc
My notes on what I've figured out so far:
https://simonwillison.net/2024/Oct/22/computer-use/Published at
2024-10-22 17:52:16Event JSON
{
"id": "2cdee601184d88e3821973a0b4903cf42401bd45d55896447cddc30c7ff2b66b",
"pubkey": "8b0be93ed69c30e9a68159fd384fd8308ce4bbf16c39e840e0803dcb6c08720e",
"created_at": 1729619536,
"kind": 1,
"tags": [
[
"proxy",
"https://fedi.simonwillison.net/users/simon/statuses/113352345939182214",
"activitypub"
]
],
"content": "Anthropic released a fascinating new capability today called \"Computer Use\" - a mode of their Claude 3.5 Sonnet model where it can do things like accept screenshots of a remotely operated computer and send back commands to click on specific coordinates, enter text etc\n\nMy notes on what I've figured out so far: https://simonwillison.net/2024/Oct/22/computer-use/",
"sig": "735a436600261e65bf775852b105ab876f82e4717887299a6b2976362de9c411eb87cb0994adc30a7c9ff09d4af676d3dd849db3a22857ce5bb40c0b21939f53"
}
