AT Protocol

The AT Protocol at:// (Authenticated Transfer Protocol, or atproto) is a standard for public conversation and an open-source framework for building social apps.

It creates a standard format for user identity, follows, and data on social apps, allowing apps to interoperate and users to move across them freely. It is a federated network with account portability.

Used by Bluesky. Frontpage and Smoke Signal (see below ).

In a sense, it brings Markdown files to the web, or was Make world.space said:

AT Protocol brings File Over App philosophy to the network. Read full article about A Social Filesystem at Overreacted.

Read Basic Concepts ​, Identity, Data repositories​, Federation and more.

# Extracting data via the protocol

Besides the dedicated tools to Bluesky, here are the more related to extract and ATProto.

# Web Applications

• Frontpage - HackerNews-style interface for Bluesky
• WhiteWind atproto blog: Hosting blog article on the Protocol
• Smoke Signal - RSVP management system using ATProto
• ATProto Browser - Browse AT Protocol data
  • Example user: atproto-browser.vercel.app/at/ssp.sh
  • Source Code
  • ! new: PDSls
• Bluesky Firehose 3D - 3D visualization of the firehose
• TUI Interface - Terminal-based interface
• Bluecast: Bluecast is a real-time audio streaming service that anyone with a Bluesky account can use this service.
• GitHub alternative on top of AT Proto: Tangled

# Development Tools

• SDKs & Libraries:
  • atproto (Python) - Official Python SDK with documentation
  • bluesky-social-graph - Social graph visualization tool
  • skeetstream - Stream posts to gzipped files

# Data Analysis & Processing

• Jetstream Tools:
  • Jetstream Example - Guide to shrinking AT Proto Firehose
  • Event Stream Documentation
  • Jetstream GitHub - A simplified JSON event stream
  • Related: Bluesky uses Jetstreams - See Eric’s comment
• Data Tools & Tutorials:
  • atproto-data-tools - Scripts for AT Protocol data analysis
  • David Gasquez - Exploring AT Protocol with Python - Tutorial by David Gasquez
  • David’s Bluesky Post
  • Followers Edge List Script - Generate followers edge list from hashtag search
  • Save Firehose directly to Kafka: SQLFlow
• Dashboards
  • Rill Dashboard on ATProto Data: A demo of Rill dashboards exploring BlueSky metrics. Check the deployed demo here.
• ClickHouse:
  • Jetstream

# Data Access & Querying

• DuckDB Resources:
  • Hive Catalog - Access via https://hive.buz.dev/bluesky/catalog or via select * from read_parquet('https://hive.buz.dev/bluesky/jetstream/latest.parquet') by Jake Thomas
  • Skyfirehose - Query Jetstream with DuckDB
• Firehose Access:
  • Guide to consuming firehose cheaply

Based on Jetstream: Shrinking the AT Proto Firehose by >99% · Jaz’s Blog, it’s better to access data through their Firehose protocol rather than REST APIs.

# DuckDB

But there’s a browser for checking your content:

• Use-Case: Reading data via at:// (AT Protocol) with DuckDB

# Bsky Engagement Stats

Querying Bluesky with DuckDB and SQL

# Graph: Accounts within 5 hops

Interesting discovery: there are roughly 9,000 accounts within 5 hops of those I currently follow! 🚀 (I assume I didn’t mess up the data gathering) Powered by duckdb with the seamless
SQL / PGQ syntax supported in the DuckPGQ extension. Amazing what you can uncover with the right tools! Post by @dtenwolde.bsky.social — Bluesky

Check some stuff on Future of Web, and Bluesky

# Implement a DuckDB Community Extension

Just wapping the relevant macros to do the queries. pivot_table – DuckDB Community Extensions (by @a13x.bsky.social ) is a good example of an extension that just exposes a bunch of SQL macros, chsql – DuckDB Community Extensions is another cool one in that vibe. Post by @carlopi.bsky.social — Bluesky

# Further Reads

• Atmospheric Computing | Paul’s Dev Notes
• A Social Filesystem — overreacted
• AT Protocol for Agents by David Gasquez

---

Origin: Bluesky protocol