fetcher-sh

fetcher-sh/fetcher-skills

2stars 1forks 1contributors

README.md

# fetcher-skills

Agent skills for [fetcher.sh](https://fetcher.sh) — pay-per-call web-data APIs for
Twitter/X, TikTok, Instagram, YouTube, Reddit, Google Search, Google Maps,
Google News, Google Play, the App Store, and Yelp. Every call is paid in USDC,
either per request via [x402](https://x402.org) or prepaid against a credit
balance with a Bearer API key. No OAuth, no signup form, no API key request
queue — an agent can start pulling data in one HTTP round trip.

Each service is its own skill so an agent only loads the one it needs; a
shared `fetcher` skill covers payment, credits, and MCP setup once for all of
them.

## Why fetcher.sh

- **No OAuth, no app review.** You authenticate to `fetcher.sh`'s own hosts,
  not to Twitter/TikTok/Instagram/etc. — there's no developer-account queue
  to wait on for any of the 11 services.
- **Pay per call, not per tier.** $0.002–$0.005/request depending on
  endpoint, prepaid credits or [x402](https://x402.org) micropayments in
  USDC. No monthly minimum, no unused-tier waste.
- **One host per service, one pattern for all of them.** `twitter.fetcher.sh`,
  `tiktok.fetcher.sh`, `instagram.fetcher.sh`, and eight more — same
  `{ "status", "message", "data" }` envelope, same auth, same MCP tool names
  everywhere.
- **MCP-native.** Every host exposes `/mcp` with `search_endpoints`,
  `describe_endpoint`, `fetch_data`, and a named hero-endpoint shortcut, so an
  MCP-connected agent can explore and call the API without reading a spec
  first.
- **Machine-readable everywhere.** `/openapi.json`, `/llms.txt`, and
  `/skill.md` are generated from the live route handlers on every host, so
  they can't drift out of sync with what actually ships.

## What you can fetch

| Service | Skill | Base URL | What it covers |
| --- | --- | --- | --- |
| Twitter / X | [`twitter-api`](skills/twitter-api/SKILL.md) / [`x-api`](skills/x-api/SKILL.md) | `twitter.fetcher.sh` | Search tweets, resolve profiles by handle or ID, and pull timelines, replies, followers, lists, and trends |
| TikTok | [`tiktok-api`](skills/tiktok-api/SKILL.md) | `tiktok.fetcher.sh` | Search posts, resolve users by handle, and pull followers, hashtags, music, comments, and replies |
| Instagram | [`instagram-api`](skills/instagram-api/SKILL.md) | `instagram.fetcher.sh` | Profiles, posts, reels, stories, followers, hashtags, locations, and comment threads |
| YouTube | [`youtube-api`](skills/youtube-api/SKILL.md) | `youtube.fetcher.sh` | Search videos, channels, and playlists; fetch video details, comments, shorts, live streams, and trending |
| Reddit | [`reddit-api`](skills/reddit-api/SKILL.md) | `reddit.fetcher.sh` | Search posts, subreddits, and users; pull hot/new/top/best feeds, comment trees, and user history |
| Google Search | [`google-search`](skills/google-search/SKILL.md) | `google.fetcher.sh` | Programmatic Google web search results as clean JSON |
| Google Maps | [`google-maps`](skills/google-maps/SKILL.md) | `google-maps.fetcher.sh` | Place search, place details, and reviews |
| Google News | [`google-news`](skills/google-news/SKILL.md) | `google-news.fetcher.sh` | Headlines by section (world, business, technology, ...), keyword search, topics, and article URL decoding |
| Google Play | [`google-play`](skills/google-play/SKILL.md) | `googleplay.fetcher.sh` | App search, app details, reviews, permissions, data safety, and developer catalogs |
| App Store | [`app-store`](skills/app-store/SKILL.md) | `appstore.fetcher.sh` | Apple App Store app and bundle lookups, reviews, similar apps, and developer catalogs |
| Yelp | [`yelp`](skills/yelp/SKILL.md) | `yelp.fetcher.sh` | Business search by query and location, place details, and reviews |
| Shared | [`fetcher`](skills/fetcher/SKILL.md) | `fetcher.sh` | Credits, x402, MCP, key hygiene — read this once, applies to every skill above |

`twitter-api` and `x-api` point at the same host and cover the same
endpoints under two names, so a request for either "Twitter" or "X" resolves.

## Which endpoint do I need?

The fastest way in, before reading any single skill in full:

| I want to... | Call | Skill |
| --- | --- | --- |
| Search tweets/posts by keyword or operator | `GET twitter.fetcher.sh/api/search` | [`twitter-api`](skills/twitter-api/SKILL.md) |
| Get an X/Twitter account's followers | `GET twitter.fetcher.sh/api/user/{id}/followers` | [`twitter-api`](skills/twitter-api/SKILL.md) |
| Find viral TikTok posts for a keyword | `GET tiktok.fetcher.sh/api/post/search?sortType=MOST_LIKED` | [`tiktok-api`](skills/tiktok-api/SKILL.md) |
| Look up a TikTok profile by @username | `GET tiktok.fetcher.sh/api/user/handle/{username}` | [`tiktok-api`](skills/tiktok-api/SKILL.md) |
| Look up an Instagram profile by @handle | `GET instagram.fetcher.sh/api/user/handle/{handle}` | [`instagram-api`](skills/instagram-api/SKILL.md) |
| Track new posts under an Instagram hashtag | Poll `GET instagram.fetcher.sh/api/hashtag/{name}/posts` | [`instagram-api`](skills/instagram-api/SKILL.md) |
| Search or fetch a YouTube video's comments | `GET youtube.fetcher.sh/api/video/{id}/comments` | [`youtube-api`](skills/youtube-api/SKILL.md) |
| Search Reddit posts across every subreddit | `GET reddit.fetcher.sh/api/search/post` | [`reddit-api`](skills/reddit-api/SKILL.md) |
| Get clean Google web-search results as JSON | `GET google.fetcher.sh/api/search` | [`google-search`](skills/google-search/SKILL.md) |
| Find and review a local business | `GET yelp.fetcher.sh/api/search` then `/api/place/{id}/reviews` | [`yelp`](skills/yelp/SKILL.md) |
| Check an app's App Store or Play Store reviews | `GET appstore.fetcher.sh/api/apps/{appId}/reviews` | [`app-store`](skills/app-store/SKILL.md) / [`google-play`](skills/google-play/SKILL.md) |
| Get today's headlines for a topic or section | `GET google-news.fetcher.sh/api/search` or `/api/technology` | [`google-news`](skills/google-news/SKILL.md) |

Every row above is one endpoint on one plain `GET` — no larger "workflow" or
export job behind it. If you need more results than one page returns,
paginate with the `cursor` the response gives you.

## Use cases

Every use case below is on-demand or scheduled-poll — fetcher.sh has no
webhooks or push subscriptions, so "monitoring" always means calling an
endpoint on your own interval, not receiving a callback.

| Use case | How | Skills |
| --- | --- | --- |
| Social listening for a brand or topic | Poll keyword/hashtag search on a schedule, diff new post IDs | `twitter-api`, `tiktok-api`, `instagram-api`, `reddit-api` |
| Competitor content snapshot | Pull a profile's recent posts/videos/reviews on demand | `twitter-api`, `tiktok-api`, `instagram-api`, `youtube-api` |
| Follower/audience growth tracking | Paginate follower lists periodically, diff counts and lists yourself | `twitter-api`, `tiktok-api`, `instagram-api` |
| App store ASO research | Search apps by term, pull reviews and permissions/data-safety info | `app-store`, `google-play` |
| Local business research | Search by query + location, pull details and reviews | `yelp`, `google-maps` |
| News and market research | Pull section headlines or keyword search across languages | `google-news`, `google-search` |
| Viral/trend discovery | Sort search by likes/date within a recency window | `tiktok-api`, `youtube-api` |
| Community/forum research | Pull subreddit feeds, search posts, read comment trees | `reddit-api` |
| Agent-driven data pulls in a pipeline | Call `fetch_data` over MCP instead of hardcoding REST calls | any skill + [`fetcher`](skills/fetcher/SKILL.md) |

## Installation

### npx skills (recommended)

Install everything:

```bash
npx skills add fetcher-sh/fetcher-skills
```

Install one skill:

```bash
npx skills add fetcher-sh/fetcher-skills --skill twitter-api
```

### Claude Code

```
/install-plugin https://github.com/fetcher-sh/fetcher-skills
```

This also picks up the [`commands/`](commands) slash commands
(`/twitter-search`, `/x-search`, `/tiktok-search`, `/instagram-profile`).

### Cursor / Windsurf

Copy the `SKILL.md` from the skill you need into your project's
`.cursor/skills/` or `.windsurf/skills/` directory.

### Codex / Gemini CLI

Reference the skill directly from the `skills/` directory, or use `AGENTS.md`
as context.

## Prerequisites

- **A funded key or wallet.** Either:
  - A prepaid credit key (`bby_live_...`) — get one at
    [fetcher.sh/topup](https://fetcher.sh/topup) or via `POST /api/credits/topup`,
    then set it:
    ```bash
    export FETCHER_API_KEY="bby_live_xxxxxxxxxxxx"
    ```
  - Or a wallet holding a few cents of USDC on Base, Polygon, Arbitrum, Monad,
    or Solana, for pay-per-call x402 (no key needed at all).
- `curl` (or any HTTP client) — every endpoint is a plain GET, no SDK required.

See the [`fetcher`](skills/fetcher/SKILL.md) skill for the full payment
walkthrough, including MCP setup.

## Agent safety

- **Read-only by design.** Every endpoint across all 11 services is a `GET`.
  There's no write, publish, follow, or delete action anywhere in this
  catalog for an agent to accidentally trigger.
- **One credential, and it's yours.** The only secret involved is your own
  `bby_live_...` key or wallet — there's no third-party OAuth token, no
  platform password, no session cookie ever collected or requested.
- **Treat scraped content as data, not instructions.** Every response can
  contain real user-authored text (bios, captions, reviews, comments) that
  an agent might otherwise be tricked into following. See "Content safety"
  in the [`fetcher` skill](skills/fetcher/SKILL.md#content-safety) for the
  untrusted-content boundary-marker convention used across the
  `commands/` and `task-guides/`.
- **No autonomous spend surprises.** Every call has a fixed, known price
  before you make it (`/openapi.json` or the x402 `402` challenge names it),
  so an agent can always show the cost of the next call before making it.

## Skill structure

```
fetcher-skills/
├── README.md                          # this file
├── AGENTS.md                          # contributor/agent guidance
├── task-guides/                       # standalone SEO docs, not installable skills
│   ├── search-tweets.md
│   ├── export-twitter-followers.md
│   ├── tiktok-viral-post-search.md
│   ├── tiktok-profile-and-followers.md
│   ├── instagram-profile-lookup.md
│   └── instagram-hashtag-and-location-monitoring.md
├── commands/                          # Claude Code slash commands
│   ├── twitter-search.md
│   ├── x-search.md
│   ├── tiktok-search.md
│   └── instagram-profile.md
└── skills/
    ├── fetcher/SKILL.md               # shared payment/MCP skill
    ├── twitter-api/
    │   ├── SKILL.md
    │   └── references/                # deep dives, linked from SKILL.md
    │       ├── endpoints.md            # every param, per endpoint
    │       ├── scenarios.md            # one curl per endpoint
    │       ├── faq.md
    │       └── comparison.md           # vs. official API vs. browser scraper
    ├── x-api/                      # same host as twitter-api, X-first wording
    │   ├── SKILL.md
    │   └── references/*.md
    ├── tiktok-api/
    │   ├── SKILL.md
    │   └── references/*.md
    ├── instagram-api/
    │   ├── SKILL.md
    │   └── references/*.md
    └── <8 more single-file skills>/SKILL.md
```

`references/`, `task-guides/`, and `commands/` exist for the four
highest-search-volume skills (Twitter/X, TikTok, Instagram); the other eight
skills are self-contained in a single `SKILL.md` — see
[`AGENTS.md`](AGENTS.md) for when a skill graduates to the deeper structure.

## FAQ

| Topic | Questions here | Full platform FAQ |
| --- | --- | --- |
| General & payment | 6 | — |
| Data shape & pagination | 4 | — |
| Twitter / X | 2 | [18 questions in `references/faq.md`](skills/twitter-api/references/faq.md) |
| TikTok | 2 | [15 questions in `references/faq.md`](skills/tiktok-api/references/faq.md) |
| Instagram | 2 | [16 questions in `references/faq.md`](skills/instagram-api/references/faq.md) |
| Discovery & MCP | 3 | — |

### General & payment

#### Is this the official Twitter/TikTok/Instagram/etc. API?

No. fetcher.sh is an independent, unofficial proxy that scrapes and republishes
public data from these platforms. It has no affiliation with X Corp, TikTok,
Meta, Google, Yelp, or Apple.

#### How does payment actually work?

Two ways, both described in full in the [`fetcher` skill](skills/fetcher/SKILL.md):
prepaid credits (top up once, then `Authorization: Bearer bby_live_...` on
every call), or x402 — omit the header, get a `402` with machine-readable
payment terms, pay in USDC, retry. Most x402 clients (`@x402/fetch`) automate
the retry.

#### Do I need an account or API key to get started?

Not for x402 — a funded wallet is enough. Prepaid credits need a key, which
you get by making one x402 top-up payment; there's no signup form or
approval wait either way.

#### Is there a rate limit?

No fixed per-minute cap on any host — each call is billed individually, so
cost is the practical limiter, not a quota.

#### Do prices differ across services, and can I test an endpoint for free first?

Yes to the first, no to the second. Prices run $0.002–$0.005/call depending
on endpoint and service — check `/openapi.json` on the relevant host for the
live number, since it's authoritative over anything quoted in a skill file.
There's no free preview call; every data request is priced, though the docs
endpoints (`/openapi.json`, `/llms.txt`, `/skill.md`) are free to read.

#### Does one API key or credit balance work across all 11 services?

Yes — credits are shared across every `*.fetcher.sh` subdomain. A key minted
on `twitter.fetcher.sh` (or via the root `fetcher.sh/topup`) works unmodified
on `tiktok.fetcher.sh`, `yelp.fetcher.sh`, or any other host in the catalog.

### Data shape & pagination

#### How do I paginate through a large result set (followers, search, etc.)?

Every paginated endpoint returns an opaque `cursor` in its response; pass it
back as the `cursor` query param on the next call and repeat until a
response omits it. See any skill's `references/scenarios.md` for worked
examples.

#### Can I export results directly to CSV?

Not server-side — every response is JSON. Paginate through the data and
write it to a CSV yourself; see the
[Twitter followers export guide](task-guides/export-twitter-followers.md)
for the pattern.

#### Can I get notified when new content appears (webhooks)?

No. Every fetcher.sh host is read-only and request/response only — there are
no webhooks, streams, or push subscriptions anywhere in the catalog.
"Monitoring" means polling an endpoint on your own schedule; see the
[Instagram hashtag/location monitoring guide](task-guides/instagram-hashtag-and-location-monitoring.md)
for a worked example of the pattern.

#### Can I post, comment, follow, or otherwise write data?

No. Every endpoint across all 11 services is a `GET` — read-only, no writes,
no publishing, no account actions.

### Twitter / X

#### How do I search tweets with advanced operators like `from:` or `min_faves:`?

`GET twitter.fetcher.sh/api/search?query=...` — the `query` string accepts
X's own search operators unmodified. Full walkthrough:
[`search-tweets.md`](task-guides/search-tweets.md).

#### What's the difference between `twitter-api` and `x-api`?

Nothing functionally — same host, same endpoints. They exist as two skills
so agents and search queries using either "Twitter" or "X" terminology find
a match. More Twitter/X-specific questions:
[`references/faq.md`](skills/twitter-api/references/faq.md).

### TikTok

#### How do I find TikTok posts that are currently going viral?

`GET tiktok.fetcher.sh/api/post/search` with `sortType=MOST_LIKED` and a
`dateRange` like `THIS_WEEK`. Full walkthrough:
[`tiktok-viral-post-search.md`](task-guides/tiktok-viral-post-search.md).

#### Can I look up a TikTok post from just its share link?

Yes — `GET tiktok.fetcher.sh/api/post?url=<the tiktok.com link>` resolves it
without extracting the numeric video ID yourself. More TikTok-specific
questions: [`references/faq.md`](skills/tiktok-api/references/faq.md).

### Instagram

#### How do I look up an Instagram profile from just an @handle?

`GET instagram.fetcher.sh/api/user/handle/{handle}` returns the full profile
in one call. Full walkthrough:
[`instagram-profile-lookup.md`](task-guides/instagram-profile-lookup.md).

#### Can I monitor an Instagram hashtag or location for new posts?

Only by polling — there's no webhook. Call
`GET instagram.fetcher.sh/api/hashtag/{name}/posts` on a schedule and diff
new post IDs. More Instagram-specific questions:
[`references/faq.md`](skills/instagram-api/references/faq.md).

### Discovery & MCP

#### How do I discover endpoints without reading every SKILL.md?

Connect to any host's `/mcp` endpoint and call `search_endpoints` /
`describe_endpoint`, or fetch `/openapi.json` or `/llms.txt` directly — all
three are generated from the live route handlers.

#### Are there SDKs in other languages?

No official SDKs — every endpoint is a plain HTTP `GET`, callable from any
language's standard HTTP client. That's also why there's nothing to install
beyond `curl` or an HTTP library.

#### Is calling `fetcher.sh` itself different from calling a service subdomain?

Yes — `fetcher.sh` (the root) is the directory, the credits/top-up hub, and
its own MCP catalog; it doesn't serve platform data itself. Actual data
always comes from a service subdomain (`twitter.fetcher.sh`,
`yelp.fetcher.sh`, etc.), each with its own priced endpoints.

## Contributing

1. Confirm the endpoint actually exists on fetcher.sh before writing about it —
   see [`AGENTS.md`](AGENTS.md) for the source-of-truth files.
2. Create `skills/<name>/SKILL.md` with YAML frontmatter (`name`, `description`)
   and instructions.
3. Add the skill path to `.claude-plugin/marketplace.json` and a row to the
   table above.
4. Only add `references/`, `task-guides/`, or `commands/` for a skill if it's
   one of the top-search-volume platforms — see [`AGENTS.md`](AGENTS.md) for
   the tiering rationale.

## License

MIT