Scrape URL to Markdown (rendered)

The same clean, LLM-ready markdown as scrape-markdown, except the page is loaded in a real browser first. Use it on anything that builds its content client-side — single-page apps, dashboards, most modern pricing pages — where a plain HTTP fetch returns an empty shell. Optional CSS selector to scope the extraction, and an optional selector to wait for before the page is read.

POST https://api.bottrunk.com/s/scrape-markdown-js

Add to my agent

Overview Call it Behaviour Input Output

Call it

# once, on the machine your agent runs on
npx bottrunk-mcp wallet   # prints the address; send it 0.3 ALGO, then USDC
claude mcp add bottrunk -- npx -y bottrunk-mcp

# then ask your agent for it — the tool is `bottrunk_scrape_markdown_js`

# Ask. You get a 402 with the payment requirements.
curl https://api.bottrunk.com/s/scrape-markdown-js \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com/app","wait_for":"table.results"}'

# Pay and retry. Any x402 client does this for you.
curl https://api.bottrunk.com/s/scrape-markdown-js \
  -H "PAYMENT-SIGNATURE: <signed USDC transfer>" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com/app","wait_for":"table.results"}'

# pip install "x402-avm[avm,requests]==2.0.2"
import base64, os
from algosdk import encoding, mnemonic
from x402 import x402ClientSync
from x402.http import x402HTTPClientSync
from x402.http.clients.requests import x402_requests
from x402.mechanisms.avm import ALGORAND_MAINNET_CAIP2
from x402.mechanisms.avm.exact import ExactAvmScheme

class MnemonicSigner:  # signs the payment with your agent's 25-word mnemonic
    def __init__(self, words):
        self._sk = mnemonic.to_private_key(words)
        self.address = encoding.encode_address(base64.b64decode(self._sk)[32:])

    def sign_transactions(self, unsigned_txns, indexes_to_sign):
        return [base64.b64decode(encoding.msgpack_encode(
                    encoding.msgpack_decode(base64.b64encode(t).decode()).sign(self._sk)))
                if i in indexes_to_sign else None
                for i, t in enumerate(unsigned_txns)]

# A wallet holding 0.3 ALGO, opted in to USDC, funded with the USDC it may spend.
signer = MnemonicSigner(os.environ["ALGORAND_MNEMONIC"])
client = x402ClientSync().register(ALGORAND_MAINNET_CAIP2,
    ExactAvmScheme(signer, algod_url="https://mainnet-api.algonode.cloud"))

with x402_requests(client) as session:   # answers the 402, pays, retries
    r = session.post("https://api.bottrunk.com/s/scrape-markdown-js", json={"url": "https://example.com/app", "wait_for": "table.results"})
    print(r.json())
    receipt = x402HTTPClientSync(client).get_payment_settle_response(r.headers.get)
    print(receipt.transaction)            # the settlement on-chain

// npm install @x402-avm/fetch @x402-avm/avm @x402-avm/core algosdk
import algosdk from "algosdk";
import { x402Client } from "@x402-avm/core/client";
import { ExactAvmScheme } from "@x402-avm/avm/exact/client";
import { toClientAvmSigner } from "@x402-avm/avm";
import { wrapFetchWithPayment } from "@x402-avm/fetch";

// A wallet holding 0.3 ALGO, opted in to USDC, funded with the USDC it may spend.
const { sk } = algosdk.mnemonicToSecretKey(process.env.ALGORAND_MNEMONIC!);
const signer = toClientAvmSigner(Buffer.from(sk).toString("base64"));
const client = new x402Client().register("algorand:wGHE2Pwdvd7S12BL5FaOP20EGYesN73ktiC1qzkkit8=",
  new ExactAvmScheme(signer, { algodUrl: "https://mainnet-api.algonode.cloud" })); // default is TestNet
const fetchWithPay = wrapFetchWithPayment(fetch, client);

const r = await fetchWithPay("https://api.bottrunk.com/s/scrape-markdown-js", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({"url":"https://example.com/app","wait_for":"table.results"}),
});
console.log(await r.json());

How it behaves

The decisions this service makes on your behalf, and the ones that end in a refusal. A 4xx is never settled, so a refusal costs you nothing.

When to pay for this	Only when plain scrape-markdown comes back thin or empty. If the page ships its content in the HTML, that one is a third of the price and returns the same markdown.
What is stripped	Identical to scrape-markdown: script, style, noscript, nav, footer, header, aside, form, iframe, svg, template — same pipeline, different source.
Waiting for content	wait_for takes a CSS selector the browser waits for before the page is read — the difference between catching a dashboard mid-spinner and catching it after the data arrives.
A selector that matches nothing	Refused with 422 and not charged, exactly as on scrape-markdown.
If the renderer is unavailable	The call fails and is never settled, so it costs you nothing.
Size and time	Pages over 2 MB are refused with page too large. Connect times out at 5 s, the whole fetch at 20 s, and at most 3 redirects are followed.
Who we look like	Requests go out as BotTrunk/0.1 (+https://bottrunk.com/docs). Sites that block unknown agents will block this one.
Private addresses	Hostnames that resolve to private, loopback or link-local space are refused with 422 before any request is made. Do not point this at localhost or an intranet.
Method	POST only. A GET on the same path returns this page.

Input

url	string	Public http(s) URL to render.
wait_for	string	Optional CSS selector to wait for before reading the page.
selector	string	Optional CSS selector to scope the extraction.

Output

markdown	string	Body content as GitHub-flavored markdown.
title	string	Document title.
word_count	integer	Words in markdown, for budgeting tokens.