GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - yusukebe/ax: The AI-era curl

BranchesTags
Open more actions menu

Latest commit

 

History

164 Commits
164 Commits

Folders and files

NameName
Last commit message
Last commit date
.github/workflows
.github/workflows
 
 
bench
bench
 
 
nix
nix
 
 
scripts
scripts
 
 
skills/ax
skills/ax
 
 
src
src
 
 
test
test
 
 
website
website
 
 
.gitignore
.gitignore
 
 
.oxfmtrc.json
.oxfmtrc.json
 
 
CLAUDE.md
CLAUDE.md
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
SECURITY.md
SECURITY.md
 
 
bun.lock
bun.lock
 
 
flake.lock
flake.lock
 
 
flake.nix
flake.nix
 
 
package.json
package.json
 
 
package.nix
package.nix
 
 
tsconfig.json
tsconfig.json
 
 

Repository files navigation

ax

The AI-era curl: fetch, discover, extract. One command.

ax is what a coding agent should reach for instead of curl piped into a throwaway parsing script. It fetches a page, helps the agent understand its structure, and extracts structured data — locally, deterministically, with output shaped for a context window.

ax https://api.example.com/users                     # curl parity — but never silent
ax https://example.com --outline                     # discover the page's structure
ax https://example.com '.item' --row 'title=a, href=a@href'
ax https://docs.example.com/guide --md --budget 800  # read docs as markdown

Why

Coding agents do one web loop constantly: fetch → understand → extract. Today that means curl (which prints nothing on an empty body), eyeballing raw HTML in the context window (thousands of tokens), and a regex-over-HTML script that breaks the moment the markup shifts.

ax replaces the loop with one command:

  • Fetch, never silent — {status, ok, url, redirected, ms, headers, body} for every request, including the final URL after redirects. Empty bodies and error statuses still produce a full report. JSON bodies are parsed. Fetch mode never caches — every request is live. (--body gives you the classic pipe: body only on stdout, redirect/status notes on stderr.)
  • Discover, don't dump — --outline shows a page's repeating structures; --locate 'text' answers "which selector holds this?" — no raw HTML ever hits the context. Parse-mode URLs are cached for ~2 minutes so probing is free (every hit announced on stderr; --fresh refetches, --no-cache never touches the disk). Parse requests accept -H, -u, -X, -d and -k; requests with custom headers, a non-GET method, or a body bypass the cache, and -k responses are never written to it.
  • Extract, structured — --row 'title=a, href=a@href' pulls multi-field rows in one call; --table turns <table> into keyed rows; --where filters with a safe expression language.
  • Token-cheap by design — results cap at 50 with a stderr note (never silent truncation), --budget <tokens> targets an estimated token size (cuts at item boundaries, never splits an item, always emits at least one — an oversized item is announced), rows default to header-once TSV (--json for JSON). Truncation notes name the exact --offset to continue from, so a follow-up call emits only what wasn't shown — no overlap, and the parse cache means no refetch.

For machine-readable continuation, add --json-envelope: stdout becomes {data, meta}. Continue only while meta.state is "more", rerunning the same command with --offset <meta.next_offset>; stop on "complete" or "past_end". Existing --json remains a top-level array. Do not restart from zero or increase the budget.

ax page.html '.item' --row 'title=.title' --limit 20 --json-envelope

Install

curl -fsSL https://ax.yusuke.run/install | sh

Teach your agent: npx skills add yusukebe/ax — or have it run ax agent-context.

Nix

The project provides optional Nix flake outputs for users who already use Nix. The flake builds ax from source — see flake.nix and package.nix for details.

# Run without installing
nix run github:yusukebe/ax

# Install into your profile
nix profile install github:yusukebe/ax

Why not htmlq / curl / Firecrawl?

htmlq is a selector; ax is the loop. htmlq covers one step (CSS selector → text) and can't fetch, so every use marries it to curl. Multi-field extraction means running it N times and zipping by hand — exactly the moment agents give up and write python. ax does fetch + discovery + structured rows + tables + filtering in one binary, with agent-shaped output.

curl is silent. An empty 200 body prints nothing; agents re-run it with -i, -w, guessing. ax always reports.

Firecrawl & friends return markdown blobs via metered cloud APIs. ax is local, deterministic, zero-key, and returns structure (rows, tables), not just prose. (For JavaScript-heavy SPAs you still want a browser tool — ax is the fast path for everything else.)

throwaway python curl + htmlq ax
fetch with a full report (status/headers/ms) △ write it ✗ ✓
structure discovery (--outline / --locate) ✗ ✗ ✓
CSS-selector extraction △ needs bs4 ✓ ✓
multi-field rows in one call (--row) △ write it ✗ ✓
<table> → keyed rows △ write it ✗ ✓
easier selector repair after markup drift ✗ (regex) ✓ ✓
page → readable markdown (--md) ✗ ✗ ✓
token-shaped output (caps, --budget, notes) ✗ ✗ ✓
zero code authored per task ✗ ✓ ✓

(△ = possible, but the agent writes and debugs the code every time — that authoring cost is the whole point.)

Measured

Real headless Claude Code sessions, same task, with and without ax — answers graded, both sides correct in every run:

task (Opus 4.8, agent already knows ax) without ax with ax
two pages with markup drift (breaks regex) $0.458 $0.150 (−67%)
clean extraction from a 60-item catalog $0.296 · 24s $0.104 · 14s (−65%)
live website, decoy markup (median of 3) $0.248 $0.191 (−23%)
markup drift, agent's first-ever use of ax $0.664 $0.282 (−58%)

Full method — including the runs ax lost — in bench/RESULTS.md.

The full flag reference

Run ax --help, or ax agent-context for the agent-oriented playbook (also served at ax.yusuke.run/llms.txt).

Built with

Bun (single-file binary via bun build --compile) and linkedom for standard-DOM parsing — the only runtime dependency.

About

The AI-era curl

ax.yusuke.run/

Resources

Readme
MIT license

Security policy

Security policy
Activity

Stars

736 stars

Watchers

5 watching

Forks

11 forks
Report repository

Releases

Contributors

Languages

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.