Hyperwatch is an open source, real-time access log processor. Stream logs from your CDN, load balancer or web server, enrich every request, and spot bots, AI crawlers and abuse as it happens.
The live log stream at /logs/main, with identities, verified hostnames and firewall matches. Sample data.
Ingests access logs from wherever your traffic lands
CDNs
Cloudflare · CloudFront · Akamai
Load balancers
ELB
Reverse proxies
Nginx · HAProxy
Web servers
Nginx · Apache
Applications
Node.js · Express · Ruby · PHP
oversyslogTCP / UDPfile tailHTTPWebSocketin nginx, Apache or JSON formats, or your own parser
Features
Everything you need to read your traffic
Raw access logs become enriched, queryable events: who is calling, from where, how often, and whether they're human.
A real-time stream pipeline
Every log flows through the pipeline the moment it's written. Map, filter and branch streams with plain JavaScript, and register as many inputs as you need.
// config/custom.js
pipeline
.getNode('main')
.filter((log) => log.getIn(['response', 'status']) >= 500)
.registerNode('errors');
// streams live at /logs/errors
Enrichment modules
Resolve and verify hostnames, geolocate addresses, check DNS blocklists and parse User-Agents.
geoip
hostname
dnsbl
identity
agent
language
signature
history
AI crawler detection
Bundled, refreshable IP lists tell real AI agents and CDN edges apart from impostors spoofing their User-Agent.
GPTBot
ClaudeBot
ChatGPT-User
OAI-SearchBot
Amazonbot
Cloudflare
CloudFront
Aggregations API
Live rankings of addresses, signatures and identities, as JSON, CSV or HTML. Sort and limit them with query params.
GET /addresses.json?sort=count15m GET /signatures.csv?limit=50 GET /identities
Live logs anywhere
Every pipeline node streams over HTTP and WebSocket. Tail it in a browser, grep it with a query param, or pipe it into your own tools.
GET /logs/main?grep=GPTBot WS /logs/errors
How it works
From access log to insight in milliseconds
One lightweight Node.js process sits next to your infrastructure. No agents to install on clients, no database to run.
01
Ingest
Tail files, or listen over syslog, TCP/UDP, HTTP and WebSocket. Your Express app can push logs through middleware.
file
syslog
socket
http
websocket
02
Parse
Paste your nginx or Apache log_format as is, send JSON, or plug in your own parser. Every log becomes an immutable map.
module.exports = function (hyperwatch) { const { pipeline, input, format } = hyperwatch; hyperwatch.init(); // Tail nginx logs in the richer hyperwatch_combined format const nginx = input.file.create({ path: '/var/log/nginx/access.log', parse: format.nginx.parser({ format: format.nginx.formats.hyperwatch_combined, }), }); pipeline.registerInput(nginx);};
$ hyperwatch config/custom.js
# Top addresses over the last 15 minutescurl "localhost:3000/addresses.json?sort=count15m&limit=10"# Export identities as CSVcurl -O "localhost:3000/identities.csv"# Tail only AI crawler trafficcurl -N "localhost:3000/logs/main?grep=GPTBot"
$ open http://localhost:3000/logs/main
# Check firewall.json for errorshyperwatch firewall check# Pull edits made in the Cloudflare dashboard (preview)CLOUDFLARE_API_TOKEN=... CLOUDFLARE_ZONE_ID=... \ hyperwatch firewall sync down --dry-run# Push local changes for one list to its custom rulehyperwatch firewall sync up --list block-ips
$ hyperwatch firewall sync down && hyperwatch firewall sync up
New · Firewall
Hyperwatch spots it. Your edge blocks it.
Keep IP and User-Agent lists in a single firewall.json. Hyperwatch tags matching requests in real time and keeps each list in sync with a Cloudflare WAF custom rule. Nothing is blocked on your origin.
Block, challenge or monitor
Pick an action per list. Monitor lists only tag logs locally.
Sync up and down
Push your changes to Cloudflare, or pull edits made in the dashboard into your file.
Safe by default
Hand-edited rules and lists over the 4,096-character limit are refused, with a reason.
HTTP API
Look up, add or remove entries from your own tools and dashboards.