About our fetcher
If you run a transit agency's web server, this page explains the traffic this project sends you: what it requests, how often, what it looks like in your logs, and how to reach us. Every scorecard also records exactly how its feed copy was obtained, so what we state publicly here matches what each artifact states about itself.
What we request, and how often
- Your GTFS Schedule zip, once a day. The daily scoring run downloads each configured feed one time, in the early afternoon UTC.
- Cheap liveness checks in between. Hourly refreshes use
conditional requests (
If-None-Match/If-Modified-Since), so an unchanged feed answers with an empty 304 and transfers nothing. - GTFS-Realtime in short bursts. Realtime endpoints are sampled in a brief burst every three hours, never faster than one request every 30 seconds per feed, matching the spec's own polling guidance.
Requests come from GitHub Actions runners, so they do not arrive from a fixed set of addresses. Downloads are capped in size, follow at most five redirects, and only ever touch the URLs configured for your agency in the public registry.
The User-Agent, stated honestly
Our requests currently present a standard desktop-browser User-Agent (Chrome on Linux) rather than a custom bot string. That is a deliberate, documented choice: many transit feeds sit behind bot filters that answer a non-browser User-Agent with a 403, and a feed we cannot read becomes a feed we cannot help fix. We would rather state the practice plainly here than present a bot string and silently lose the agencies this tool exists for.
An honest-first experiment (try a
gtfs-scorecard/<version> User-Agent, fall back to the browser
string, and record which one succeeded) is on the roadmap so this choice can rest on
measured data. If it ships, this page will say so.
Retries and the mirror fallback
A failed download is retried a small number of times with exponential backoff, and
only on responses that usually mean "try again" (403 from a web application firewall,
408, 425, 429, and 5xx). If your endpoint still cannot be reached, we fall back to the
MobilityData hosted mirror of your feed rather than drop your agency, and the scorecard
then says so: the artifact's fetch block records source: mirror, and the
agency page notes the copy that was scored. A feed that cannot be fetched at all is
shown as unreachable with its last known grade, never as a zero.
If our traffic is a problem
Tell us and we will fix it. The fastest route is a GitHub issue; the correct-or-claim form works too, and the listing policy explains how to update your feed URL or ask not to be listed at all. If you prefer we score the MobilityData mirror instead of your origin server, that is a one-line registry change.