A scraper API free trial is easy to waste. If you only send example.com a few times, you will learn almost nothing about the pages you actually need, and a 200 status code is not the same thing as usable content.
This is a 48-hour test plan: a URL list from your real workload, a definition of success you write down before you run anything, a small Python script that records success rate and p95 latency, and a short cost projection for after the trial. You can run it on Geonode Scraper API. The current free plan is 1,500 requests per month, with no credit card, and failed requests are not charged (live pricing, 11 November 2026). The API docs cover authentication and the extract endpoint.
If you only need parsed Google or Bing results, look at Geonode Search as well. This checklist is for extracting web pages.
What you are measuring
Write the pass rule down before the first request. A useful evaluation has four numbers, and only the first one is a status code.
| Metric | How to measure it | What "good" means |
|---|
| Success rate | HTTP 2xx and the body contains content that belongs on that page, not a challenge or empty shell | Compare providers (or plans) on the same URL list; do not treat a vendor demo rate as yours |
| Output quality | Spot-check Markdown or HTML: is the main content there, are headings and links intact, is boilerplate stripped if you asked for that? | The fields you need are present and stable across repeats |
| Latency | p50 and p95 on first attempts, in milliseconds, with render-on and render-off separated | Seconds for a rendered page can be normal; a minute usually is not |
| Cost per successful page | Requests billed ÷ successful pages, then apply the live unit price | This is the number that matters after the trial, not the headline "per 1k" |
HTTP 200 with a "just a moment" challenge page, a login wall, or an empty app shell is a failure. If you are unsure what the page should contain, open it in a browser and write down one stable marker — a product title, a heading, a unique string — before you start.
Day 0: build the test set (about an hour)
- Pick 30 to 100 URLs you are allowed to fetch, drawn from the work you actually do. Weight the list toward the domains that will make up most of your volume.
- Include easy pages, JavaScript-heavy pages, and a few known-hard cases. Also include one or two dead or redirecting URLs so you can see how failures are reported.
- For each URL, record: the expected marker or selector, whether it needs JavaScript rendering, country if it matters, and a note on robots.txt / terms. Skip any target you do not have a legitimate reason to request.
- Freeze the list. Do not swap in friendlier URLs mid-test because the first results looked bad.
Geonode's free plan currently allows 1,500 requests per month at concurrency 1. That is enough for a first-pass list plus a few repeats. If you also want to try an Unlimited plan, the live product page offers a $100, 48-hour Unlimited trial — that is a paid trial, not the free monthly allowance. Use the free 1,500 first unless you already know you need the higher concurrency.
A Python script you can run in the trial
The script below calls the documented extract endpoint (POST https://scraper.geonode.io/v1/extract, API key in X-Api-Key — see Authentication & Base URL). It does not ship with a key. Put the key in GEONODE_SCRAPER_API_KEY.
It treats a request as a success only when the HTTP status is 2xx, the body is non-empty, and none of a small set of challenge markers appear. If you added an expected substring in the URL list, that must be present too. Latency is the wall time of each first attempt. Failed requests should not count toward a Geonode allowance, but you should still log them.
#!/usr/bin/env python3
"""First-pass scraper API evaluation. Not a published benchmark."""
import csv
import json
import os
import sys
import time
from pathlib import Path
import urllib.request
import urllib.error
API_URL = "https://scraper.geonode.io/v1/extract"
API_KEY = os.environ.get("GEONODE_SCRAPER_API_KEY")
CHALLENGE_MARKERS = (
"just a moment",
"attention required",
"verify you are human",
"cf-browser-verification",
"access denied",
)
def load_targets(path):
rows = []
with open(path, newline="", encoding="utf-8") as f:
reader = csv.DictReader(f)
for row in reader:
url = (row.get("url") or "").strip()
if not url or url.startswith("#"):
continue
rows.append({
"url": url,
"marker": (row.get("marker") or "").strip(),
"render_js": (row.get("render_js") or "").strip().lower() in {"1", "true", "yes"},
})
return rows
def extract(url, render_js=False, timeout=60):
payload = {"url": url, "formats": ["markdown", "html"]}
if render_js:
payload["render_js"] = True
data = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
API_URL,
data=data,
headers={
"X-Api-Key": API_KEY,
"Content-Type": "application/json",
},
method="POST",
)
started = time.perf_counter()
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
body = resp.read().decode("utf-8", "replace")
status = resp.status
except urllib.error.HTTPError as exc:
body = exc.read().decode("utf-8", "replace")
status = exc.code
except Exception as exc:
return {"status": 0, "ms": int((time.perf_counter() - started) * 1000),
"ok": False, "reason": f"transport: {exc}", "chars": 0}
ms = int((time.perf_counter() - started) * 1000)
return {"status": status, "ms": ms, "body": body}
def is_success(result, marker=""):
if result.get("status", 0) < 200 or result.get("status", 0) >= 300:
return False, f"http {result.get('status')}"
body = result.get("body") or ""
if not body.strip():
return False, "empty body"
lowered = body.lower()
for token in CHALLENGE_MARKERS:
if token in lowered:
return False, f"challenge marker: {token}"
if marker and marker.lower() not in lowered:
return False, "expected marker missing"
return True, "ok"
def percentile(values, p):
if not values:
return None
xs = sorted(values)
idx = min(len(xs) - 1, max(0, int(round((p / 100) * (len(xs) - 1)))))
return xs[idx]
def main():
if not API_KEY:
sys.exit("Set GEONODE_SCRAPER_API_KEY to your Scraper API key.")
targets_path = sys.argv[1] if len(sys.argv) > 1 else "targets.csv"
targets = load_targets(targets_path)
if not targets:
sys.exit(f"No URLs in {targets_path}")
results = []
for item in targets:
raw = extract(item["url"], render_js=item["render_js"])
ok, reason = is_success(raw, item["marker"])
row = {
"url": item["url"],
"render_js": item["render_js"],
"status": raw.get("status"),
"ms": raw.get("ms"),
"chars": len(raw.get("body") or ""),
"ok": ok,
"reason": reason,
}
results.append(row)
print(f"{'OK' if ok else 'FAIL':4} {row['status']:>3} {row['ms']:>6}ms {row['reason']:<28} {item['url']}")
out = Path("eval-results.csv")
with out.open("w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=list(results[0].keys()))
writer.writeheader()
writer.writerows(results)
ok_rows = [r for r in results if r["ok"]]
all_ms = [r["ms"] for r in results if r["ms"] is not None]
ok_ms = [r["ms"] for r in ok_rows if r["ms"] is not None]
print("\n--- summary ---")
print(f"targets: {len(results)}")
print(f"successes: {len(ok_rows)} ({(len(ok_rows)/len(results)*100):.1f}%)")
print(f"p50 all: {percentile(all_ms, 50)} ms")
print(f"p95 all: {percentile(all_ms, 95)} ms")
print(f"p50 success: {percentile(ok_ms, 50)} ms")
print(f"p95 success: {percentile(ok_ms, 95)} ms")
print(f"wrote {out}")
if __name__ == "__main__":
main()
A starter targets.csv:
url,marker,render_js
https://example.com,Example Domain,false
https://geonode.com,Geonode,false
Replace those rows with your own authorized URLs and markers. Run:
export GEONODE_SCRAPER_API_KEY="YOUR_API_KEY"
python eval_scraper_api.py targets.csv
Get the key from the Scraper API dashboard (Get Code → Copy API Key), as described in the authentication docs. Do not commit the key.
This script is a first-attempt logger. It is not a published benchmark. For a harder definition of success (main content, tables, truncated output, billed cost), see the methodology work on the separate benchmark issue; do not copy any unverified headline rate into your notes.
The 48-hour schedule
Hours 0–2. Freeze the URL list and pass rules. Run the script once with render_js off except where you already know the page is a client-rendered app.
Hours 2–8. Re-run the failures with rendering on, if the docs for your plan include JavaScript rendering. Record how many pages only succeed when rendered — that changes cost and latency. Spot-check ten successful bodies against the live page.
Hours 8–24. Repeat the same list at least once more, ideally on a second day, so you are not judging a single blip. Keep first attempts separate from retries.
Hours 24–48. Compute success rate, p50/p95, and cost per success. On Geonode, paid request-based plans start at 10,000 requests for $3.90/month and fall to $0.20 per 1,000 on the 5M plan; pay-as-you-go starts at $0.40 per 1,000 with a $50 minimum; unused purchased requests roll over; failed requests are not charged (all figures from the live product and pricing pages, 11 November 2026). Project your monthly volume with those unit prices, then re-read the live card before you buy — these numbers change.
If you need to compare vendors, send the same URLs in the same window. A trial of vendor A this week against vendor B last month is not a comparison.
Frequently Asked Questions
How do I test a scraper API free trial properly?
Use your own URLs, write a pass rule that includes page content, run first attempts more than once, and record success rate, p95 latency, and billed cost per success. A handful of requests to a demo page is not a test.
Does Geonode require a credit card for the free plan?
No. The live product page (11 November 2026) offers 1,500 requests per month with no credit card. Failed requests are not charged. There is a separate paid Unlimited trial at $100 for 48 hours.
What counts as a successful request?
For this checklist: a 2xx response, a non-empty body, no challenge page, and — when you provided one — the expected marker. HTTP 200 alone is not enough.
Should I enable JavaScript rendering on every request?
No. Turn it on when the unrendered HTML is missing the content you need. Rendering is slower and may cost more on some plans. Measure both ways on the pages that actually require it.
Where do I find the request format?
Authentication & Base URL and the extract examples in the Scraper API docs. The production base URL is https://scraper.geonode.io. Send the key in X-Api-Key.
After the 48 hours
You should have a CSV of first-attempt results, a success rate you could explain to someone else, a p95 you trust, and a cost projection based on the live price card rather than a blog memory.
If the API clears your list, start from the Scraper API product page and keep the script as a regression check. If it does not, the CSV will tell you whether the problem is rendering, location, challenges, or the targets themselves — which is more useful than a marketing comparison table.
For a longer discussion of retries and status codes, a draft already exists under the error-handling article in this cluster. For whether you even need an API instead of raw proxies, that is the next planned piece in the series.
