We sell proxies at Geonode, so this is our own product being explained. The recommendation that costs us nothing and helps most people: if your source address is stable, use IP allowlisting rather than a username and password. It removes the credential from your commands, your scripts, your environment variables and your pasted terminal output entirely — and there is nothing to leak if there is nothing to send. Most providers offer it, including us, and it is under-used because username-and-password is what the setup guide shows first. The rest of this article covers both, plus why SOCKS5 authentication deserves more caution than it usually gets.
The Two Methods Providers Offer
Username and password. You are issued credentials and send them with each request. Works from any address, which makes it the only option when your address changes — a laptop, a mobile connection, a container with a dynamic address, or a distributed set of workers.
IP allowlisting. You register the addresses your requests will come from, and the proxy accepts anything from them without credentials. Works only from those addresses, which is exactly the property that makes it safe.
Most commercial providers support both, and many allow both simultaneously. The trade-off is straightforward:
| Credentials | IP allowlist | |
|---|---|---|
| Works from anywhere | Yes | No |
| Something to leak | Yes | No |
| Survives an address change | Yes | Needs updating |
| Suits containers and CI | Yes | Only with a stable egress address |
| Suits a fixed server | Yes | Better |
For a scraper running on a server with a static address, the allowlist is strictly better. For anything mobile or ephemeral, credentials are the only workable option. For CI runners, it depends entirely on whether your platform gives you a predictable egress address, and many do not.
How HTTP Proxy Authentication Works
The mechanism is a challenge and a response, defined in RFC 9110.
You send a request. If the proxy requires authentication and you did not supply it, the proxy answers 407 Proxy Authentication Required and includes a challenge. The specification is firm about this: "A proxy MUST send at least one Proxy-Authenticate header field in each 407 (Proxy Authentication Required) response that it generates."
HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: Basic realm="proxy.example.com"
You resend with credentials in a Proxy-Authorization header, which the RFC describes as allowing "the client to identify itself (or its user) to a proxy that requires authentication".
Proxy-Authorization: Basic dXNlcjpwYXNz
Two properties distinguish this from ordinary web authentication, and both come straight from the specification.
The challenge is hop-specific. "Unlike WWW-Authenticate, the Proxy-Authenticate header field applies only to the next outbound client on the response chain. This is because only the client that chose a given proxy is likely to have the credentials necessary for authentication."
The credentials are consumed rather than forwarded. "When multiple proxies are used in a chain, the Proxy-Authorization header field is consumed by the first inbound proxy that was expecting to receive credentials." A proxy may relay them onward if the proxies cooperate that way, but by default your credentials stop at the first hop that wanted them.
The RFC also notes a consequence for chains inside one organisation: when several proxies in the same administrative domain issue the same challenge, "it will appear as if Proxy-Authenticate is being forwarded because each proxy will send the same challenge set."
407 Versus 401: The Distinction That Matters
The most useful thing in this article for anyone debugging.
| Status | Who is asking | Response header | Request header |
|---|---|---|---|
| 401 Unauthorized | The target server | WWW-Authenticate | Authorization |
| 407 Proxy Authentication Required | The proxy | Proxy-Authenticate | Proxy-Authorization |
Different codes, different headers, different credentials, different fix.
A 407 means you never reached the target. The proxy stopped you. Your target credentials are irrelevant, and no amount of adjusting them will help.
A 401 means the proxy worked and the target wants credentials.
They can appear together in one setup, because two independent credential sets may be in play. In curl:
curl -x http://proxy.example.com:9000 \
--proxy-user proxyuser:proxypass \
-u apiuser:apipass \
https://api.example.com/private
--proxy-user for the proxy, -u for the target. Crossing them produces exactly the confusion this section exists to prevent.
To see which hop refused you without guessing:
curl -sS -o /dev/null -x "$PROXY" \
-w 'connect=%{http_connect} status=%{response_code}\n' \
https://example.com
connect=407 means the proxy rejected you and never reached the target. connect=200 status=401 means the proxy worked and the target wants credentials. Two different problems, two different fixes, one command to tell them apart.
SOCKS5 Authentication Is Different, and Weaker
Worth knowing because the difference is a genuine security consideration and it is rarely mentioned.
SOCKS5 does not use HTTP headers. Authentication happens during connection setup, in a subnegotiation defined by RFC 1929. The client sends a small binary structure containing a version byte, the username length, the username, the password length and the password, and the server replies with a status byte where X'00' indicates success. If the server returns a failure status, it "MUST close the connection".
The security note in that RFC is short and should be read carefully:
Since the request carries the password in cleartext, this subnegotiation is not recommended for environments where "sniffing" is possible and practical.
Clear text, not base64, not hashed. HTTP Basic authentication is at least base64-encoded — trivially reversible, but not literally readable in a packet capture. SOCKS5 username/password authentication puts the password on the wire as bytes.
The practical implications:
On an untrusted network, prefer an HTTP proxy over TLS, or an allowlist. The credential is more exposed with SOCKS5 than with HTTP Basic, and the difference matters on shared or hostile networks.
Rotate SOCKS credentials more readily than you would HTTP ones, since exposure is easier.
Note that this concerns authentication to the proxy only. Your onward traffic to an HTTPS site is still protected by TLS. It is the credential itself that travels unprotected.
Parameters in the Username: A Provider Convention
A pattern that confuses newcomers and is not part of any standard.
Many providers encode configuration inside the username string:
username-country-de-session-abc123:password
That is not an HTTP feature. The proxy gateway parses its own username field and treats the extra segments as instructions — a country, a session identifier, a rotation setting. The syntax varies completely between providers.
Two things follow.
Read your provider's documentation rather than guessing. A malformed parameter usually does not produce an error. It produces a working request with the wrong behaviour — a session that does not persist, or an exit in a country you did not ask for. That is a silent failure, and it is the kind that survives longest.
Some providers use ports instead, mapping a range of ports to countries or sessions rather than encoding them in the username. Neither approach is better; you simply have to know which one you bought.
Where Credentials Leak
Four places, all common, all avoidable.
Command lines. Visible in the process list to other users on the machine, and stored in shell history indefinitely. curl's manual is direct about it: sensitive data "should be retrieved from a file instead or similar and never used in clear text in a command line."
Environment variables. http_proxy=http://user:pass@host:9000 is the standard configuration idiom and it puts the password in the environment of every child process, in /proc on Linux, and in any environment dump.
Verbose output. curl -v includes the Proxy-Authorization header, and screenshots of terminals reach places nobody intended. Redact before sharing.
Source control. Credentials hard-coded in a script and committed remain in the repository history after removal.
Mitigations, in order of effectiveness: use an IP allowlist and have no credential at all; put credentials in a permission-restricted file such as ~/.netrc with chmod 600; read them from a secret manager at runtime; and at minimum keep them out of shell history with read -rs.
One encoding detail that causes real confusion: if your password contains @, : or /, it must be percent-encoded before going into a proxy URL, or the parser will split in the wrong place and you will get a 407 with correct credentials.
Configuring It in Common Tools
curl:
curl -x http://proxy.example.com:9000 --proxy-user user:pass https://example.com
wget — note that it has no --proxy flag, so the proxy itself comes from the environment or .wgetrc:
wget --proxy-user=user --proxy-password=pass https://example.com
Better, in ~/.wgetrc with chmod 600:
http_proxy = http://proxy.example.com:9000/
proxy_user = user
proxy_password = pass
Python requests:
proxies = {"http": "http://user:pass@proxy.example.com:9000",
"https": "http://user:pass@proxy.example.com:9000"}
requests.get("https://example.com", proxies=proxies)
Playwright:
const context = await browser.newContext({
proxy: { server: 'http://proxy.example.com:9000', username: 'u', password: 'p' },
});
Note that Playwright takes the credentials as separate fields rather than in the URL, which avoids the percent-encoding problem entirely.
Environment variables, honoured by many tools:
export http_proxy=http://user:pass@proxy.example.com:9000
export https_proxy=http://user:pass@proxy.example.com:9000
export no_proxy=localhost,127.0.0.1,.internal
Use lowercase. Uppercase HTTP_PROXY has a documented hazard in CGI environments, where request headers become uppercase environment variables and a client honouring it can be steered by an attacker-supplied Proxy: header.
Troubleshooting
407 with credentials you believe are correct. Check for special characters needing percent-encoding in the password. Check whether you are also on an IP allowlist that has expired. Check that the tool is sending the credentials at all — curl -v ... 2>&1 | grep -i proxy-auth shows you.
407 that appears intermittently. Usually a distributed setup where some workers have a different egress address than the one on your allowlist, or a rotating gateway where only some endpoints require authentication.
Works with curl, fails in your application. The library may not support proxy authentication for HTTPS, or may not be applying the proxy at all. Check whether your requests are actually going through it — a service that echoes your address is the fastest test.
Works for HTTP, fails for HTTPS. For HTTPS, the client issues a CONNECT first, and some libraries handle proxy authentication for the tunnel differently or not at all.
Authentication succeeds, requests still fail. Then it was never an authentication problem. A 403 after a successful CONNECT comes from the target, not the proxy.
Managing Credentials Across a Team or a Fleet
Once more than one person or one machine is involved, credential handling stops being a convenience question and becomes an operational one.
Issue separate credentials per person and per service where your provider allows it. A single shared account means you cannot tell whose job caused a usage spike, cannot revoke access for one departing colleague without disrupting everyone, and cannot attribute a leak. Sub-accounts are worth asking about even when they cost a little more.
Keep credentials out of images and out of source. A password baked into a container image is present in every layer and every registry copy of it, including after you remove it in a later build. Inject at runtime through the orchestrator's secret mechanism, or fetch from a secret manager at start-up.
Prefer an allowlist for anything with a stable egress. A fixed server, a NAT gateway with a static address, or a Kubernetes cluster behind a known egress IP can all use an allowlist and hold no credential at all. That is a strictly better position than any amount of careful credential handling.
Plan for rotation before you need it. Read the credential from a single place — an environment variable populated by a secret manager, or a config file with restricted permissions — so that rotating it is one change rather than a search across a codebase. The moment you actually need to rotate is the moment you least want to be grepping.
Watch for the accidental-commit failure. A credential committed and then removed remains in the repository history, and a public repository means it is compromised regardless of how quickly the commit was reverted. Repository scanning is cheap insurance, and rotating immediately is the only real remedy.
And instrument usage per credential. If your provider reports traffic by sub-account, a sudden change is the earliest signal you will get that a credential has been shared, leaked, or is being used by a job you forgot about. That is also the cheapest way to catch the most expensive kind of mistake — a runaway loop billing bandwidth you did not intend to buy.
People Also Ask
What is a 407 error?
407 Proxy Authentication Required means the proxy wants credentials and did not receive valid ones. It comes from the proxy rather than the website, and RFC 9110 requires the proxy to include a Proxy-Authenticate header naming the scheme it expects. Your target credentials are irrelevant to it.
What is the difference between 401 and 407?
401 comes from the target server and uses WWW-Authenticate and Authorization. 407 comes from the proxy and uses Proxy-Authenticate and Proxy-Authorization. A 407 means you never reached the target at all.
Is IP allowlisting better than a username and password?
Where your source address is stable, yes. There is no credential to leak, nothing to appear in your shell history, and nothing to commit to a repository by accident. It fails when your address changes, which is why credentials remain necessary for laptops, mobile connections and most CI environments.
Are proxy credentials encrypted?
HTTP Basic proxy authentication base64-encodes them, which is reversible but not plainly readable. SOCKS5 username/password authentication sends them in clear text — RFC 1929 says so explicitly and advises against it "where sniffing is possible and practical". Neither is encryption; the transport is what protects you.
Why does my proxy password with a special character fail?
Because @, : and / have meaning in a URL. http://user:p@ss@host:9000 parses in a way you did not intend. Percent-encode them, or use a client that takes username and password as separate fields rather than embedding them in the URL.
What does the extra text in my proxy username mean?
It is a provider convention for encoding options — a country, a session identifier, a rotation setting — into the username field. It is not part of any standard, the syntax differs by provider, and a mistake usually produces a working request with the wrong behaviour rather than an error.
Can I use both credentials and an allowlist?
With many providers, yes, and it is a sensible arrangement: the allowlist covers your fixed servers with no credential in play, while credentials cover developer machines and anything with a changing address.
Do I need separate credentials for the proxy and the website?
If both require authentication, yes — they are entirely separate mechanisms with separate headers. In curl that is --proxy-user for the proxy and -u for the target, and mixing them up produces a 407 when you expected a 401 or the reverse.
Wrapping Up
Proxy authentication is a separate layer from website authentication, and the protocol was designed to make that obvious. Different status code, different headers, different credentials. Once you read 407 as "the proxy stopped me" and 401 as "the target wants credentials", a whole class of confusing failures resolves itself.
The mechanism itself is simple: a challenge in Proxy-Authenticate, a response in Proxy-Authorization, consumed by the first hop that asked. The wrinkle worth remembering is SOCKS5, where the specification is candid that credentials travel in clear text — a reason to prefer an HTTP proxy or an allowlist on any network you do not control.
And the recommendation that costs a vendor nothing to make: if your requests come from a stable address, use an IP allowlist. Every leak described here — shell history, process lists, environment dumps, committed source, pasted terminal output — depends on there being a credential to leak. Remove the credential and you remove the category.