🚀 Production-Ready Async Proxy Management - v2.0.0b1
The modernized successor to ProxyBroker with Python 3.10-3.13 support, zero critical bugs, and 100% test coverage
ProxyBroker2 is an open source tool that asynchronously finds public proxies from 50+ sources, validates them against judge servers, and can operate as a rotating proxy server.
- Finds more than 7000 working proxies from ~50 sources.
- Support protocols: HTTP(S), SOCKS4/5. Also CONNECT method to ports 80 and 23 (SMTP).
- Proxies may be filtered by type, anonymity level, response time, country and status in DNSBL.
- Work as a proxy server that distributes incoming requests to external proxies. With automatic proxy rotation.
- All proxies are checked to support Cookies and Referer (and POST requests if required).
- Automatically removes duplicate proxies.
- Is asynchronous.
- ✅ Zero Critical Bugs - Fixed all signal handler leaks, deadlocks, and heap corruption
- ✅ 100% Test Coverage - All 131 tests passing with comprehensive validation
- ✅ Modern Async Patterns - Updated from deprecated asyncio patterns for Python 3.10-3.13
- ✅ Type Safety - Enhanced API with proper type validation and error handling
- ✅ Resource Management - Comprehensive cleanup and connection management
- ✅ Cache Optimization - Smart scheme caching with proper invalidation
- 🔄 Asynchronous Operations - Built on asyncio for high-performance concurrent processing
- 🌐 Protocol Support - HTTP, HTTPS, SOCKS4, SOCKS5, CONNECT:80, CONNECT:25
- 🕵️ Anonymity Detection - Transparent, Anonymous, and High anonymity validation
- 📡 50+ Proxy Sources - Automatically discovers and validates from multiple providers
- 🖥️ Proxy Server Mode - Deploy your own rotating proxy server with automatic failover
- 🎯 Smart Filtering - Filter by country, protocol, anonymity level, response time
- 🐍 Modern Python - Full support for Python 3.10, 3.11, 3.12, and 3.13
Docker Hub https://hub.docker.com/r/bluet/proxybroker2
$ docker run --rm bluet/proxybroker2 --help
usage: proxybroker [--max-conn MAX_CONN] [--max-tries MAX_TRIES]
[--timeout SECONDS] [--judge JUDGES] [--provider PROVIDERS]
[--verify-ssl]
[--log [{NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
[--min-queue MINIMUM_PROXIES_IN_QUEUE]
[--version] [--help]
{find,grab,serve,update-geo} ...
Proxy [Finder | Checker | Server]
Commands:
These are common commands used in various situations
{find,grab,serve,update-geo}
find Find and check proxies
grab Find proxies without a check
serve Run a local proxy server
update-geo Download and use a detailed GeoIP database
Options:
--max-conn MAX_CONN The maximum number of concurrent checks of proxies
--max-tries MAX_TRIES
The maximum number of attempts to check a proxy
--timeout SECONDS, -t SECONDS
Timeout of a request in seconds. The default value is
8 seconds
--judge JUDGES Urls of pages that show HTTP headers and IP address
--provider PROVIDERS Urls of pages where to find proxies
--verify-ssl, -ssl Flag indicating whether to check the SSL certificates
--min-queue MINIMUM_PROXIES_IN_QUEUE The minimum number of proxies in the queue for checking connectivity
--log [{NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}]
Logging level
--version, -v Show program's version number and exit
--help, -h Show this help message and exit
Run 'proxybroker <command> --help' for more information on a command.
Suggestions and bug reports are greatly appreciated:
<https://github.com/bluet/proxybroker2/issues>
- Python 3.10-3.13 (latest stable versions supported)
- Core Dependencies (automatically installed):
🎉 ProxyBroker2 v2.0.0b1 is production-ready! Zero critical bugs, 100% test coverage, and full Python 3.10-3.13 support.
⚠️ WARNING: The PyPI packageproxybrokeris outdated (v0.3.2) and no longer maintained. Use ProxyBroker2 from GitHub for the latest production-ready version.
Install the latest stable release:
# Install stable v2.0.0b1 release
$ pip install -U git+https://github.com/bluet/[email protected]
# Or install latest development version
$ pip install -U git+https://github.com/bluet/proxybroker2.git
Why ProxyBroker2 v2.0.0b1?
- 🚀 Production Ready: Zero critical bugs, thoroughly tested
- 🐍 Modern Python: Full Python 3.10-3.13 compatibility
- ⚡ High Performance: Modern async patterns and optimizations
- 🛡️ Stable API: Contract-based testing ensures backward compatibility
- 📚 Great Docs: Comprehensive guides and API documentation
$ docker pull bluet/proxybroker2
Supported Operating System: Windows, Linux, MacOS
On UNIX-like systems (Linux / macOSX / BSD)
Install these tools
- upx
- objdump (this tool is usually in the binutils package)
$ sudo apt install -y upx-ucl binutils # On Ubuntu / Debian
pip install pyinstaller \
&& pip install . \
&& mkdir -p build \
&& cd build \
&& pyinstaller --onefile --name proxybroker --add-data "../proxybroker/data:data" --workpath ./tmp --distpath . --clean ../py2exe_entrypoint.py \
&& rm -rf tmp *.spec
The executable is now in the build directory
After installation, you can immediately start finding proxies:
# Find 5 working HTTP proxies
$ python -m proxybroker find --types HTTP --limit 5
# Find 10 US HTTP proxies
$ python -m proxybroker find --types HTTP --countries US --limit 10
# Run local proxy server on port 8888
$ python -m proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS
Find and show 10 HTTP(S) proxies from United States with the high level of anonymity:
$ python -m proxybroker find --types HTTP HTTPS --lvl High --countries US --strict -l 10
Find and save to a file 10 US proxies (without a check):
$ python -m proxybroker grab --countries US --limit 10 --outfile ./proxies.txt
Run a local proxy server that distributes incoming requests to a pool of found HTTP(S) proxies with the high level of anonymity:
$ python -m proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High --min-queue 5
Run python -m proxybroker --help for more information on the options available.
Run python -m proxybroker <command> --help for more information on a command.
Find and show 10 working HTTP(S) proxies:
import asyncio
from proxybroker import Broker
async def show(proxies):
while True:
proxy = await proxies.get()
if proxy is None:
break
print("Found proxy: %s" % proxy)
async def main():
proxies = asyncio.Queue()
broker = Broker(proxies)
await asyncio.gather(
broker.find(types=["HTTP", "HTTPS"], limit=10),
show(proxies)
)
if __name__ == "__main__":
asyncio.run(main())
ProxyBroker2 implements a comprehensive contract-based testing strategy that ensures reliability while enabling innovation:
- User-visible behavior - "Does proxy finding work?" vs internal algorithms
- API signatures - Method parameters and return types users depend on
- Protocol support - HTTP, HTTPS, SOCKS4/5 compatibility
- Error contracts - Exception types and error handling behavior
- Internal algorithms - Allow optimization without breaking tests
- Exact protocol bytes - Enable protocol improvements and IPv6 support
- Provider specifics - Adapt to website changes without test failures
- Performance metrics - Implementation details that can evolve
This approach protects your code from breaking changes while allowing ProxyBroker2 to continuously improve its internals.
Check X-Proxy-Info header in response.
$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://httpbin.org/get
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET http://httpbin.org/get HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< X-Proxy-Info: 174.138.42.112:8080
< Date: Mon, 04 May 2020 03:39:40 GMT
< Content-Type: application/json
< Content-Length: 304
< Server: gunicorn/19.9.0
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< X-Cache: MISS from ADM-MANAGER
< X-Cache-Lookup: MISS from ADM-MANAGER:880
< Connection: keep-alive
<
{
"args": {},
"headers": {
"Accept": "*/*",
"Cache-Control": "max-age=259200",
"Host": "httpbin.org",
"User-Agent": "curl/7.58.0",
"X-Amzn-Trace-Id": "Root=1-5eaf8e7c-6a1162a1387a1743a49063f4"
},
"origin": "...",
"url": "http://httpbin.org/get"
}
* Connection #0 to host 127.0.0.1 left intact
We are not able to modify HTTPS traffic to inject custom header once they start being encrypted. A X-Proxy-Info will be sent to client after HTTP/1.1 200 Connection established but not sure how clients can read it.
(env) bluet@ocisly:~/workspace/proxybroker2$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v https://httpbin.org/get
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to httpbin.org:443
> CONNECT httpbin.org:443 HTTP/1.1
> Host: httpbin.org:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
< X-Proxy-Info: 207.148.22.139:8080
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
...
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5560b2e93580)
> GET /get HTTP/2
> Host: httpbin.org
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Mon, 04 May 2020 03:39:35 GMT
< content-type: application/json
< content-length: 256
< server: gunicorn/19.9.0
< access-control-allow-origin: *
< access-control-allow-credentials: true
<
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.58.0",
"X-Amzn-Trace-Id": "Root=1-5eaf8e77-efcb353b0983ad6a90f8bdcd"
},
"origin": "...",
"url": "https://httpbin.org/get"
}
* Connection #0 to host 127.0.0.1 left intact
For HTTP, it's easy.
$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/history/url:http://httpbin.org/get
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET http://proxycontrol/api/history/url:http://httpbin.org/get HTTP/1.1
> Host: proxycontrol
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 34
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{"proxy": "..."}
For HTTPS, we're not able to know encrypted payload (request), so only hostname can be used.
$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/history/url:httpbin.org:443
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET http://proxycontrol/api/history/url:httpbin.org:443 HTTP/1.1
> Host: proxycontrol
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 34
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{"proxy": "..."}
* Connection #0 to host 127.0.0.1 left intact
$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/remove/PROXY_IP:PROXY_PORT
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET http://proxycontrol/api/remove/... HTTP/1.1
> Host: proxycontrol
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 204 No Content
<
* Connection #0 to host 127.0.0.1 left intact
If you're upgrading from the original ProxyBroker v0.3.2, here are the key changes:
Python Version
# v0.3.2: Python 3.5.3+ supported
# v2.0.0+: Python 3.10+ required
python --version # Must be 3.10+Installation
# v0.3.2: pip install proxybroker
# v2.0.0+: Install from GitHub (original is abandoned)
pip uninstall proxybroker # Remove old version if installed
pip install git+https://github.com/bluet/proxybroker2.gitCLI Usage
# v0.3.2: proxybroker find --types HTTP --limit 10
# v2.0.0+: python -m proxybroker find --types HTTP --limit 10The Python API remains 100% compatible:
# This code works in both versions
import asyncio
from proxybroker import Broker
async def main():
proxies = asyncio.Queue()
broker = Broker(proxies)
await broker.find(types=['HTTP'], limit=10)
asyncio.run(main()) # Modern async pattern- Zero critical bugs - Fixed deadlocks, memory leaks, and race conditions that existed in v0.3.2
- Active maintenance - Regular updates vs abandoned original project
- Better performance - Modern async patterns and optimizations
- Python 3.10-3.13 support - Latest Python features and compatibility
- Comprehensive testing - Reliable test suite vs limited testing in v0.3.2
- Better documentation - Updated examples and comprehensive guides
📚 Complete Documentation: https://proxybroker2.readthedocs.io/
Our documentation uses a modern approach:
- Auto-generated API reference - Always current with source code
- Hand-written guides - Installation, tutorials, architecture
- Enhanced Sphinx setup - MyST-Parser, auto-linking, cross-references
- Live API docs - Generated from docstrings (19.6% coverage, high quality)
- Multiple formats - HTML, PDF, downloadable archives
- Modern Markdown - Enhanced syntax with MyST-Parser
- Cross-references - Links to Python and aiohttp documentation
- Check the ping, response time and speed of data transfer
- Check site access (Google, Twitter, etc) and even your own custom URL's
- Information about uptime
- Checksum of data returned
- Support for proxy authentication
- Finding outgoing IP for cascading proxy
- The ability to specify the address of the proxy without port (try to connect on defaulted ports)
We welcome contributions! The project has excellent test coverage and development tooling.
- Fork it: https://github.com/bluet/proxybroker2/fork
- Clone and setup:
git clone https://github.com/yourusername/proxybroker2.git cd proxybroker2 poetry install # Install dependencies
- Create your feature branch:
git checkout -b my-new-feature - Make changes and format:
# Auto-format code (required before commit) ruff check . --fix && ruff format . # Run tests to ensure everything works pytest tests/ -v
- Commit with conventional format:
# Use conventional commit format for better automation git commit -m "feat: add SOCKS5 authentication support" git commit -m "fix: resolve memory leak in proxy pool" git commit -m "docs: update installation instructions"
- Push to the branch:
git push origin my-new-feature - Submit a pull request!
- Poetry 2.1.3+: Modern dependency management and virtual environments
- ruff: Ultra-fast linting and formatting (replaces flake8/isort)
- pytest 8.3.5+: Modern testing framework with async support
- pytest-asyncio 0.26.0+: Enhanced async testing capabilities
- pytest-cov 6.1.1+: Comprehensive coverage reporting
- Sphinx 8.0+ + MyST-Parser 4.0+: Modern documentation with auto-generation
- ReadTheDocs: Professional documentation hosting
- Conventional commits: Structured commit format for automation
- Architecture guide: See CLAUDE.md for detailed insights
Licensed under the Apache License, Version 2.0
This product includes GeoLite2 data created by MaxMind, available from http://www.maxmind.com.
Thanks goes to these wonderful people (emoji key):
 a5r0n 💻 |
 C.M. Yang 💻 🤔 👀 |
 Ivan Villareal 💻 |
 Quancore 💻 |
 Felipe 🤔 |
 vincentinttsh 💻 👀 |
 Ziloka 💻 |
 hms5232 💻 |
 Stefan Machmeier 💻 |
 steven 📖 |
This project follows the all-contributors specification. Contributions of any kind welcome!
