Refactor Python proxy examples to use libraries directly#21
Merged
Conversation
added 10 commits
April 5, 2026 14:42
Remove python-proxy-headers and scrapy-proxy-headers usage. Add per-library *-proxy.py scripts matching the Ruby/JS/PHP style, plus python/requirements.txt. Update run_tests.py to match (including Scrapy via runspider and optional skips when pycurl or scrapy imports fail). Refresh README Python section. Made-with: Cursor
…headers Made-with: Cursor
…torms Made-with: Cursor
Strip userinfo from proxy URL and send Proxy-Authorization via proxy_headers so HTTPS CONNECT matches authenticated requests behavior. Pin integration workflow Python to 3.12 for stable wheels. Made-with: Cursor
Pins actions/checkout v6.0.2, setup-python v6.2.0, setup-node v6.3.0 to clear GitHub Actions Node 20 deprecation warnings on integration workflows. Made-with: Cursor
Made-with: Cursor
Default to example.com HTML with WANTED_TEXT; document JSON is a poor fit. Made-with: Cursor
Made-with: Cursor
…_args Made-with: Cursor
proxymesh
approved these changes
Apr 6, 2026
added 5 commits
April 6, 2026 14:06
Made-with: Cursor
…tch) Made-with: Cursor
…ilar Derive wanted text from normalize(unescape(html)) like _get_soup. Pass html= to get_result_similar so the second HTTP response cannot differ from the first. Made-with: Cursor
AutoScraper's own tests use build(html=...) with no network; document README proxy pattern for live URLs. Removes flaky httpbin/proxy HTML drift. Made-with: Cursor
beautifulsoup4 4.13+ breaks autoscraper build() (empty matches); CI used 4.14.x. Made-with: Cursor
proxymesh
approved these changes
Apr 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace python-proxy-headers and scrapy-proxy-headers-based scripts with plain per-library examples (requests, urllib3, aiohttp, httpx, pycurl, cloudscraper, autoscraper, Scrapy), aligned with the Ruby, JavaScript, and PHP examples. Add python/requirements.txt, update run_tests.py (Scrapy via runspider; skip pycurl or scrapy when imports fail), and refresh the README Python section.