有其他提供免费代理的网站在这里说下，我添加到项目里 #71

jhao104 · 2017-09-21T10:14:33Z

现在的代理网站不是很多，这样可用的代理IP就很少。我也尝试过扫描的方法，但是效率比较低

CokkyWoo · 2017-09-28T09:39:21Z

这2个可以看看
http://www.xicidaili.com/nn
http://www.kuaidaili.com/ops/

jhao104 · 2017-10-16T05:29:14Z

@CokkyWoo 这两个网站都是有的

qinyongliang · 2018-05-14T07:35:09Z

http://www.gatherproxy.com

Purek · 2018-08-13T13:55:22Z

http://free-proxy.cz/zh/proxylist/country/US/https/ping/all，您好，能麻烦您添加下这个嘛？

1yzz · 2019-02-24T14:29:47Z

http://proxydb.net/?protocol=https&anonlvl=4

abc1763613206 · 2019-03-25T03:42:52Z

https://ip.ihuan.me

llliuwenjie · 2019-09-16T06:38:28Z

http://www.xiladaili.com/ 西拉代理

dota2heqiuzhi · 2019-11-19T06:25:22Z

@jhao104 你好。这边我看有人提供了几个墙外的代理网址，似乎都不错。可以抽空添加一下吗？（我自己没搞定，有的做了反爬，有的浏览器能打开，但是request连不上····）
谢谢了。
http://free-proxy.cz/zh/proxylist/country/US/https/ping/all
http://www.gatherproxy.com
http://proxydb.net/?protocol=https&anonlvl=4

目前代理墙外的代理网址只有3个，能抓到的太少了

顺便请教一下，为什么li浏览器可以打开，但是requests连不上

jhao104 · 2019-11-19T09:14:39Z

@dota2heqiuzhi #385 (comment)

dota2heqiuzhi · 2019-11-19T09:18:41Z

@jhao104 那你有时间添加这些网址吗？
没空我就自己琢磨了···

jhao104 · 2019-11-19T09:22:27Z

@jhao104 那你有时间添加这些网址吗？
没空我就自己琢磨了···

墙外的你可以自己先搞

dota2heqiuzhi · 2019-11-21T06:53:32Z

@jhao104 那你有时间添加这些网址吗？
没空我就自己琢磨了···

墙外的你可以自己先搞

这两个网址都做了反爬···搞不定。
大佬空了可以搞定一个，我学习学习？
http://free-proxy.cz/zh/proxylist/country/US/https/ping/all
http://proxydb.net/?protocol=https&anonlvl=4

@jhao104

jhao104 · 2019-11-25T04:29:37Z

@jhao104 那你有时间添加这些网址吗？
没空我就自己琢磨了···

墙外的你可以自己先搞

这两个网址都做了反爬···搞不定。
大佬空了可以搞定一个，我学习学习？
http://free-proxy.cz/zh/proxylist/country/US/https/ping/all
http://proxydb.net/?protocol=https&anonlvl=4

@jhao104

就是js动态生成的，你把这段j s扣出来用pyv8或者pyexecjs执行就能拿到了

dota2heqiuzhi · 2019-11-25T04:33:04Z

我主要是不会js，只会一点python，当时用 pyexecjs试了一会没搞出来😂 空了我再试试，谢谢！ J_hao104 <[email protected]> 于2019年11月25日周一下午12:29写道：

…

@jhao104 <https://github.com/jhao104> 那你有时间添加这些网址吗？没空我就自己琢磨了··· 墙外的你可以自己先搞这两个网址都做了反爬···搞不定。大佬空了可以搞定一个，我学习学习？ http://free-proxy.cz/zh/proxylist/country/US/https/ping/all http://proxydb.net/?protocol=https&anonlvl=4 @jhao104 <https://github.com/jhao104> [image: image] <https://user-images.githubusercontent.com/15058920/69512436-f9d4f300-0f7e-11ea-8710-be649443a79c.png> [image: image] <https://user-images.githubusercontent.com/15058920/69512474-24bf4700-0f7f-11ea-8a46-142b7c9197d2.png> 就是js动态生成的，你把这段j s扣出来用pyv8或者pyexecjs执行就能拿到了 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#71?email_source=notifications&email_token=AIUSUJANS6QA4BH7CP5I6NTQVNIDFA5CNFSM4D34MZLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFBCKLQ#issuecomment-557983022>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIUSUJFTYOXHWMW6FBQYA23QVNIDFANCNFSM4D34MZLA> .

1yzz · 2019-12-05T03:17:58Z

http://proxydb.net/?protocol=https&anonlvl=4

    @staticmethod
    def proxyDBNet():
        urls = [
            'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=CN',
            'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=',
            'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=SG',
            'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=US',
            'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=CZ',
            'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=AR',
        ]
        request = WebRequest()

        for url in urls:
            r = request.get(url, timeout=20)
            proxies = re.findall(r'(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}:\d+)', r.text)
            for proxy in proxies:
                yield proxy

dota2heqiuzhi · 2019-12-05T03:20:25Z

这个网站也做了反爬的（js)。你的爬取逻辑应该抓不到数据吧 1yzz <[email protected]> 于2019年12月5日周四上午11:18写道：

…

http://proxydb.net/?protocol=https&anonlvl=4 @staticmethod def proxyDBNet(): urls = [ 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=CN', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=SG', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=US', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=CZ', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=AR', ] request = WebRequest() for url in urls: r = request.get(url, timeout=20) proxies = re.findall(r'(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}:\d+)', r.text) for proxy in proxies: yield proxy — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#71?email_source=notifications&email_token=AIUSUJDU3ONTHN5BM2KNKZLQXBXGRA5CNFSM4D34MZLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF7KUXQ#issuecomment-561949278>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIUSUJHR4KAZ7HOACHQYOS3QXBXGRANCNFSM4D34MZLA> .

1yzz · 2019-12-05T04:06:50Z

这个网站也做了反爬的（js)。你的爬取逻辑应该抓不到数据吧 1yzz [email protected] 于2019年12月5日周四上午11:18写道：
…
http://proxydb.net/?protocol=https&anonlvl=4 @staticmethod def proxyDBNet(): urls = [ 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=CN', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=SG', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=US', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=CZ', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=AR', ] request = WebRequest() for url in urls: r = request.get(url, timeout=20) proxies = re.findall(r'(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}:\d+)', r.text) for proxy in proxies: yield proxy — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#71?email_source=notifications&email_token=AIUSUJDU3ONTHN5BM2KNKZLQXBXGRA5CNFSM4D34MZLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF7KUXQ#issuecomment-561949278>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUSUJHR4KAZ7HOACHQYOS3QXBXGRANCNFSM4D34MZLA .

 <td>
                        <script>
                            var  q =
                             '32.5.301'.split('').reverse().join('');
                            var yxy = /* */ atob('\x4d\x69\x34\x78\x4e\x44\x59\x3d'.replace(/\\x([0-9A-Fa-f]{2})/g,function(){return String.fromCharCode(parseInt(arguments[1], 16))}));
                            var  pp =  (8080 - ([]+[]))/**//**/ +  (+document.querySelector('[data-rnnumg]').getAttribute('data-rnnumg'))-[]+[];
                            document.write('<a href="/' + q + yxy + '/' + pp + '#http">' + q + yxy + String.fromCharCode(58) + pp + '</a>');
                        </script>
                    </td>

找到这个元素，script里面的内容定义一个函数，pyv8执行一下。document.querySelector('[data-rnnumg]').getAttribute('data-rnnumg') 这个值也能在DOM里面找到，可以解析DOM树，替换内容。

1yzz · 2019-12-05T04:08:43Z

这个网站也做了反爬的（js)。你的爬取逻辑应该抓不到数据吧 1yzz [email protected] 于2019年12月5日周四上午11:18写道：
…
http://proxydb.net/?protocol=https&anonlvl=4 @staticmethod def proxyDBNet(): urls = [ 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=CN', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=SG', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=US', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=CZ', 'http://proxydb.net/?protocol=https&anonlvl=4&min_uptime=75&max_response_time=5&country=AR', ] request = WebRequest() for url in urls: r = request.get(url, timeout=20) proxies = re.findall(r'(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}:\d+)', r.text) for proxy in proxies: yield proxy — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#71?email_source=notifications&email_token=AIUSUJDU3ONTHN5BM2KNKZLQXBXGRA5CNFSM4D34MZLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF7KUXQ#issuecomment-561949278>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUSUJHR4KAZ7HOACHQYOS3QXBXGRANCNFSM4D34MZLA .

https://github.com/scrapinghub/splash 有个这个东西，html丢过去就完事了。但是不确定会不会影响爬虫效率/。

hanjackcyw · 2020-01-16T09:33:49Z

我看好多人要这个网站的代理，我刚好才爬过，贴一下代码如下，需要安装scrapy包, 主要是我用scrapy用习惯了，当然用其它各种包做xpath解析也行。

    @staticmethod
    def freeProxy21():
        url = 'http://free-proxy.cz/en/proxylist'

        request = WebRequest()
        r = request.get(url, timeout=10)

        sel = scrapy.Selector(text=r.text)

        max_page = max([int(v) for v in sel.xpath('//div[@class="paginator"]/a/text()').extract() if v.isdigit()])
        print(max_page)

        for page in range(1, max_page + 1):
            r = request.get(url+'/main/{}'.format(page), timeout=10)

            sel = scrapy.Selector(text=r.text)

            proxies = sel.xpath('//table[@id="proxy_list"]/tbody/tr/td/script[contains(text(),"decode")]/text()').extract()
            ports = sel.xpath('//table[@id="proxy_list"]/tbody/tr/td/span/text()').extract()

            for index, value in enumerate(proxies):
                try:
                    proxy_ip = re.search('.*decode\(\"(.*)\"\)', value).group(1)
                    if proxy_ip:
                        proxy = '{}:{}'.format(base64.b64decode(proxy_ip).decode('utf-8'), ports[index])
                        yield proxy
                except Exception as e:
                    pass

hailiang-wang · 2020-03-07T14:35:51Z

@hanjackcyw scrapy会带来很大体积，如果只是为了使用 Selector可以用Scrapy底层的库。

https://parsel.readthedocs.io/en/latest/

dpawsbear · 2020-09-11T08:57:37Z

好像这个代理也不错：https://proxy.mimvp.com/freeopen

lyonLeeLPL · 2020-12-21T04:27:07Z

https://www.feizhuip.com/News-getInfo-id-1307.html 这个也许不错

TophTab · 2020-12-23T01:31:20Z

可以看看这个，蜻蜓的免费
https://proxy.horocn.com/free-china-proxy/all.html?page=lr&max_id=2N

另外大佬，用docker搭在云服务器上，命令里的redis是改成自己的吗？

jwdeaa · 2021-02-16T16:32:51Z

A new proxy list: http://pzzqz.com/

jhao104 · 2021-04-02T06:44:01Z

A new proxy list: http://pzzqz.com/

已添加

jingshaoqi · 2021-08-10T14:00:51Z

https://zhimahttp.com/?utm-source=bdtg&utm-keyword=?400359
芝麻免费代理

jhao104 · 2021-12-27T01:53:41Z

https://zhimahttp.com/?utm-source=bdtg&utm-keyword=?400359 芝麻免费代理

他这个免费的代码很挫，更新时间都很久了

xswwxx · 2022-12-31T08:25:51Z

https://openproxylist.xyz/http.txt 这种的添加模式要怎么弄

CaoYunzhou · 2023-05-30T03:07:14Z

https://openproxylist.xyz/http.txt 这种的添加模式要怎么弄

@staticmethod
def freeProxy17():
    urls = [
        'https://openproxylist.xyz/http.txt',
        'http://pubproxy.com/api/proxy?limit=3&format=txt&http=true&type=https',
        'https://www.proxy-list.download/api/v1/get?type=https',
        'https://raw.githubusercontent.com/shiftytr/proxy-list/master/proxy.txt'
    ]
    request = WebRequest()
    for url in urls:
        r = request.get(url, timeout=20)
        for proxy in r.text.split('\n'):
            if proxy:
                yield proxy

djme0 · 2023-06-22T11:33:15Z

https://uu-proxy.com/
https://www.proxyscan.io/
http://www.kxdaili.com/dailiip.html
https://www.xsdaili.cn/

xiumao-cat · 2023-12-04T04:11:42Z

https://ip.uqidata.com/free/index.html
https://www.69ip.cn/?page=3
https://proxy.ip3366.net/free/
https://www.binglx.cn

jhao104 added the help wanted label Sep 21, 2017

jhao104 mentioned this issue Nov 13, 2017

请问爬虫过程中一个代理失效后是自动启动下一个代理ip吗？ #84

Closed

lsq mentioned this issue Jan 22, 2020

ip proxy lsq/officetools#1

Open

duyuefeng0708 mentioned this issue May 31, 2021

几个免费代理 #557

Open

This comment was marked as spam.

Sign in to view

jhao104 closed this as completed Aug 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

有其他提供免费代理的网站在这里说下，我添加到项目里 #71

有其他提供免费代理的网站在这里说下，我添加到项目里 #71

jhao104 commented Sep 21, 2017

CokkyWoo commented Sep 28, 2017

jhao104 commented Oct 16, 2017

qinyongliang commented May 14, 2018

Purek commented Aug 13, 2018

1yzz commented Feb 24, 2019

abc1763613206 commented Mar 25, 2019

llliuwenjie commented Sep 16, 2019

dota2heqiuzhi commented Nov 19, 2019 •

edited

Loading

jhao104 commented Nov 19, 2019 •

edited

Loading

dota2heqiuzhi commented Nov 19, 2019

jhao104 commented Nov 19, 2019

dota2heqiuzhi commented Nov 21, 2019 •

edited

Loading

jhao104 commented Nov 25, 2019

dota2heqiuzhi commented Nov 25, 2019 via email

1yzz commented Dec 5, 2019

dota2heqiuzhi commented Dec 5, 2019 via email

1yzz commented Dec 5, 2019

1yzz commented Dec 5, 2019

hanjackcyw commented Jan 16, 2020 •

edited

Loading

hailiang-wang commented Mar 7, 2020

dpawsbear commented Sep 11, 2020

lyonLeeLPL commented Dec 21, 2020

TophTab commented Dec 23, 2020

jwdeaa commented Feb 16, 2021

jhao104 commented Apr 2, 2021

jingshaoqi commented Aug 10, 2021

jhao104 commented Dec 27, 2021

This comment was marked as spam.

xswwxx commented Dec 31, 2022

CaoYunzhou commented May 30, 2023

djme0 commented Jun 22, 2023

xiumao-cat commented Dec 4, 2023

有其他提供免费代理的网站在这里说下，我添加到项目里 #71

有其他提供免费代理的网站在这里说下，我添加到项目里 #71

Comments

jhao104 commented Sep 21, 2017

CokkyWoo commented Sep 28, 2017

jhao104 commented Oct 16, 2017

qinyongliang commented May 14, 2018

Purek commented Aug 13, 2018

1yzz commented Feb 24, 2019

abc1763613206 commented Mar 25, 2019

llliuwenjie commented Sep 16, 2019

dota2heqiuzhi commented Nov 19, 2019 • edited Loading

jhao104 commented Nov 19, 2019 • edited Loading

dota2heqiuzhi commented Nov 19, 2019

jhao104 commented Nov 19, 2019

dota2heqiuzhi commented Nov 21, 2019 • edited Loading

jhao104 commented Nov 25, 2019

dota2heqiuzhi commented Nov 25, 2019 via email

1yzz commented Dec 5, 2019

dota2heqiuzhi commented Dec 5, 2019 via email

1yzz commented Dec 5, 2019

1yzz commented Dec 5, 2019

hanjackcyw commented Jan 16, 2020 • edited Loading

hailiang-wang commented Mar 7, 2020

dpawsbear commented Sep 11, 2020

lyonLeeLPL commented Dec 21, 2020

TophTab commented Dec 23, 2020

jwdeaa commented Feb 16, 2021

jhao104 commented Apr 2, 2021

jingshaoqi commented Aug 10, 2021

jhao104 commented Dec 27, 2021

This comment was marked as spam.

xswwxx commented Dec 31, 2022

CaoYunzhou commented May 30, 2023

djme0 commented Jun 22, 2023

xiumao-cat commented Dec 4, 2023

dota2heqiuzhi commented Nov 19, 2019 •

edited

Loading

jhao104 commented Nov 19, 2019 •

edited

Loading

dota2heqiuzhi commented Nov 21, 2019 •

edited

Loading

hanjackcyw commented Jan 16, 2020 •

edited

Loading