Proxies ip changes and anonymity
Die Verbindung zum Internet wird über diverse Kanäle aufgebaut und hilft somit die ausgehenden Verbindungen zu anonymisieren.
VPN[Bearbeiten]
über unseren jetzigen VPN-Anbieter stehen uns über 5'000 IP-Adresse zur Verfügung. Diese sind aber leider manchmal bereits bekannt und somit nur für einfache Spiderings nutzbar.
Es exisiteren jedoch auch VPN dienste, welche dies anbieten
- https://www.starvpn.com/ (45$/Month am 20.08.2022)
- https://www.tuxlervpn.com/ (8$/Month am 20.08.2022, jedoch sehr komisch. Es gibt kostenlosen Dienst, welcher zur Verfügung steht)
LTE[Bearbeiten]
Restart der Router[Bearbeiten]
Die Router werden für die netgears mit einem script namens ipchange.js neu gestartet...
für die teltonika router wird ein cronjob erstellt mit folgendem inhalt
- /8 * * * * uqmi -d /dev/cdc-wdm0 --uim-power-off --uim-slot 1
VDSL[Bearbeiten]
VDSL mit erneuern von ips ist leider nicht mehr möglich in der schweiz oder es ist kein anbieter bekannt der dies unterstüzt. die leasetime ist sehr hoch.
lease or buy ips[Bearbeiten]
- https://www.ipxo.com (An IP address marketplace that helps lease and monetize unused IP resources and alleviates the IPv4 shortage problem. We provide our clients with a full automation stack that ensures accessibility and innovative solutions for all.)
Proxies[Bearbeiten]
PAYED PROXIES[Bearbeiten]
- https://dataimpulse.com/
- https://dashboard.statproxies.com/?refferal=scrapegraph#residential
- https://smartproxy.com/
- https://razorproxy.com/
- https://proxyscrape.com/residential-proxies?ref=head#pricing
- https://brightdata.com/ – great success rate (>99% to infra), very fast (0.7 s response time with UK proxies, 1.3 s with US), large IP pool, very expensive (esp. advanced features like unlimited traffic for datacenter or city targeting for resi). Recently reduced PAYG prices, increased more expensive plan rates, simplified KYC.
- https://www.ninjasproxy.com/
- https://stormproxies.com/ 04.05.2022: TOO SLOW (less than 0.5 MBit/s Down and Up), DETECTED BY GOOGLE, CLOUDFLARE, DATADOME – pretty bad quality and speed, mixes residential and DC proxies with few targeting options, a lot of Asian IPs there (Korean, Chinese, etc).
- https://intoli.com/
- https://zenscrape.com/
- https://microleaves.com/now called Shifter, had a pool based on malware (google: Microleaves malware), okay performance (~95% success rate, 1.37 s US response time), average pool. Expensive but offers unlimited traffic, limits websites you can target, plans that support country filtering cost double.
- https://www.scrapinghub.com/crawlera
- https://blazingseollc.com/
- http://dslroot.com/
- https://fusionproxy.com/
- https://www.ipproxy.info/
- https://instantproxies.com/
- https://www.proxy-n-vpn.com/ they're dc proxies are good. you get what you pay for.
- https://anyip.io/ discountcode: JULY15
Speed of Proxies[Bearbeiten]
https://www.bestproxyreviews.com/fast-proxy/
Shifter[Bearbeiten]
IP Pool Size: Over 31 million Locations: 130 countries Concurrency Allowed: Unlimited Cost: Starts at $125 monthly for 10 ports Shifter is the new and improved Microleaves (yes, Microleaves rebranded to Shifter). In terms of IP rotation, Shifter does not support IP rotating for every request. However, they have support for time-based IP rotation, and the IP Assigned to you is changed after every 5 minutes.
Unlike Luminati, Shifter is priced based on a number of ports, and you are allowed to consume unlimited bandwidth. Their proxies are residential – with about 31 million IPs in their pool. They sell high quality dedicated proxies too.
Shifter Backconnect Residential Proxies for Scraping
Soax[Bearbeiten]
https://soax.com/residential-proxiesgreat
success rate (>99%), average speed (1.68 s UK, 1.92 s US), moderate-sized pool skewed toward Eastern Europe (Russia, Ukraine, etc). Similar price to Smartproxy, offers premium features out of the box (city, ASN targeting), limited concurrent connections via ports.
IP Pool Size: Over 8 million Locations: worldwide Concurrency Allowed: 300 threads Bandwidth Allowed: Unlimited Cost: Starts at $20 daily for 300 ports Soax is a proxy provider that provide a mix of residential proxies and mobile proxies. It proxies are some of the clean and finest proxies. when using their proxies, the risk of getting detectable and blocked is extremely low.
Currently, they have over 8 million IPs in their proxy pool. They have proxies in many countries in the world. You can go as far as going for city-targeted proxies with Soax proxies. their proxies are quite affordable too, and they are rotating proxies.
Smartproxy[Bearbeiten]
https://smartproxy.com/comparable
success rate and performance, fewer advanced features, large IP pool, cheaper than both until terabytes of data. No mobile or static residential proxies. Great value.
IP Pool Size: Over 10 million Locations: 195 locations across the globe Concurrency Allowed: Unlimited Bandwidth Allowed: Starts at 5GB Cost: Starts at $75 monthly for 5GB Smartproxy is one of the premium residential proxies that have cheap plans targeted at small Internet marketers. Smartproxy residential proxies are rotating proxies. you can choose between their high-rotating proxies that changes IP after every request or their time-based proxies that change IP Address after every 10 minutes. Smartproxy has a large proxy pool with over a million residential proxy. It has proxies in about 195 countries and in 6 major cities around the world.
Smartproxy for Scraping
Proxy-cheap[Bearbeiten]
IP Pool Size: over 6 million Locations: Worldwide location coverage Cost: $5 per GB monthly Proxy-cheap is the proxy of choice for those looking for residential proxies to use but are on a low budget. Their mobile proxies and IPv6 datacenter proxies are also cheap.
Proxy-cheap has about 6 million residential IPs distributed across over 100 countries across different continents. Their mobile proxies are located in just the United States and France. Their proxies are not static proxies. their proxies are rotating and priced based on bandwidth.
Proxyrack[Bearbeiten]
https://www.proxyrack.com/pricing/#residential-proxies
IP Pool Size: over 2 million Locations: 140 countries Concurrency Allowed: unlimited Cost: $120 for 250 proxies for a month Proxyrack is one of the cheap residential proxy service providers in the market, with an average of half a million proxies always online in a day. Their residential proxy strength is put at over 2 million; looking at this; you can tell they still have a lot you can use.
Their proxies are priced based on ports. When it comes to IP rotation, their proxies change after a randomly determined period of time between 3 minutes to 30 minutes. They also have entry points that are high-rotating that changes IP per request.
Blazing Proxies / rayobyte[Bearbeiten]
Locations: 9 countries Concurrency Allowed: Unlimited Bandwidth Allowed: Unlimited Cost: Starts at $11 monthly Blazing Proxies is developed by Blazingseo LLC, a company with interest in web services ranging from web servers, windows VPS, and proxies. their proxies are datacenter proxies, but they are rotating. Blazing Proxies comes with unlimited bandwidth and allows you to create the number of threads you want to create.
They do not support the changing of IP after every request. However, you can go for their time-based IP rotation, which changes IP after a randomly determined period of time, which could range from 10 minutes to 120 minutes.
[Bearbeiten]
https://www.webshare.io/residential-isp-proxy
03.03.2022: ONLY STATIC IPS
IP Pool Size: Not disclosed Locations: worldwide Concurrency Allowed: 500 threads Bandwidth Allowed: Unlimited Cost: Starts at $5.44 for 5 ports for a month Webshare is different from the ones discussed above in the sense that it has free proxies you can use. However, we do not advise our users to go for free proxies. going for their paid plans is the best.
They do not have proxies that will change after every request – their proxies change per defined period of time, and this is determined by the plan you go for. You can either go for proxies that change after every 5 minutes or the ones that will maintain a session for as long as one hour.
RotatingProxies[Bearbeiten]
Locations: US only. Europe locations available on request only Concurrency Allowed: Unlimited Bandwidth Allowed: Unlimited Cost: Starts at $39 monthly for 10 ports RotatingProxies proxies are USA dedicated proxies – you can request for proxies in some European locations. From the name, you can tell that they are rotating proxies. what is interesting is that they change the whole pool for you every 5 minutes.
Let say you got 500 proxies from them; after every 5 minutes, you get the whole 500 assigned to you changed. Very important is the fact that their proxies come with unmetered bandwidth. Their proxies are undetectable and evergreen.
LocalProxies[Bearbeiten]
Locations: the US and EU region only Concurrency Allowed: only one device per port Cost: Starts at $50 monthly for one port LocalProxies, for reasons known by them, did not disclose the number of proxies they have in their pool. However, we know that their proxies are residential proxies,and they are rotating proxies. you can decide to go for their time-based rotating proxies or session-based rotating proxies. for their time-based rotating proxies, they change after every 10 minutes.
Netnut[Bearbeiten]
~93% success rate, fast proxies (tested a mix of residential and ISP IPs: 1.05 s UK, 0.86s US), many proxies in the US, though some are identified as datacenter. Starts off expensive, scales well. Terrible at communication (they have 1 GB plans only found in the dashboard), pretty bad UX.
IP Pool Size: Over 500,000 Locations: worldwide Concurrency Allowed: Unlimited Bandwidth Allowed: Starts at 20GB Cost: Starts at $300 monthly for 20GB Netnut is a residential proxy provider that calls itself, ‘the Finest Residential IP Proxy Network.’ If you are looking for a proxy provider that you can use their proxies for scraping and crawling from complex websites without being detected, then Netnut is one of the options available to you. Their proxies support IP rotation by time and session. For their time-based IP rotation, IP is changed after every 30 minutes. Their proxies are priced based on bandwidth.
GeoSurf[Bearbeiten]
okay success rate (~94%) and speed (2.35 s UK, 2.25s US), moderate and well balanced proxy pool. No self-service, haven't updated their website for ages, so some products aren't well represented there.
IP Pool Size: Over 2.5 million Locations: 130 countries Concurrency Allowed: Unlimited Bandwidth Allowed: Starts at 38GB Cost: Starts at $450 monthly for 38GB GeoSurf is another premium residential provider in the class of the likes of Luminati. Just like Luminati, GeoSurf is an extensive option and not pocket-friendly, especially for small marketers. However, it is one of the best in the market; It is secure, reliable, and fast. It currently has a little over 2 million residential proxies in its pool. In terms of IP rotation, it has support for both rotation by time and for every request. Based on timing, you can choose between 1 minute, 10 minutes, or 30 minutes.
ProxyMesh[Bearbeiten]
Locations: 7 countries Concurrency Allowed: Unlimited Bandwidth Allowed: unlimited Cost: Starts at $10 monthly for 10 ports ProxyMesh is a rotating proxy service you can use to avoid IP bans and exceed limits. They are undetectable, and as such, it is easier for them to evade blocks. Integrating ProxyMesh proxies is very easy, thanks to the simple configuration it requires. Their proxies are located in multiple locations in Europe, North America, and Asia. Their proxies are secure and fast enough for web scraping.
Need really the fastest proxies? Visit Here.
PacketStream[Bearbeiten]
https://packetstream.io/ good success rate (over 97%), average speed (1.59 s UK, 3.08 s US), small and likely abused IP pool (<13k US IPs from 300k requests over 14 days, ~5k French and German IPs). Very cheap but you have to top up at least $50, resold by many.
IP Pool Size: Over 500,000
Locations: worldwide
Concurrency Allowed: Unlimited
Bandwidth Allowed: Starts at 1GB
Cost: Starts at $1 monthly for 1GB
PacketStream has a uniqueness that differentials it from the other providers on the list. Aside from selling proxies to their clients, they also provide opportunities for Internet users to share their Internet connection and bandwidth in exchange for some cash.
Their proxy network is built on a pair to pair network model. One thing you will come to love about PacketStream is that their proxies are cheap. Interestingly, their proxies are rotating proxies, and you can use them for web scraping without experiencing any form of blocks aside from occasional Captchas.
Oxylabs[Bearbeiten]
success rate, as fast or faster (0.5 s UK, 1.09 s US), Expensive, no pay as you go, but cheaper than Bright Data. They mix DC proxies into their residential proxies, so make sure you check when you get a new ip.
IP Pool Size: Over 30 million
Locations: worldwide
Concurrency Allowed: Unlimited
Bandwidth Allowed: Starts at 50GB
Cost: Starts at $600 monthly for 50GB
Oxylabs is targeted strictly at medium to large scale marketers – even more, expensive than Luminati and GeoSurf. The team behind Oxylabs is an experienced one. While this was developed for web scraping and crawling, it has since found application in most other areas proxies are useful.
They currently have a residential pool of over 30 million IPs. This makes them one of the largest proxy network. Their proxies support IP rotation based on time and per after every request. When it comes time-based rotation, the IP assigned to you changes every 5 minutes.
HomeIP[Bearbeiten]
IP Pool Size: Over 13 million
Locations: 157 countries
Concurrency Allowed: Unlimited
Bandwidth Allowed: Starts at 10GB
Cost: Starts at $200 monthly for 10GB
HomeIP provides you access to over 13 million residential IPs in 157 countries around the world. One thing with HomeIP is that you can decide to go for their proxies that maintain a persistent session or go for their rotating proxies.
Their proxies are built by developers for developers, and as such, you do not have anything to worry about as a developer. Their proxies are reliable and dependable – you should expect nothing less than a 99.9 percent uptime.
Russian services[Bearbeiten]
- https://nosok.org/en
- https://proxywhite.com/
- https://youproxy.ru/
- https://proxy6.net/en
- https://proxyline.net/en
- https://buy.fineproxy.org/
- https://911.re/ 31.07.202: DONT USE THIS. See Printscreen 04.05.2022:ONLY PAYABLE BY ASIAN-MONEY-TRANSFERS OR COINS
- https://www.proxyrack.com 10.06.2022: 10 - 20MBit/s DOWN AND UP SPEED ON PRIVATE-RESIDENTIAL PROXIES. PRIVATE-RESIDENTIAL PROXIES ARE STILL DETECTED BY Cloudeflare and Google
- https://rsocks.net/ – DONT USE THIS!!!!! THERE ARE HACKED DEVICED THEY USE TO SELL IP's https://www.justice.gov/usao-sdca/pr/russian-botnet-disrupted-international-cyber-operation. reseller. Their exclusive proxies perform well (similar to PacketStream), pool dominated by Russian IPs, cheap. Other residential packages are pretty low quality. Questionable business practices (giving away trial for positive TrustPilot reviews).
- https://5socks.net/ - ACHTUNG konnte zu rsocks.net gehören.
DIY / Roll your own[Bearbeiten]
FREE PROXIES[Bearbeiten]
- https://github.com/chill117/proxy-lists (Attention: only use with fraud-check)
PROXYNPM[Bearbeiten]
const useProxy = require('puppeteer-page-proxy');
PROXY SETTINGS CHROMIUM[Bearbeiten]
https://chromium.googlesource.com/chromium/src/+/HEAD/net/docs/proxy.md#Manual-proxy-settings
PROXY NPM[Bearbeiten]
https://github.com/3proxy/3proxy