​
What is BeautifulSoup?

BeautifulSoup is a Python library that simplifies extracting and organizing data from HTML and XML documents. By integrating Toolip proxies, you can scrape data securely and anonymously, reducing the risk of detection and blocking.

Targeting search engines like Google, Bing, or Yandex requires a specialized proxy to ensure stable access and avoid blocks. Toolip’s Search Engine ISP Proxies are designed specifically for this, providing reliable performance where standard proxies may fail. If your proxy test isn’t working on search engines, switching to Search Engine ISP Proxies can resolve the issue.

​
How to Integrate Toolip with BeautifulSoup

1

Install Required Dependencies

1. Download and install the latest Python version from python.org.
2. Install BeautifulSoup and the requests library by running:

pip install beautifulsoup4 requests
2

Set Up the Proxy

1. Log in to your Toolip account and retrieve your proxy credentials:

  • Host: proxy.toolip.io

  • Port: Use the port number from your dashboard.

  • Username: Your Toolip username (modify it for geo-specific proxies, e.g., your-username-country-US).

  • Password: Your Toolip password.

2. Define your proxy details in your Python script:

proxy = {
  "http": "http://[USERNAME]:[PASSWORD]@[HOST]:[PORT]",
  "https": "http://[USERNAME]:[PASSWORD]@[HOST]:[PORT]"
}
3

Write a Scraper Using BeautifulSoup

Use the script below to integrate Toolip proxies with BeautifulSoup for secure and anonymous web scraping:

import requests
from bs4 import BeautifulSoup

# Toolip Proxy Configuration
proxy = {
    "http": "http://[USERNAME]:[PASSWORD]@[HOST]:[PORT]",
    "https": "http://[USERNAME]:[PASSWORD]@[HOST]:[PORT]"
}

# Target URL to verify the proxy
url = "https://httpbin.org/ip"

try:
    # Send the request using the proxy
    response = requests.get(url, proxies=proxy, timeout=10)
    response.raise_for_status()  # Handle HTTP errors

    # Parse the HTML content
    soup = BeautifulSoup(response.text, "html.parser")

    # Print the formatted page content
    print("Response Content (IP Address):")
    print(soup.prettify())

except requests.exceptions.RequestException as e:
    print("Error occurred while using the proxy:", e)
4

Verify the Output

Run your script, and if everything is set up correctly, you should see the IP address of your Toolip proxy in the output:

{
  "origin": "123.45.67.89"
}

If you see your real IP instead, double-check your username, password, and proxy format.

By integrating Toolip proxies with BeautifulSoup, you can collect web data securely and efficiently while maintaining anonymity. Whether you’re extracting structured data, accessing geo-restricted content, or handling large-scale scraping projects, Toolip ensures reliable and seamless operations. 🚀

ApifyHelium Scraper