How to Scrape Bing Search Results With Python (2026)

I scraped Bing with Python three times in July 2026 before I trusted a single row of the output. The first pass, one requests.get against bing.com/search from my laptop, returned clean HTML with ten organic results. The second, the same code from a cloud box, returned HTTP 200 and zero results behind a CAPTCHA. This guide is the version that survives that gap.

I show the full requests-plus-BeautifulSoup parse, the pagination parameter that trips people up, the redirect links you have to decode, and the managed route I switch to once the blocks start. Bing still holds roughly 10% of global desktop search per StatCounter, so its ranking data is worth pulling alongside Google. Every code block below is something I ran.

What do you need before you start?

Before you scrape Bing with Python you need three things: Python 3.9 or newer, the requests library to fetch the page, and beautifulsoup4 to parse it. Bing’s core search results are server-rendered into the initial HTML, so you do not need Selenium or a headless browser for the organic block. That keeps the whole scraper to a handful of lines.

Install the two libraries and you are ready:

pip install requests beautifulsoup4

The fourth thing you need is a clean IP, and it matters more than the code. The exact same script returns data from a residential connection and an empty CAPTCHA page from a flagged datacenter range, so where the request originates decides whether any of the parsing below runs at all.

How do you scrape Bing search results with Python?

You scrape Bing search results with Python by sending a GET request to bing.com/search and parsing the organic results out of the returned HTML with BeautifulSoup. The results are not behind an API or a JavaScript render for a logged-out visitor, so a single request gives you the page, and the parsing is a matter of knowing which classes Bing uses. Here is the process in three steps.

Step 1: Send a request to bing.com/search

The request is one requests.get with your query in the q parameter. This returns the SERP HTML with a 200 status when it works:

import requests

r = requests.get("https://www.bing.com/search?q=web+scraping", timeout=25)
print(r.status_code)          # 200
print("b_algo" in r.text)     # True from a clean IP, False behind a challenge

One counterintuitive detail from my testing: a plain request with no User-Agent slipped through, while adding a realistic Chrome User-Agent from the same IP triggered Bing’s CAPTCHA and returned zero results. The status code stayed 200 in both cases, so you cannot trust it alone. Check that b_algo is actually present in the body before you parse.

Step 2: Parse the organic results

Bing wraps each organic result in an li with the class b_algo, so you select that list and pull the fields from inside each item. The title and link live in an h2 > a, and the snippet sits in .b_caption p:

import requests
from bs4 import BeautifulSoup

r = requests.get("https://www.bing.com/search?q=web+scraping", timeout=25)
soup = BeautifulSoup(r.text, "html.parser")

for pos, li in enumerate(soup.select("li.b_algo"), start=1):
    h2 = li.find("h2")
    link = h2.find("a") if h2 else None
    if not link:
        continue
    title = h2.get_text(strip=True)
    href = link["href"]
    caption = li.select_one(".b_caption p")
    snippet = caption.get_text(strip=True) if caption else ""
    print(pos, title)
    print(href)      # a bing.com/ck/a redirect, not the real URL yet
    print(snippet)

The enumerate gives you a position index, which is the field a rank tracker actually cares about. Titles and snippets parse cleanly with get_text(strip=True). The href is the part that is not what it looks like, which is Step 3.

Step 3: Decode the Bing redirect links

The href you pulled is a Bing click-tracking redirect, not the destination, so you have to resolve it before the link is usable. Every organic URL comes back shaped like https://www.bing.com/ck/a?!&&p=...&u=a1aHR0cHM..., with the real target base64url-encoded inside the u parameter. The reliable way to recover it is to follow the redirect and read the final URL:

import requests

# href came from the parse loop above
real_url = requests.get(href, allow_redirects=True, timeout=25).url
print(real_url)   # the actual destination

That works, but it costs one extra request per result, which multiplies your request count and hands Bing more chances to challenge you. The faster route is to decode the u parameter directly (strip its two-character prefix, then base64url-decode), though the marker Bing uses has changed before, so I treat the follow-the-redirect method as the stable one and the decode as an optimization to test against live output.

How do you scrape more than one page of Bing results?

You scrape more than one page of Bing results with the first query parameter, which is an offset, not a page number. Page one omits first entirely, page two uses first=11, page three uses first=21, and it climbs in steps of 10 because Bing returns roughly ten organic results per page. Getting this wrong is the most common reason a Bing scraper silently repeats page one.

Here is a loop that walks the first three pages and counts results on each:

import requests
from bs4 import BeautifulSoup

def bing_page(query, page):
    params = {"q": query}
    if page > 1:
        params["first"] = 1 + (page - 1) * 10   # page 2 -> 11, page 3 -> 21
    r = requests.get("https://www.bing.com/search", params=params, timeout=25)
    soup = BeautifulSoup(r.text, "html.parser")
    return soup.select("li.b_algo")

for page in range(1, 4):
    results = bing_page("web scraping", page)
    print(f"page {page}: {len(results)} results")

Add a real delay between pages, a few seconds at least, because a tight loop from one IP is the fastest way to earn the challenge page that the next section is about.

Why does Bing block your Python scraper?

Bing blocks your Python scraper on IP reputation and the full request fingerprint, and it does it without an honest error code. When a request looks automated, Bing returns HTTP 200 with a page that quietly swaps the results for a challenge/verify CAPTCHA, so a status-code check passes while li.b_algo comes back empty. There is no 403, no 429, no retry-after to catch.

Datacenter IP ranges are the first signal Bing flags, which is why the same script behaves differently from your laptop and a cloud server. Bing’s robots.txt also disallows /search for every user-agent, so you are working against the site’s stated crawl policy from the first request. A second, quieter problem is localisation: Bing geolocates the SERP by exit IP, so a proxied query without a pinned market returns the wrong country’s ranking before blocking even enters the picture. I break the three-way block test down with measured numbers in how to scrape Bing without getting blocked.

How do you tell if your Bing scrape was blocked?

You tell if your Bing scrape was blocked by checking the parsed result count and the body for the challenge markers, never the status code on its own. Because Bing serves the CAPTCHA inside an HTTP 200, an r.status_code == 200 check passes straight through a block and quietly fills your dataset with empty pages. The honest signal is an empty b_algo list plus the presence of challenge/verify in the HTML:

from bs4 import BeautifulSoup

def looks_blocked(resp):
    if "challenge/verify" in resp.text:
        return True
    soup = BeautifulSoup(resp.text, "html.parser")
    return len(soup.select("li.b_algo")) == 0

Treat a block as a retry, not a failure. When looks_blocked returns true, route the same query through a fresh IP rather than discarding it, and back off before the next attempt so you do not deepen the flag on that address. Building that retry-and-rotate loop yourself is exactly the maintenance work that pushes most teams toward a managed route once the query list grows.

Is there still an official Bing Search API?

No, there is no dependable official Bing Search API in 2026. Microsoft retired the standalone Bing Search and Bing Custom Search APIs on August 11, 2025, decommissioning existing instances and closing new signups, per the Microsoft Lifecycle announcement. The endpoints that developers leaned on for a decade, including the free F1 tier, stopped serving after that date.

Microsoft’s recommended replacement is Grounding with Bing Search inside Azure AI Foundry, and it solves a different problem. It is priced at $14 per 1,000 transactions on the Grounding with Bing pricing page, and it feeds live web data into an LLM answer rather than handing you ten ranked organic results as JSON. For structured SERP data the practical route is scraping the public page, which puts you back on the blocking and parsing work above.

How do you scrape Bing at scale without managing proxies?

You scrape Bing at scale without managing proxies by sending your query to a scraper API that returns parsed JSON, with the proxy rotation, the CAPTCHA handling, and the geolocation done server side. You send one request and get position-indexed organic results back, with no challenge page to detect and no ck/a redirect to unwrap. That removes every failure mode from the raw route in one call.

I use ChocoData for this. The call is a single GET with your query and an API key:

curl "https://chocodata.com/api/v1/bing/search?q=web+scraping&api_key=$CHOCO_API_KEY"

The same request from Python, which is the version I run in pipelines:

import os, requests

resp = requests.get(
    "https://chocodata.com/api/v1/bing/search",
    params={"q": "web scraping", "api_key": os.environ["CHOCO_API_KEY"]},
    timeout=30,
)
data = resp.json()
for item in data["organic_results"]:
    print(item["position"], item["title"])
    print(item["url"])      # real destination URL, ck/a redirect already resolved

The url field is the resolved destination, so the redirect-following step disappears, and the market is pinned so the ranking is reproducible run to run. In my testing this returned clean organic JSON at about a 2.6-second median including the anti-bot work, on a free tier of 1,000 requests, then roughly $0.60 per 1,000 on the paid plan and $0.90 pay-as-you-go, billed only on successful calls. For a one-off pull of a few hundred queries the raw Python route is fine from a clean IP. For continuous collection the managed call is cheaper once you price in your own time, and I compare the managed options in best Bing scrapers and APIs in 2026.

Is scraping Bing with Python legal?

Scraping publicly visible Bing results with Python is generally treated as lower risk than scraping data behind a login, and US courts have leaned toward scrapers of public data. In hiQ Labs v. LinkedIn the Ninth Circuit found that accessing publicly available data is unlikely to be unauthorized access under the CFAA, which is the case most often cited on this question.

That is general information, not legal advice. You are still responsible for respecting Bing’s terms of use, its robots.txt position on /search, copyright in the content you collect, and personal-data rules such as the GDPR. Aggregate SERP research sits on far safer ground than republishing scraped content wholesale, and I go deeper on the rules in is scraping Bing legal.

What I would do

If I needed Bing data with Python tomorrow, I would match the tool to the volume. For a quick experiment I would run the requests-plus-BeautifulSoup parse above from a residential IP, follow the redirects to clean URLs, and accept the occasional challenge page. For anything ongoing I would not rebuild a proxy-and-CAPTCHA stack that Bing is designed to break, and I would not wait for the retired official API to return, because it will not. I would send the query to a scraper API, keep my code to the twenty lines that read the JSON, and spend the saved time on the data instead of the blocking.