#HighPrice
November 11, 2026 at 1:15 PM
November 11, 2026 at 1:15 PM
November 11, 2026 at 1:15 PM
Scrape Product Prices with Python from JSON-LD
To scrape product prices with Python, fetch the page with `requests`, parse every `<script type="application/ld+json">` tag with `json.loads`, find the schema.org `Product` nodes, and read `offers`: `price`, `priceCurrency` and `availability`. Handle the real-world shapes: `@graph` arrays, lists of offers, `AggregateOffer` (`lowPrice`/`highPrice`), prices inside `priceSpecification`, and `ProductGroup` variants. Below is a script that handles all of these, its output on live stores on 2026-10-20, the stores where the same approach returned nothing that day, and how E-commerce Product Scraper Pro handles the cases plain JSON-LD can't. ## Why JSON-LD Many stores embed schema.org product data in their HTML for search engines, usually as JSON-LD. It's machine-readable data the store publishes on purpose, so it changes less often than the page's CSS classes. The fields you need are defined by schema.org: * Offer: `price`, `priceCurrency`, `availability` (a URL such as `https://schema.org/InStock`), often `sku` and `url`. * AggregateOffer: used "when a single product is associated with multiple offers", with `lowPrice`, `highPrice` and `offerCount`. * ProductGroup: "a group of Products that vary only in certain well-described ways, such as by size, color", with the variants in `hasVariant`. ## The shapes you'll actually meet Every row below is a live page I parsed on 2026-10-20: Store (platform) | Where the price was ---|--- IKEA US | A plain `Offer` with `"price": "79"` as a string, `priceCurrency` alongside Marks & Spencer | No top-level `price`: it's in `offers.priceSpecification` (a `UnitPriceSpecification` with `price`, `minPrice`, `maxPrice`, `priceCurrency`) Nalgene (WooCommerce) | Inside `@graph`; a `ProductGroup` whose `hasVariant` items each have an offer; `priceSpecification` serialised as an object with a `"0"` key holding the currency Death Wish Coffee (Shopify) | One `Product` with a list of `Offer`s, one per variant, each with its own `sku` and price Skullcandy (Shopify) | A `ProductGroup` with `hasVariant`, plus a separate `Product` for the default variant A parser that only reads `data["offers"]["price"]` gets two of these five. The script walks the whole document instead. ## The script """Extract product prices from schema.org JSON-LD. pip install requests beautifulsoup4""" import json import sys import requests from bs4 import BeautifulSoup HEADERS = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " "(KHTML, like Gecko) Chrome/141.0 Safari/537.36", "Accept-Language": "en-US,en;q=0.9", } def walk(node): """Yield every dict in a JSON-LD document: top-level lists, @graph, nested values.""" if isinstance(node, list): for item in node: yield from walk(item) elif isinstance(node, dict): yield node for value in node.values(): if isinstance(value, (dict, list)): yield from walk(value) def has_type(node, name): t = node.get("@type") return any(str(x).rsplit("/", 1)[-1] == name for x in (t if isinstance(t, list) else [t]) if x) def number(value): try: return float(str(value).replace(",", "")) if value not in (None, "") else None except ValueError: return None def price_spec(spec): """priceSpecification can be a dict, a list, or (WooCommerce) a dict with a "0" key.""" specs = spec if isinstance(spec, list) else [spec] for s in specs: if not isinstance(s, dict): continue nested = s.get("0") price = s.get("price", nested.get("price") if isinstance(nested, dict) else None) currency = s.get("priceCurrency") or (nested.get("priceCurrency") if isinstance(nested, dict) else None) if price is not None: return number(price), currency return None, None def offers(product): """One dict per offer: handles Offer, a list of Offers and AggregateOffer.""" raw = product.get("offers") or [] out = [] for o in raw if isinstance(raw, list) else [raw]: if not isinstance(o, dict): continue if has_type(o, "AggregateOffer") and o.get("offers"): out += offers({"offers": o["offers"]}) # real offers nested inside continue price, currency = number(o.get("price")), o.get("priceCurrency") if has_type(o, "AggregateOffer"): price = number(o.get("lowPrice")) if price is None and "priceSpecification" in o: price, spec_currency = price_spec(o["priceSpecification"]) currency = currency or spec_currency out.append({ "price": price, "high_price": number(o.get("highPrice")), # AggregateOffer only "currency": currency, "availability": str(o.get("availability") or "").rsplit("/", 1)[-1] or None, "sku": o.get("sku") or product.get("sku"), "url": o.get("url"), }) return out def extract(html): """All (product name, offer) pairs in the page's JSON-LD, de-duplicated.""" soup = BeautifulSoup(html, "html.parser") rows, seen = [], set() for tag in soup.find_all("script", type="application/ld+json"): try: doc = json.loads(tag.string or "") except json.JSONDecodeError: continue # malformed JSON-LD is common; skip it for node in walk(doc): # also reaches ProductGroup.hasVariant if not has_type(node, "Product"): continue for o in offers(node): key = (o["sku"], o["price"], o["url"]) if o["price"] is not None and key not in seen: seen.add(key) rows.append({"name": node.get("name"), **o}) return rows def fetch(url): try: r = requests.get(url, headers=HEADERS, timeout=20) except requests.RequestException as e: return None, f"request failed: {type(e).__name__}" return r, f"HTTP {r.status_code}" if __name__ == "__main__": for url in sys.argv[1:]: r, status = fetch(url) rows = extract(r.text) if r is not None and r.ok else [] print(f"\n{url}\n {status}, {len(rows)} offer(s) in JSON-LD") for row in rows[:3]: print(f" {row['name'][:45]!r} | {row['sku']} | {row['price']} {row['currency']} | {row['availability']}") Design notes: * **`walk()` visits every dict**, so it finds `Product` nodes at the top level, in a list, inside `@graph`, and inside `ProductGroup.hasVariant` without special cases. The `seen` set removes the duplicates that this produces when a page describes the same variant twice (Skullcandy does). * **`has_type()` accepts lists and full URLs**, because `@type` can be `["Product", "Thing"]` or `"http://schema.org/Product"`. * **`number()` handles strings**, because `"79"` and `79` are both common. * **Bad JSON is skipped, not fatal.** Some stores publish invalid JSON-LD. * **A browser-like`User-Agent`** is sent because several stores answer the default `python-requests` agent differently. It doesn't get you past real bot protection (see below). Run it with Python 3.12 (`pip install requests beautifulsoup4`): python product_prices.py https://www.ikea.com/us/en/p/billy-bookcase-white-20522046/ https://www.marksandspencer.com/fair-isle-crew-neck-relaxed-jumper-with-wool/p/clp61248839 https://nalgene.com/product/32oz-wide-mouth-surrealist-print-bottles/ https://www.deathwishcoffee.com/products/power-surge-coffee https://www.skullcandy.com/products/crusher-540-active Output on 2026-10-20: https://www.ikea.com/us/en/p/billy-bookcase-white-20522046/ HTTP 200, 1 offer(s) in JSON-LD 'BILLY Bookcase - white 31 1/2x11x79 1/2 "' | 205.220.46 | 79.0 USD | InStock https://www.marksandspencer.com/fair-isle-crew-neck-relaxed-jumper-with-wool/p/clp61248839 HTTP 200, 1 offer(s) in JSON-LD 'Fair Isle Crew Neck Relaxed Jumper' | P61248839 | 34.0 GBP | InStock https://nalgene.com/product/32oz-wide-mouth-surrealist-print-bottles/ HTTP 200, 3 offer(s) in JSON-LD '32oz Wide Mouth Surrealist Bottles - landscap' | 682024-0052 | 18.99 USD | None '32oz Wide Mouth Surrealist Bottles - hummingb' | 682024-0051 | 18.99 USD | None '32oz Wide Mouth Surrealist Bottles - astro-mo' | 682024-0050 | 18.99 USD | None https://www.deathwishcoffee.com/products/power-surge-coffee HTTP 200, 8 offer(s) in JSON-LD 'Power Surge Coffee' | LGPWSR16 | 19.99 USD | InStock 'Power Surge Coffee' | LGPWSR32K | 39.98 USD | InStock 'Power Surge Coffee' | LGPWSR48K | 59.97 USD | InStock https://www.skullcandy.com/products/crusher-540-active HTTP 200, 6 offer(s) in JSON-LD 'Crusher® 540 Active - Skullcandy X Baker Skat' | S6EVW-U054 | 229.99 USD | InStock 'Crusher® 540 Active - Smoke' | S6EVW-T013 | 209.99 USD | InStock 'Crusher® 540 Active - Forest' | S6EVW-T016 | 209.99 USD | InStock Five stores, five shapes, one parser. Nalgene's variant offers carry no `availability`, so it prints `None`. That is "unknown", not "out of stock". ### Testing the AggregateOffer branch None of the pages I tried on 2026-10-20 used `AggregateOffer`, so the branch is tested on a small inline document shaped like schema.org's definition, together with a broken tag: from product_prices import extract html = """ <script type="application/ld+json"> {"@context": "https://schema.org", "@type": "Product", "name": "Demo kettle", "sku": "K-1", "offers": {"@type": "AggregateOffer", "lowPrice": "39.99", "highPrice": "54.00", "offerCount": 4, "priceCurrency": "USD"}} </script> <script type="application/ld+json">{"@type": "Product", "name": "Broken", </script> """ for row in extract(html): print(row) {'name': 'Demo kettle', 'price': 39.99, 'high_price': 54.0, 'currency': 'USD', 'availability': None, 'sku': 'K-1', 'url': None} An `AggregateOffer` is a price range, not a price. Store `lowPrice` and `highPrice` separately and don't compare a range with a single price in a price-change alert. ## Where it breaks The same script, same day, same connection (one home connection outside the US; results depend on where and when you run it): https://www.target.com/p/lego-marvel-spider-man-vs-anti-venom-heist-30725/-/A-94828796 HTTP 200, 0 offer(s) in JSON-LD https://www.amazon.com/dp/B088NRLMPV HTTP 200, 0 offer(s) in JSON-LD https://www.sephora.com/product/peptide-glazing-fluid-P517531 HTTP 403, 0 offer(s) in JSON-LD https://www.zalando.de/puma-club-ii-era-unisex-sneaker-low-whiteblackvapor-gray-pu115o0sb-a11.html HTTP 403, 0 offer(s) in JSON-LD Each failure is a different problem: 1. **The data is in the app's state, not JSON-LD.** Target's and Walmart's product pages returned HTTP 200 with the product title, but no Product JSON-LD in what `requests` received. Both pages contained a `__NEXT_DATA__` script (Target also `__TGT_DATA__`): the page is built from a JavaScript state object. You'd parse that object per site, or render the page. 2. **The store doesn't use JSON-LD for the price.** Amazon's page returned 200 with the price in its HTML (`a-price` elements), not in JSON-LD. That needs site-specific parsing. 3. **Bot protection.** Sephora and Zalando answered 403; John Lewis and Waitrose didn't answer at all within 20 seconds (`ReadTimeout`) in earlier tries. A different User-Agent doesn't fix this; it needs other network routes or a real browser, and respect for the site's terms. 4. **Variants.** Some pages describe only the selected variant. Nalgene and Death Wish publish every variant; other stores don't, and then sizes and colours need the store's own product data. 5. **Pagination and discovery.** JSON-LD describes one product page. To cover a category you need its product links first, through page parameters, "load more" APIs or the store's search, and a rule for when you've seen everything. ## Scaling up: the same pages through the actor The actor starts with the same structured data and adds what the script lacks: per-site adapters (Shopify and WooCommerce product APIs, Walmart's page data, Amazon's HTML), a route ladder that moves from datacenter to a crawler User-Agent, residential proxies or a browser only when a page is blocked, variants as `variants[]`, pagination up to 200 pages per listing where the site supports it, and one status row per input explaining any failure. The README's supported-sites matrix says what works where; Sephora product pages are best-effort through a browser, and eBay, Best Buy, Home Depot, Costco and Etsy are unsupported. """Same pages through the Apify actor. pip install apify-client""" import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("constructive_calm/ecommerce-product-scraper-pro").call( run_input={ "detailsUrls": [ {"url": "https://www.marksandspencer.com/fair-isle-crew-neck-relaxed-jumper-with-wool/p/clp61248839"}, {"url": "https://nalgene.com/product/32oz-wide-mouth-surrealist-print-bottles/"}, {"url": "https://www.target.com/p/lego-marvel-spider-man-vs-anti-venom-heist-30725/-/A-94828796"}, {"url": "https://www.amazon.com/dp/B088NRLMPV"}, ], "maxProductResults": 4, }, max_total_charge_usd=0.10, # hard cap on what this run may cost ) for row in client.dataset(run["defaultDatasetId"]).iterate_items(): print(f"{row['site']:<16} {row['price']:>7} {row['currency']} inStock={row['inStock']}" f" variants={row['variantCount']:<2} tier={row['tierUsed']:<11} from={'+'.join(row['dataSources'])}") Output on 2026-10-20 (the client also streams the run's log while it waits): marksandspencer 34 GBP inStock=True variants=0 tier=datacenter from=jsonld+meta target 4.99 USD inStock=True variants=0 tier=crawler-ua from=jsonld+meta amazon-us 9.99 USD inStock=True variants=12 tier=datacenter from=amazon-html nalgene.com 18.99 USD inStock=True variants=3 tier=datacenter from=jsonld+woo-api `dataSources` shows where each row came from. Target delivered through the crawler User-Agent route, where its page includes JSON-LD. Amazon came from its HTML. Nalgene combined JSON-LD with the WooCommerce Store API, and its row has a stock flag (`inStock=True`) where the page's JSON-LD had none. Cost: the actor is pay per event, and these four products are one start plus four product rows on datacenter routes (no traffic fee): 4 × $0.0048 + $0.005 = **$0.0242** on Apify's FREE tier, 4 × $0.0008 + $0.005 = **$0.0082** on GOLD, from the README's pricing table. Residential and browser sites add per-MB and per-render fees. Apify's free plan gives $5 of usage a month with no credit card required; there's no in-actor free trial. ## During Black Friday and the holiday season The script is enough to start a baseline for a handful of stores that serve JSON-LD: run it once a day from cron or Task Scheduler and append the rows to a CSV, beginning now, about five weeks before Black Friday (Friday 27 November 2026). In the sale window, prices change several times a day, so increase the frequency for the few products you care about and keep the daily readings for medians. Compare the sale price with your own 30-day median, not with the store's "was" price: how to spot fake Black Friday discounts shows the method, and the holiday sales calendar lists when each event's baseline should start. Related: scraping product variants, Shopify products.json pagination, competitor price monitoring, and the e-commerce price tracker query builder. Respect each site's terms and robots rules, and keep request rates low. _Disclosure: I built the actor and this site. The JSON-LD script above works on its own, without either._ Sources: schema.org Offer, schema.org AggregateOffer, schema.org ProductGroup, E-commerce Product Scraper Pro README (supported sites, pricing), and live requests to the listed product pages on 2026-10-20.
dev.to
October 20, 2026 at 10:16 AM
July 1, 2025 at 3:57 PM
Because of the #HighPrice of #groceries, there are people who've decided to start #VegetableGardens. I'm looking into doing that myself.
amzn.to/3FwZ7Nn

#vegetables #gardens #garden #gardening #homegrown #seeds #food #affordablefood #lowcost
June 9, 2025 at 3:15 PM
In case you missed it: there's a pretty huge catch in @exxonmobil's support for a carbon tax... Immunity from all climate lawsuits in the future. @voxdotcom @umairfan #highprice #carbontax #climatechange #corporateresponsibility

buff.ly/2tPaMyw
Exxon is lobbying for a carbon tax. There is, obviously, a catch.
The oil giant wants immunity from lawsuits that would make it pay for the damages of climate change.
buff.ly
December 3, 2024 at 4:11 PM
There's a pretty huge catch in @exxonmobil's support for a carbon tax...
Immunity from all climate lawsuits in the future.

@voxdotcom @umairfan #highprice #carbontax #climatechange #corporateresponsibility

ow.ly/Ws5Z30miFoQ
Exxon is lobbying for a carbon tax. There is, obviously, a catch.
The oil giant wants immunity from lawsuits that would make it pay for the damages of climate change.
ow.ly
December 3, 2024 at 4:05 PM
Turkey discounts offer relief, but bird flu, supply disruptions and tariffs continue to push other food prices higher
#HighPrice #BirdFlu #TurkeyDiscounts

Read More: www.ibtimes.co.uk/thanksgiving...
November 25, 2025 at 11:05 AM
Beef, it’s NOT for dinner.

#beef #inflation #HighPrice
June 13, 2026 at 3:22 PM
I’ve been with Xfinity for years, but the price increase 2 years ago was significant. It is nearly impossible to reach a real person. It’s frustrating that loyal customers don’t get better pricing, I may have to switch. Xfinity, it is becoming difficult to stay with you. #xfinity #Highprice #at&t
May 3, 2025 at 5:05 PM
Wipf & Stock's annual conference sale ends tonight. Last chance to get What Jesus Learned from Women, Theology and Science Fiction, or The Burial of Jesus for half off plus free shipping with coupon code CONFSHIP !

wipfandstock.com/search-resul...
Theology & Science Fiction
Showing results 1-10 of 32 Books Journals Site Content Filter Results OPEN + Sort by:RelevanceNewest to OldestOldest to NewestTitle - A to ZTitle - Z to APrice - Low to HighPrice - High to LowAuthor -...
wipfandstock.com
November 30, 2025 at 8:15 PM
I should also mention that two books by my pastor, Rev. Dr. Michael Robinson, are on sale as part of the current Wipf & Stock sale.

wipfandstock.com/search-resul...
Christianity: A Brief Survey
Showing results 1-10 of 101 Books Journals Site Content Filter Results OPEN + Sort by:RelevanceNewest to OldestOldest to NewestTitle - A to ZTitle - Z to APrice - Low to HighPrice - High to LowAuthor...
wipfandstock.com
November 16, 2025 at 2:15 AM
Wipf and Stock has opened its conference discount to everyone!

That means you get 50% off all my books (and any other Wipf and Stock books) plus free shipping using the coupon code CONFSHIP

wipfandstock.com/search-resul...
What Jesus Learned from Women, Theology and Science Fiction, The Burial of Jesus, & More!
Showing results 1-8 of 8 Books Journals Site Content Filter Results OPEN + Sort by:RelevanceNewest to OldestOldest to NewestTitle - A to ZTitle - Z to APrice - Low to HighPrice - High to LowAuthor - A...
wipfandstock.com
October 23, 2025 at 2:01 AM
Hey, Highprice two dolls, oil at $60.00, prices jacked to the ceiling!
Where’s your outrage?
Or do you just lie and say they’re down?

Lie and take my cut, of course!

Thought so
May 10, 2025 at 6:39 PM
Gabigol's Match Worn Boots Auctioned for Over 16K Euros to Lucas Tylty for Charity

#auction #Footballboots #Gabigol #HighPrice #memorabilia
Gabigol's Match Worn Boots Auctioned for Over 16K Euros to Lucas Tylty for Charity
Sometimes a simple piece of sports gear becomes a treasure. That is exactly what happened with a pair of cleats worn by star striker Gabigol. These boots,
blazetrends.com
October 9, 2025 at 9:31 AM
The inflation report shows, egg prices reached a new record high.
A dozen eggs averages $6.23.
Prices remained high, despite wholesale prices falling and no farms reporting an outbreak of bird flu.
The high demand through Easter could keep prices high for a while
#egg #eggprices #grocery #highprice
April 10, 2025 at 6:09 PM
Feed: "Horror Sleaze Trash"
Published on Sunday, June 8, 2025
Casey Renee Kiser
Ohh Snap, Now Add it Up! Oh, I fell in love with a broken calculator-con manIt was sure a highprice for cheap, apocalyptic love, manWell yeah, that’s ok, you know,‘cause I’m a holy rubber band My h…
horrorsleazetrash.com
June 9, 2025 at 7:19 AM
Anyone who says you can't put a price on love has not seen a matchmaking agency price schedule. #dating #matchmaking #alot #highprice #expensive #unaffordable #service #lookingforlove #cheapertohireafssw #seriously #exploitation #preyingondesperatepeople
November 22, 2025 at 11:26 PM
Very misleading Post by #DNCChair Ken Martin about the highprice of beef @minnpost.bsky.social @startribune.com A simple google search shoes why beef is high. Mainly A long drought in the Southwest where feed is grown. #resisters Is this post by Ken Helpful or a MAGA like tactic to the Base
August 1, 2025 at 12:17 PM
trump has almost destroyed the tourist industry already. First is was fear among travelers of being grabbed by ICE and lost into the detension system, then it was the highprice of admission and now these new rules and charge. He really doesn't plan for the consequences of his midnight rages
December 14, 2025 at 4:07 AM