{
  "version": 1,
  "slug": "scrape-impressum-python-api",
  "title": "How to Scrape Impressum Data from German Websites in Python Using an API",
  "excerpt": "Scrape German Impressum pages with Python and Datablist's API, then turn website URLs into structured, source-backed company research records.",
  "cover": {
    "src": "/howto_images/scrape-impressum-python-api/scrape-impressum-python-api-cover.png",
    "optimized": "https://www.datablist.com/_next/image?url=%2Fhowto_images%2Fscrape-impressum-python-api%2Fscrape-impressum-python-api-cover.png&w=1200&q=75"
  },
  "url": "https://www.datablist.com/how-to/scrape-impressum-python-api",
  "contentMarkdown": "\nTo scrape Impressum data in Python without building a custom crawler, use\nDatablist's Impressum Scraper API with the Requests library. You supply company\nwebsite URLs, the API finds each website's Impressum page, and it extracts\nstructured company and contact details from that page.\n\nFor this example, I use two established German brands: clothing manufacturer\nTrigema and outdoor brand VAUDE. A small runnable Python example performs the\nlookup, then keeps the extracted details and source Impressum URL with each\nsupplier record.\n\n> 📌 **Summary For Those In a Rush**\n>\n> Use Datablist's Impressum Scraper API to turn German business websites into\n> structured company and contact data. The Python example submits two websites,\n> waits for the background task, retrieves every result page, and keeps the\n> discovered legal-page URL with each record.\n>\n> - **Input:** German business website URLs.\n> - **Output:** Company and contact details plus the source Impressum link.\n\n> 💡 **Prefer a no-code solution?** Use Datablist's\n> **[Impressum Scraper](/enrichments/impressum-scraper)** to extract these\n> company details without writing code.\n\n## Set up your API key and Python environment {#set-up-your-api-key-and-python-environment}\nCreate a [Datablist account](https://app.datablist.com) and then create a\n[Personal API Key](https://app.datablist.com/account/api). REST API access is\navailable on the Growth plan. Here is a sample key format:\n\n```text\nauth0|exampleuser:exampleaccount:EXAMPLE_TOKEN_NOT_VALID:1\n```\n\nExport your own key as `DATABLIST_API_KEY`:\n\n```shell\nexport DATABLIST_API_KEY=\"your-personal-api-key\"\n```\n\nFor this Python example, I use the `requests` package. Create a file named\n`scrape_impressum.py` and install the package:\n\n```shell\npython -m pip install requests\n```\n\nEach submitted website uses 10 credits. The two-website example below uses 20\ncredits when both inputs are processed.\n\n## Build the `scrape_impressums()` function {#build-the-scrapeimpressums-function}\nPaste this complete script into `scrape_impressum.py`:\n\n```python\nimport json\nimport os\nimport time\nimport uuid\n\nimport requests\n\n\ndef scrape_impressums(suppliers, api_key):\n    \"\"\"Submit one task, wait for it to finish, and return every result.\"\"\"\n    tasks_url = \"https://api.datablist.com/v1/enrichments/tasks\"\n    headers = {\"X-API-Key\": api_key}\n    inputs = [\n        {\n            \"inputId\": supplier[\"id\"],\n            \"data\": {\"websiteUrl\": supplier[\"website_url\"]},\n        }\n        for supplier in suppliers\n    ]\n\n    payload = {\n        \"enrichmentId\": \"impressum-scraper\",\n        \"uniqueId\": str(uuid.uuid4()),\n        \"settings\": {},\n        \"inputs\": inputs,\n    }\n    response = requests.post(tasks_url, headers=headers, json=payload)\n    response.raise_for_status()\n    task = response.json()\n    task_id = task[\"@id\"]\n    task_url = f\"{tasks_url}/{task_id}\"\n\n    while task[\"status\"] not in (\"completed\", \"failed\", \"stopped\"):\n        time.sleep(3)\n        response = requests.get(task_url, headers=headers)\n        response.raise_for_status()\n        task = response.json()\n\n    if task[\"status\"] != \"completed\":\n        raise RuntimeError(\n            f\"Task {task_id} {task['status']}: {task.get('error')}\"\n        )\n\n    results = []\n    results_url = f\"{task_url}/results\"\n    params = {\"limit\": 100}\n\n    while True:\n        response = requests.get(results_url, headers=headers, params=params)\n        response.raise_for_status()\n        page = response.json()\n        results.extend(page[\"results\"])\n\n        next_cursor = page[\"nextCursor\"]\n        if next_cursor is None:\n            break\n        params[\"cursor\"] = next_cursor\n\n    return results\n\n\nif __name__ == \"__main__\":\n    api_key = os.environ.get(\"DATABLIST_API_KEY\")\n    if not api_key:\n        raise RuntimeError(\"Set DATABLIST_API_KEY before running this script.\")\n\n    suppliers = [\n        {\n            \"id\": \"supplier-trigema\",\n            \"label\": \"Trigema\",\n            \"website_url\": \"https://www.trigema.de\",\n        },\n        {\n            \"id\": \"supplier-vaude\",\n            \"label\": \"VAUDE\",\n            \"website_url\": \"https://www.vaude.com\",\n        },\n    ]\n\n    impressum_results = scrape_impressums(suppliers, api_key)\n    for result in impressum_results:\n        print(json.dumps(result, ensure_ascii=False))\n```\n\nRun the complete script with:\n\n```shell\npython scrape_impressum.py\n```\n\nThe function follows the API in three steps. It submits the website inputs,\nwaits for the task to finish, and then retrieves every result page.\n\n### Submit the website inputs {#submit-the-website-inputs}\nThe script turns each supplier into an API input like this:\n\n```python\ninputs = [\n    {\n        \"inputId\": supplier[\"id\"],\n        \"data\": {\"websiteUrl\": supplier[\"website_url\"]},\n    }\n    for supplier in suppliers\n]\n```\n\n`inputId` is your own stable identifier. `data.websiteUrl` is the company\nhomepage that the scraper will inspect. Use the business's own website rather\nthan a directory or social profile. The payload selects the public\n`impressum-scraper` enrichment and uses an empty `settings` object because this\nAPI does not require public settings.\n\nThe `POST` request creates a background task. Datablist accepts the request and\nreturns a task object whose `@id` identifies that run. The function uses that ID\nto build the status and result URLs.\n\nA shortened creation response looks like this:\n\n```json\n{\n  \"@id\": \"task-id\",\n  \"enrichmentId\": \"impressum-scraper\",\n  \"status\": \"queued\",\n  \"links\": {\n    \"self\": \"https://api.datablist.com/v1/enrichments/tasks/task-id\",\n    \"results\": \"https://api.datablist.com/v1/enrichments/tasks/task-id/results\"\n  }\n}\n```\n\n`queued` means the request was accepted, not that the result is ready. The\nscript carries the returned `@id` into the next two steps, so you do not have to\ncopy task URLs by hand. It also calls `raise_for_status()` before reading each\nresponse. An authentication or validation error therefore stops at the HTTP\nrequest that failed instead of being mistaken for a completed lookup.\n\n### Wait for the background task {#wait-for-the-background-task}\nDatablist processes the task in the background. The example checks its status\nevery three seconds until it reaches `completed`, `failed`, or `stopped`.\nRequests is a blocking Python library, so your local script waits during these\ncalls even though the API task itself runs in the background.\n\nIf the task finishes with `failed` or `stopped`, the function raises an error\nthat includes the task ID and terminal status. Keep that ID when investigating\nthe existing task rather than immediately submitting the same paid work again.\n\n### Retrieve every result page {#retrieve-every-result-page}\nOnce the task is complete, the script requests up to 100 results at a time. The\npagination rule is simple: **fetch the next page until `nextCursor` is `null`.**\nWhen the cursor contains a value, the script sends it with the next request.\n\nDo not join the returned records to your application by list position. Results\ncan arrive in a different order from the submitted inputs. Use `inputId` as the\njoin key.\n\n## How the API finds an Impressum page {#how-the-api-finds-an-impressum-page}\nThe homepage is the starting point, not the page the API expects to extract.\nThe Impressum Scraper uses AI to inspect internal links on the supplied host\nand identify an Impressum or an equivalent legal or contact page. This matters\nbecause the link may sit under labels such as *Rechtliches*, *Kontakt*, or\n*Legal* instead of appearing at `/impressum`.\n\nIf AI does not find a suitable internal link, the scraper tries the common\n`/impressum` and `/imprint` paths on the same host. After it reaches a source\npage, AI extracts the legal-company, address, contact, registration, VAT, and\nsource fields into one result. It does not use a search engine or move to an\nexternal directory.\n\n![Impressum Scraper follows a German website's legal link or common path before extracting structured company details and the source Impressum URL](/howto_images/scrape-impressum-python-api/illustrations/impressum-discovery-and-extraction.png)\n\nThe two results show that path variation in practice. Trigema returned\n`/kundenservice/rechtliches/impressum/`, while VAUDE returned\n`/de/de/impressum.html`. Keep `impressumUrl` with each result because it records\nthe exact source page behind the extracted fields.\n\n## Read the Trigema and VAUDE results {#read-the-trigema-and-vaude-results}\nHere is a compact excerpt from the two example results.\n\n> 📘 **Example email addresses:** Emails are masked with `**` below. The API\n> returns the full addresses.\n\n```json\n[\n  {\n    \"inputId\": \"supplier-trigema\",\n    \"status\": \"success\",\n    \"outputs\": {\n      \"firstName\": \"Wolfgang\",\n      \"lastName\": \"Grupp\",\n      \"companyName\": \"Trigema W. Grupp KG\",\n      \"postalAddress\": \"Josef-Mayer-Str. 31-35, D-72393 Burladingen\",\n      \"emailAddress\": \"b**@trigema.de\",\n      \"registrationNumber\": \"HRA 740365\",\n      \"vatNumber\": \"DE144846566\",\n      \"impressumUrl\": \"https://www.trigema.de/kundenservice/rechtliches/impressum/\",\n      \"agentConfidenceScore\": 90\n    },\n    \"creditsUsed\": \"10\",\n    \"error\": null\n  },\n  {\n    \"inputId\": \"supplier-vaude\",\n    \"status\": \"success\",\n    \"outputs\": {\n      \"firstName\": \"Antje\",\n      \"lastName\": \"von Dewitz\",\n      \"companyName\": \"VAUDE Sport GmbH & Co. KG\",\n      \"postalAddress\": \"Vaude Straße 2, 88069 Tettnang\",\n      \"emailAddress\": \"s**@service.vaude.com\",\n      \"registrationNumber\": \"HRA 631071\",\n      \"vatNumber\": \"DE203894755\",\n      \"impressumUrl\": \"https://www.vaude.com/de/de/impressum.html\",\n      \"agentConfidenceScore\": 95\n    },\n    \"creditsUsed\": \"10\",\n    \"error\": null\n  }\n]\n```\n\nStart with the names. `Trigema` is the supplier label in the Python input, while\n`Trigema W. Grupp KG` is the legal company name extracted from the Impressum.\nThe same distinction appears for VAUDE: the input label is `VAUDE`, and the\nreturned company is `VAUDE Sport GmbH & Co. KG`. Keeping both values lets you\nretain the recognizable name used by your team while adding the legal entity\nshown on the company's own website. This is especially useful when comparing a\nsupplier record with an invoice, contract, or another company dataset that may\nuse the longer legal name.\n\nThe returned fields answer different research questions:\n\n- `firstName` and `lastName` contain the managing-director or founder name\n  extracted from the page when found. Here they are Wolfgang Grupp and Antje\n  von Dewitz.\n- `postalAddress` gives the business address extracted from the legal page,\n  while the email and phone fields provide the contact details published there.\n- `registrationNumber` and `vatNumber` preserve the identifiers printed on the\n  page. For Trigema, those values are `HRA 740365` and `DE144846566`; for VAUDE,\n  they are `HRA 631071` and `DE203894755`. They give you concrete values to\n  compare with the identifiers already stored in your supplier data.\n- `impressumUrl` identifies the page used for extraction. It is separate from\n  the submitted homepage because one records the business you started with and\n  the other records the source of the legal-company details.\n\nBoth records have `status: success` and used 10 credits. The\n`agentConfidenceScore` is a 0 to 100 signal that helps you prioritize review. It\nis not an accuracy percentage. Output fields can still be empty when a value is\nabsent from the page or cannot be extracted, so keep the item status and open\nthe returned Impressum URL when a detail matters to your next step.\n\nThe documented output can also include gender, phone number, and other contact\ndetails. See the\n[Impressum Scraper API documentation](https://platform.datablist.com/docs/enrichments/tools/impressum-scraper)\nfor the current schema.\n\n## Turn the output into supplier research records {#turn-the-output-into-supplier-research-records}\nThe direct API call already works. Now put it in the context of a supplier\nresearch task. Supplier discovery or onboarding may give you a stable ID, the\nshort name your team uses, and a homepage. The Impressum result adds the legal\ncompany behind that website, its published identifiers and contact details,\nand the page where those values appeared.\n\nThe following snippet indexes the results by `inputId` and builds local\ncompany-research records. Add it after the `scrape_impressums()` call in the\ncomplete script:\n\n```python\nresults_by_id = {\n    result[\"inputId\"]: result for result in impressum_results\n}\ncompany_research = []\n\nfor supplier in suppliers:\n    result = results_by_id[supplier[\"id\"]]\n    company_research.append(\n        {\n            \"supplier_id\": supplier[\"id\"],\n            \"supplier_label\": supplier[\"label\"],\n            \"website_url\": supplier[\"website_url\"],\n            \"status\": result[\"status\"],\n            \"outputs\": result.get(\"outputs\", {}),\n            \"credits_used\": result[\"creditsUsed\"],\n            \"error\": result.get(\"error\"),\n        }\n    )\n\nprint(json.dumps(company_research, indent=2, ensure_ascii=False))\n```\n\nFor Trigema, the record preserves `supplier-trigema`, the `Trigema` display\nlabel, and `https://www.trigema.de`. The nested `outputs` object adds\n`Trigema W. Grupp KG`, its address and identifiers, Wolfgang Grupp's returned\nname, and the source Impressum URL. The VAUDE record follows the same chain from\nthe `VAUDE` label and homepage to `VAUDE Sport GmbH & Co. KG` and its own source\npage. You can now compare or group records by legal company without losing the\noriginal supplier identity that triggered the research.\n\nThe record keeps the returned details, successful API status, 10-credit usage,\nand public error value together with the supplier ID, label, and website.\n\nThe important merge rule is the dictionary lookup by supplier ID. It still\nconnects the right result when the API returns VAUDE before Trigema. The snippet\nalso keeps unsuccessful item results in `company_research` because it does not\nfilter by status. A `no_result` or `error` item remains available with its ID\nand error instead of disappearing from your review queue.\n\nThis code prepares local research records. You can serialize the list or pass\nit to your own downstream code without adding another API call.\n\n## Estimate the API cost {#estimate-the-api-cost}\nThe Impressum Scraper uses a fixed 10 credits per submitted website. The two\nexample websites used 20 credits in total. At the same rate, 1,000 websites use\n10,000 credits.\n\nThe Growth plan provides REST API access and includes subscription credits.\nOptional top-ups add more credits when you need them. Plan access, included\ncredits, and top-up purchases are separate parts of the cost, so check the\n[current Datablist pricing](/pricing) before planning a larger run.\n\n<div format=\"cost-calculator\"\n     data-heading=\"Estimate your API cost\"\n     data-unit-label=\"Websites\"\n     data-credits-per-unit=\"10\"\n     data-lower-rate-label=\"Impressum scraping\"\n     data-initial-quantity=\"1000\"\n     data-max-quantity=\"100000\">\n<p>Each submitted website uses 10 credits.</p>\n</div>\n\n## Handle common problems without repeating paid work {#handle-common-problems-without-repeating-paid-work}\n### The script stops before making a request {#the-script-stops-before-making-a-request}\nIf `DATABLIST_API_KEY` is missing, the script raises `Set DATABLIST_API_KEY\nbefore running this script.` Export the variable in the same terminal and run\nthe file again:\n\n```shell\nexport DATABLIST_API_KEY=\"your-personal-api-key\"\npython scrape_impressum.py\n```\n\n### The API rejects the request {#the-api-rejects-the-request}\n`response.raise_for_status()` stops on an HTTP error. Check that you supplied a\nvalid Personal API Key and that your account has REST API access. This differs\nfrom the missing-variable case: an invalid value reaches the API and receives\nan HTTP response.\n\n### The task fails or stops {#the-task-fails-or-stops}\nThe raised error contains the task ID and status. Inspect that existing task,\nits results, and its credit fields before creating another task. A failed or\nstopped task can retain results and credit usage, so repeating it immediately\ncan duplicate paid work.\n\n### One result is incomplete {#one-result-is-incomplete}\nA completed task can contain item-level `no_result`, `error`, or incomplete\noutputs. Keep the record's `inputId`, `status`, `impressumUrl` when available,\nand public `error`. Check that the input is the company's own homepage or root\ndomain, then inspect the site before deciding whether another lookup is useful.\n\nThe [task documentation](https://platform.datablist.com/docs/enrichments/tasks)\nexplains statuses, result pagination, task history, limits, and retention in\nmore detail.\n\n## Next steps {#next-steps}\nYou now have a runnable Python Requests integration that submits German company\nwebsites, waits for the Impressum Scraper task, retrieves every result page, and\nkeeps structured company details with their source URLs in local supplier\nresearch records.\n\nReplace the two sample suppliers with your own stable IDs and website URLs.\nThen review each returned `impressumUrl` before using the extracted fields in\nyour downstream process. You can download the maintained\n[synchronous Requests example](https://github.com/datablist/impressum-scraper-api/blob/main/python/sync_requests.py)\nor switch to the repository's\n[async HTTPX example](https://github.com/datablist/impressum-scraper-api/blob/main/python/async_httpx.py)\nwhen you need non-blocking concurrency."
}