{ "version": 1, "slug": "scrape-linkedin-profiles-python-api", "title": "Scrape LinkedIn Profiles in Python Using an API", "excerpt": "Scrape LinkedIn profile details from known URLs in Python, retrieve role and company data, and build structured lead records for personalized outreach.", "cover": { "src": "/howto_images/scrape-linkedin-profiles-python-api/scrape-linkedin-profiles-python-api-cover.png", "optimized": "https://www.datablist.com/_next/image?url=%2Fhowto_images%2Fscrape-linkedin-profiles-python-api%2Fscrape-linkedin-profiles-python-api-cover.png&w=1200&q=75" }, "url": "https://www.datablist.com/how-to/scrape-linkedin-profiles-python-api", "contentMarkdown": "\nYou can scrape LinkedIn profiles in Python by sending known public profile URLs\nto Datablist's **LinkedIn Profile Scraper API** with the Requests library. The API\nreturns structured identity, location, work history, education, and company\nfields for building lead lists and personalizing outreach with relevant role and\ncompany context.\n\nThis guide uses Satya Nadella's public profile URL as a compact example. You will\nrun a complete profile-scraping task, interpret the returned Microsoft role and\ndated work entries, then keep the result with your source lead before\npreparing outreach.\n\n> 🔑 **Summary For Those In a Rush:** Send known LinkedIn person-profile URLs to\n> Datablist and retrieve available profile, location, work-history, education,\n> and company data as structured JSON. The Python example below keeps those\n> details with each source lead so you can qualify the list and prepare more\n> relevant outreach.\n>\n> - **Input:** Known LinkedIn person-profile URLs.\n> - **Output:** Structured profile details stored with each source contact.\n\n> 💡 **Prefer a no-code solution?** Use Datablist's **[LinkedIn Profile\n> Scraper](/enrichments/linkedin-profile-scraper-bulk)** to extract details from\n> known profile URLs without writing code.\n\n## Set up the Datablist API and Python environment {#set-up-the-datablist-api-and-python-environment}\nDatablist REST API access is included with the Growth plan. Profile retrieval\nalso consumes enrichment credits, which I break down in the cost section.\n\nCreate a Personal API Key from your [Datablist API account\npage](https://app.datablist.com/account/api). This example shows the Personal\nAPI Key format:\n\n```text\nauth0|exampleuser:exampleaccount:EXAMPLE_TOKEN_NOT_VALID:1\n```\n\nThe script reads your key from the `DATABLIST_API_KEY` environment variable and\nsends it in the `X-API-Key` header.\n\nFor this Python example, I use the `requests` package. These setup commands use\na macOS or Linux shell. Create a project folder, open it in your terminal, and\nrun:\n\n```shell\npython -m venv .venv\nsource .venv/bin/activate\npython -m pip install requests\n```\n\nNext, create a file named `scrape_linkedin_profiles.py` and export your key:\n\n```shell\nexport DATABLIST_API_KEY=\"your-personal-api-key\"\n```\n\n## Run a complete LinkedIn profile scrape in Python {#run-a-complete-linkedin-profile-scrape-in-python}\nThe source record in this walkthrough contains a stable contact ID and one known\nprofile URL:\n\n```json\n{\n \"contact_id\": \"contact-satya-nadella\",\n \"profile_url\": \"https://www.linkedin.com/in/satyanadella/\"\n}\n```\n\nThe script sends `contact_id` as the API's `inputId`. Results do not need to\narrive in the same order as the inputs, so this ID gives you a reliable way to\nmatch every result to its source contact.\n\nPaste the following complete script into `scrape_linkedin_profiles.py`:\n\n```python\nimport json\nimport os\nimport time\nimport uuid\n\nimport requests\n\n\ndef scrape_profiles(source_contacts, api_key):\n \"\"\"Submit one task, wait for completion, and return every result page.\"\"\"\n tasks_url = \"https://api.datablist.com/v1/enrichments/tasks\"\n headers = {\"X-API-Key\": api_key}\n inputs = [\n {\n \"inputId\": contact[\"contact_id\"],\n \"data\": {\"profileUrl\": contact[\"profile_url\"]},\n }\n for contact in source_contacts\n ]\n\n unique_id = str(uuid.uuid4())\n print(f\"Task uniqueId: {unique_id}\", flush=True)\n\n payload = {\n \"enrichmentId\": \"linkedin-profile-scraper\",\n \"uniqueId\": unique_id,\n \"settings\": {\n \"experiencesNb\": \"5\",\n \"educationsNb\": \"3\",\n \"acceptCachedProfiles\": False,\n \"includeUrn\": False,\n \"includePictures\": False,\n },\n \"inputs\": inputs,\n }\n\n response = requests.post(tasks_url, headers=headers, json=payload)\n response.raise_for_status()\n task = response.json()\n task_id = task[\"@id\"]\n task_url = f\"{tasks_url}/{task_id}\"\n print(f\"Task ID: {task_id}\")\n\n while task[\"status\"] not in (\"completed\", \"failed\", \"stopped\"):\n time.sleep(3)\n response = requests.get(task_url, headers=headers)\n response.raise_for_status()\n task = response.json()\n\n if task[\"status\"] != \"completed\":\n raise RuntimeError(\n f\"Task {task_id} {task['status']}: {task.get('error')}\"\n )\n\n results = []\n results_url = f\"{task_url}/results\"\n params = {\"limit\": 100}\n\n while True:\n response = requests.get(results_url, headers=headers, params=params)\n response.raise_for_status()\n page = response.json()\n results.extend(page[\"results\"])\n\n next_cursor = page[\"nextCursor\"]\n if next_cursor is None:\n break\n params[\"cursor\"] = next_cursor\n\n return results\n\n\nif __name__ == \"__main__\":\n api_key = os.environ.get(\"DATABLIST_API_KEY\")\n if not api_key:\n raise RuntimeError(\"Set DATABLIST_API_KEY before running this script.\")\n\n source_contacts = [\n {\n \"contact_id\": \"contact-satya-nadella\",\n \"profile_url\": \"https://www.linkedin.com/in/satyanadella/\",\n }\n ]\n\n profile_results = scrape_profiles(source_contacts, api_key)\n print(json.dumps(profile_results, indent=2))\n```\n\nRun it with:\n\n```shell\npython scrape_linkedin_profiles.py\n```\n\nThe settings request five work experiences and three education entries. The API\nexpects those counts as strings. `acceptCachedProfiles: false` asks for a fresh\nretrieval, while the picture and internal-ID options stay off because this\nworkflow uses durable text and URL fields.\n\nUse a standard public `/in/` profile URL or a Sales Navigator lead URL for each\ninput. A person's name, a LinkedIn search page, and a company page identify\ndifferent tasks and are not valid profile inputs for this scraper. If your\nsource contains Sales Navigator links, the returned `profileUrl` can be the\nnormalized public profile URL. For a larger list, keep every `inputId` unique\nand stable instead of deriving it from the row position.\n\nThe history counts control how many flat, numbered work and education fields the\nAPI can return. They do not change the per-profile credit rate. Request enough\nhistory for your research decision, but avoid selecting ten work entries when\nthe latest few roles are all you plan to inspect.\n\n### What the three API calls do {#what-the-three-api-calls-do}\nThe `POST` request sends the selected settings and profile inputs. Datablist\naccepts the work as a background task and returns its `@id`. The script prints\nboth the generated `uniqueId` and the returned task ID, so you have the values\nneeded to inspect the task later.\n\nThe first `GET` call checks the task status every three seconds. The Python\nclient blocks while it waits, but the API task continues in the background. The\nloop stops at `completed`, `failed`, or `stopped`.\n\nThe second `GET` call retrieves results in pages of up to 100 records. When\n`nextCursor` contains a value, the script fetches the next page. It stops when\n`nextCursor` is `null`.\n\n## Read the profile result and choose the right retrieval settings {#read-the-profile-result-and-choose-the-right-retrieval-settings}\nThe submitted URL determines which profile the scraper retrieves. Datablist\naccepts public person-profile URLs and Sales Navigator lead URLs, normalizes the\nprofile URL when needed, and maps the available public data into flat profile,\nwork, education, and company fields.\n\nThe fresh lookup used in this guide completed successfully and used 25 credits.\nHere is the complete result:\n\n```json\n{\n \"inputId\": \"contact-satya-nadella\",\n \"status\": \"success\",\n \"outputs\": {\n \"profileUrl\": \"https://www.linkedin.com/in/satyanadella/\",\n \"fullName\": \"Satya Nadella\",\n \"firstName\": \"Satya\",\n \"lastName\": \"Nadella\",\n \"linkedInHeadline\": \"Chairman and CEO at Microsoft\",\n \"linkedInSummary\": \"As chairman and CEO of Microsoft, I define my mission and that of my company as empowering every person and every organization on the planet to achieve more.\",\n \"linkedInProfilePicture\": \"https://media.licdn.com/dms/image/v2/C5603AQHHUuOSlRVA1w/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1579726625483?e=1794441600&v=beta&t=I1sL---bTuvjxqb1vdbNsNy-fAsLCpBj3-WtJjOGbOc\",\n \"linkedInInternalIdentifier\": \"ACoAAAEkwwAB9KEc2TrQgOLEQ-vzRyZeCDyc6DQ\",\n \"linkedInLocation\": \"Redmond, Washington, United States\",\n \"linkedInCity\": \"Redmond\",\n \"linkedInState\": \"Washington\",\n \"linkedInCountryName\": \"United States\",\n \"linkedInCountryCode\": \"US\",\n \"linkedInLanguages\": \"\",\n \"linkedInConnections\": 832,\n \"linkedInlastExpTitle\": \"Chairman and CEO\",\n \"linkedInlastExpCompany\": \"Microsoft\",\n \"linkedInlastExpEmploymentType\": null,\n \"linkedInlastExpLocation\": \"Greater Seattle Area\",\n \"linkedInlastExpStartsAt\": \"2014-02-01\",\n \"linkedInlastExpEndsAt\": null,\n \"linkedInlastExpDescription\": null,\n \"linkedInlastExpCompanyWebsite\": \"https://news.microsoft.com/\",\n \"linkedInlastExpCompanyPage\": \"https://www.linkedin.com/company/microsoft/\",\n \"linkedInlastExpCompanyIndustry\": \"Software Development\",\n \"linkedInlastExpCompanyRange\": \"10001+\",\n \"linkedInlastExpCompanyDescription\": \"Every company has a mission. What's ours? To empower every person and every organization to achieve more. We believe technology can and should be a force for good and that meaningful innovation contributes to a brighter world in the future and today. Our culture doesn’t just encourage curiosity; it embraces it. Each day we make progress together by showing up as our authentic selves. We show up with a learn-it-all mentality. We show up cheering on others, knowing their success doesn't diminish our own. We show up every day open to learning our own biases, changing our behavior, and inviting in differences. Because impact matters. \\n\\nMicrosoft operates in 190 countries and is made up of approximately 228,000 passionate employees worldwide.\",\n \"linkedInlastExpCompanyLogo\": \"https://media.licdn.com/dms/image/v2/D560BAQH32RJQCl3dDQ/company-logo_200_200/B56ZYQ0mrGGoAM-/0/1744038948046/microsoft_logo?e=1793836800&v=beta&t=4wXD40LQU4IfLY4DiEvT2bwvjUoPQPGgi5_T-vPsbQ8\",\n \"linkedIn2ExpTitle\": \"Chairman\",\n \"linkedIn2ExpCompanyName\": \"The Business Council U.S.\",\n \"linkedIn2ExpEmploymentType\": null,\n \"linkedIn2ExpStartsAt\": \"2021-01-01\",\n \"linkedIn2ExpEndsAt\": \"2023-01-01\",\n \"linkedIn2ExpDescription\": null,\n \"linkedIn2ExpLogo\": \"https://media.licdn.com/dms/image/v2/C4D0BAQGcnaZNpjMvrw/company-logo_200_200/company-logo_200_200/0/1630557702704/the_business_council_us_logo?e=1793836800&v=beta&t=Ejb0D8lV5sMTFwcz_-FKdfQvIvq-mKx_Nl8vRMib53k\",\n \"linkedIn3ExpTitle\": \"Member Board Of Trustees\",\n \"linkedIn3ExpCompanyName\": \"University of Chicago\",\n \"linkedIn3ExpEmploymentType\": null,\n \"linkedIn3ExpStartsAt\": \"2018-01-01\",\n \"linkedIn3ExpEndsAt\": null,\n \"linkedIn3ExpDescription\": null,\n \"linkedIn3ExpLogo\": \"https://media.licdn.com/dms/image/v2/D4E0BAQGDP-PXY3u8dw/company-logo_200_200/B4EaENqZTQGoAM-/0/1791230580227/uchicago_logo?e=1793836800&v=beta&t=NJGoETr0tY868Z2pvlyXeh939iXwW0WbSFwRctabEVU\",\n \"linkedIn4ExpTitle\": \"Board Member\",\n \"linkedIn4ExpCompanyName\": \"Starbucks\",\n \"linkedIn4ExpEmploymentType\": null,\n \"linkedIn4ExpStartsAt\": \"2017-01-01\",\n \"linkedIn4ExpEndsAt\": \"2024-01-01\",\n \"linkedIn4ExpDescription\": null,\n \"linkedIn4ExpLogo\": \"https://media.licdn.com/dms/image/v2/C4D0BAQEQxk9y2rk7Hw/company-logo_200_200/company-logo_200_200/0/1631316692275?e=1793836800&v=beta&t=NVjljhe0oI5KzvJHodAJjss5EdvxaOIzQQ2rAGyh_AQ\",\n \"linkedIn5ExpTitle\": \"Board Member\",\n \"linkedIn5ExpCompanyName\": \"Fred Hutch\",\n \"linkedIn5ExpEmploymentType\": null,\n \"linkedIn5ExpStartsAt\": \"2016-01-01\",\n \"linkedIn5ExpEndsAt\": \"2022-01-01\",\n \"linkedIn5ExpDescription\": null,\n \"linkedIn5ExpLogo\": \"https://media.licdn.com/dms/image/v2/D4D0BAQGF4eOou62nvQ/company-logo_200_200/company-logo_200_200/0/1664795263764/fredhutch_logo?e=1793836800&v=beta&t=dJWSO-KkuBPd6I6aRYBoIcQVzcSEmGLwqYy-VMFXJQc\",\n \"linkedInlastEduSchool\": \"The University of Chicago Booth School of Business\",\n \"linkedInlastEduDegree\": null,\n \"linkedInlastEduField\": null,\n \"linkedInlastEduStartsAt\": \"1994-01-01\",\n \"linkedInlastEduEndsAt\": \"1996-01-01\",\n \"linkedInlastEduPage\": \"https://www.linkedin.com/school/universityofchicagoboothschoolofbusiness/\",\n \"linkedInlastEduLogo\": \"https://media.licdn.com/dms/image/v2/D560BAQFZO05jhYKSkw/company-logo_200_200/company-logo_200_200/0/1692627816910/universityofchicagoboothschoolofbusiness_logo?e=2147483647&v=beta&t=BTjFA4UVU5iKIIyicg8d44rhoiuMbetNDB97BmqpwhQ\",\n \"linkedIn2EduSchool\": \"Manipal Institute of Technology\",\n \"linkedIn2EduDegree\": \"Bachelor’s Degree\",\n \"linkedIn2EduField\": \"Electrical Engineering\",\n \"linkedIn2EduStartsAt\": null,\n \"linkedIn2EduEndsAt\": null,\n \"linkedIn2EduPage\": \"https://www.linkedin.com/school/manipal-institute-of-technology/\",\n \"linkedIn2EduLogo\": \"https://media.licdn.com/dms/image/v2/D4D0BAQHodG_7cUqs6w/company-logo_200_200/B4DZ8hC8AUJQAE-/0/1782965846314/manipal_institute_of_technology_logo?e=2147483647&v=beta&t=jqJK15_r7lxUNgWdbbzdMq47aLa_PqnkDYxg4Jun4Zo\",\n \"linkedIn3EduSchool\": \"University of Wisconsin-Milwaukee\",\n \"linkedIn3EduDegree\": \"Master’s Degree\",\n \"linkedIn3EduField\": \"Computer Science\",\n \"linkedIn3EduStartsAt\": null,\n \"linkedIn3EduEndsAt\": null,\n \"linkedIn3EduPage\": \"https://www.linkedin.com/school/uwmilwaukee/\",\n \"linkedIn3EduLogo\": \"https://media.licdn.com/dms/image/v2/D4D0BAQE47QlO_LG8SQ/company-logo_200_200/company-logo_200_200/0/1729615751602/uwmilwaukee_logo?e=2147483647&v=beta&t=l-w3ttd1mZRc-_j5aCPXEzNiYJzl1iJjMGw7q8B3eRI\"\n },\n \"creditsUsed\": \"25\",\n \"error\": null\n}\n```\n\nRead the title, employer, start date, and end date together. The Microsoft entry\nstarts in February 2014 and has no end date. The Business Council U.S. entry\nruns from 2021 to January 2023. Those dates distinguish an open entry from an\nended one and also show how roles can overlap.\n\n`linkedInlastExpTitle` and `linkedInlastExpCompany` contain the current or last\nmapped experience. That first mapped entry can include richer company details,\nsuch as the company page, website, industry, description, and employee range.\nNumbered entries add earlier titles, employers, and dates. In this response, the\nreturned Microsoft website is `https://news.microsoft.com/`. That URL gives the\nlead record a useful company source alongside the person's role history.\n\nThe `linkedInlastEduSchool`, `linkedIn2EduSchool`, and `linkedIn3EduSchool`\nfields name The University of Chicago Booth School of Business, Manipal\nInstitute of Technology, and the University of Wisconsin-Milwaukee. The Manipal\nentry includes a Bachelor's Degree in Electrical Engineering, and the\nWisconsin-Milwaukee entry includes a Master's Degree in Computer Science. The\nChicago degree and field values are empty.\n\nUse `experiencesNb` values of `\"3\"`, `\"5\"`, or `\"10\"` and `educationsNb`\nvalues of `\"1\"`, `\"3\"`, or `\"5\"` to choose how much history you need. Read\nthose fields directly from `outputs` when they support your research. An empty\noptional value does not make the whole result a failure.\n\nFresh mode fits lead-list preparation when you want a new retrieval of the\navailable public profile data. Cached mode can return an available stored\nprofile for fewer credits and fall back to a fresh retrieval when no cached\nprofile is available. Choose based on your desired recency and credit budget.\n\n![A known LinkedIn profile URL becomes a structured contact research record through the LinkedIn Profile Scraper API](/howto_images/scrape-linkedin-profiles-python-api/illustrations/linkedin-profile-contact-refresh.png)\n\n## Prepare lead records for outreach {#prepare-lead-records-for-outreach}\nOnce `profile_results` is available, you can attach each response to its source\ncontact without submitting another task. The following block reuses the\nvariables created by the complete script:\n\n```python\nresults_by_input_id = {\n result[\"inputId\"]: result\n for result in profile_results\n}\n\ncontact_research = []\nfor contact in source_contacts:\n result = results_by_input_id[contact[\"contact_id\"]]\n\n contact_research.append(\n {\n \"contact_id\": contact[\"contact_id\"],\n \"source_profile_url\": contact[\"profile_url\"],\n \"result_status\": result[\"status\"],\n \"profile\": result.get(\"outputs\", {}),\n \"credits_used\": result[\"creditsUsed\"],\n \"error\": result.get(\"error\"),\n }\n )\n\nresearch_record = contact_research[0]\nprofile = research_record[\"profile\"]\n\nprint(\"Contact:\", profile.get(\"fullName\"))\nprint(\"Headline:\", profile.get(\"linkedInHeadline\"))\nprint(\"Location:\", profile.get(\"linkedInLocation\"))\nprint(\n \"First work entry:\",\n profile.get(\"linkedInlastExpTitle\"),\n \"at\",\n profile.get(\"linkedInlastExpCompany\"),\n)\n```\n\nThe dictionary keyed by `inputId` handles out-of-order responses.\n`contact-satya-nadella` remains both the source ID and the API ID, the known\nLinkedIn URL remains `source_profile_url`, and `profile` keeps the original\n`outputs` object unchanged.\n\nThis wrapper keeps the complete returned profile together with the source URL,\nAPI status, and credit usage. You can use the printed headline, location, and\nfirst work entry to qualify the lead and add relevant role or company context to\noutreach. Unsuccessful API rows keep their returned status and error, so they\nremain visible in the lead list.\n\nYou can download the maintained [blocking Requests\nexample](https://github.com/datablist/linkedin-profile-scraper-api/blob/main/python/sync_requests.py)\nor use the [async HTTPX\nalternative](https://github.com/datablist/linkedin-profile-scraper-api/blob/main/python/async_httpx.py)\nafter the blocking flow is clear.\n\n## Estimate the LinkedIn profile scraping cost {#estimate-the-linkedin-profile-scraping-cost}\nA fresh retrieval uses 25 credits per profile. When cached profiles are enabled,\nan available cached result uses 20 credits, while a cache miss can still fall\nback to a 25-credit fresh retrieval. The fresh Satya Nadella sample used 25\ncredits.\n\nFor 1,000 profiles, the retrieval therefore uses 20,000 credits if every\nrequested cached profile is available, or up to 25,000 credits for fresh\nretrieval. Datablist REST API access requires the Growth plan. The subscription,\nits included credits, and optional credit top-ups are separate parts of the\ncost. Check the [pricing page](/pricing) for the live catalog.\n\nUsing the current USD base top-up of $20 for 20,000 credits as a planning\nexample, the consumed credits are worth an estimated $20 for an all-cached\n1,000-profile run or $25 for a fresh run. These estimates are separate from the\nGrowth subscription cost.\n\n
\n

Each available cached profile uses 20 credits; fresh retrieval uses 25 credits.

\n
\n\n## Fix common errors and recover a task {#fix-common-errors-and-recover-a-task}\n- **`DATABLIST_API_KEY` is missing:** the script stops before making a request.\n Export the variable, then rerun the file.\n- **The API rejects the key:** check the Personal API Key and send only the\n `X-API-Key` credential used in this example.\n- **The profile URL is missing or malformed:** use a public LinkedIn person URL\n or a supported Sales Navigator lead URL. A person's name, search page, or\n company page is not a profile input.\n- **Credits are insufficient:** check your balance, then use included credits or\n add a top-up before creating another task.\n- **The task is `failed` or `stopped`:** inspect its error, available results,\n and credit usage before deciding whether another paid submission is needed.\n- **A result is sparse, `no_result`, or `error`:** inspect its status and public\n error, then confirm the URL and settings. Available fields can vary with the\n profile, its visibility, and retrieval.\n\nIf the connection drops during the `POST`, use the `uniqueId` printed before\nsubmission. List recent scraper tasks and compare their values before creating\nanother task:\n\n```python\nresponse = requests.get(\n \"https://api.datablist.com/v1/enrichments/tasks\",\n headers={\"X-API-Key\": api_key},\n params={\"enrichmentId\": \"linkedin-profile-scraper\", \"limit\": 100},\n)\nresponse.raise_for_status()\n\nprinted_unique_id = \"paste-the-printed-unique-id\"\ntasks = response.json()[\"tasks\"]\nmatching_task = None\n\nfor task in tasks:\n if task[\"uniqueId\"] == printed_unique_id:\n matching_task = task\n break\n\nif matching_task:\n print(\"Task ID:\", matching_task[\"@id\"])\n```\n\nThe task list can be filtered by `enrichmentId`, but not by `uniqueId`. Follow\nits `nextCursor` if the task is not on the first page. Once you have the task\nID, continue with the same status and results URLs used by the main function.\nThe [task API documentation](https://platform.datablist.com/docs/enrichments/tasks)\nexplains the lifecycle, pagination, and interruption behavior in more detail.\n\n## Next steps {#next-steps}\nYou now have one reusable Python Requests function that turns known LinkedIn\nprofile URLs into structured results, plus lead records matched by stable ID.\nReplace the example with a small reviewed set of URLs, choose fresh or cached\nretrieval based on the recency and credit cost you need, and use the returned\nrole and company context to qualify leads or personalize outreach.\n\nIf you still need to discover profile URLs from names and a company or school\nkeyword, follow the separate guide to [find LinkedIn profiles in Python with an\nAPI](/how-to/find-linkedin-profiles-python-api) before running this scraper.\n" }