[ { "path": ".github/FUNDING.yml", "content": "# These are supported funding model platforms\n\ngithub: [ospfranco]\n" }, { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n**Describe the bug**\nA clear and concise description of what the bug is.\n\n**To Reproduce**\nSteps to reproduce the behavior:\n1. Go to '...'\n2. Click on '....'\n3. Scroll down to '....'\n4. See error\n\n**Expected behavior**\nA clear and concise description of what you expected to happen.\n\n**Screenshots**\nIf applicable, add screenshots to help explain your problem.\n\n**Desktop (please complete the following information):**\n - OS: [e.g. iOS]\n - Browser [e.g. chrome, safari]\n - Version [e.g. 22]\n\n**Smartphone (please complete the following information):**\n - Device: [e.g. iPhone6]\n - OS: [e.g. iOS8.1]\n - Browser [e.g. stock browser, safari]\n - Version [e.g. 22]\n\n**Additional context**\nAdd any other context about the problem here.\n" }, { "path": ".github/ISSUE_TEMPLATE/feature_request.md", "content": "---\nname: Feature request\nabout: Suggest an idea for this project\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n**Is your feature request related to a problem? Please describe.**\nA clear and concise description of what the problem is. Ex. I'm always frustrated when [...]\n\n**Describe the solution you'd like**\nA clear and concise description of what you want to happen.\n\n**Describe alternatives you've considered**\nA clear and concise description of any alternative solutions or features you've considered.\n\n**Additional context**\nAdd any other context or screenshots about the feature request here.\n" }, { "path": ".github/workflows/publish.yml", "content": "name: \"publish\"\n\non:\n push:\n tags:\n - \"*\"\n\njobs:\n publish:\n permissions:\n contents: write\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-node@v4\n with:\n node-version: \"22\"\n\n - name: Cache dependencies\n id: yarn-cache\n uses: actions/cache@v3\n with:\n path: |\n **/node_modules\n .yarn/install-state.gz\n key: ${{ runner.os }}-yarn-${{ hashFiles('yarn.lock') }}-${{ hashFiles('**/package.json', '!node_modules/**') }}\n restore-keys: |\n ${{ runner.os }}-yarn-${{ hashFiles('yarn.lock') }}\n ${{ runner.os }}-yarn-\n - run: yarn install\n - name: Update package.json version\n run: |\n TAG=${GITHUB_REF#refs/tags/}\n jq --arg version \"$TAG\" '.version = $version' package.json > tmp.$$.json && mv tmp.$$.json package.json\n - name: Compile typescript\n run: yarn build\n - uses: JS-DevTools/npm-publish@v3\n with:\n token: ${{ secrets.NPM_TOKEN }}\n\n - name: Create a Release\n uses: elgohr/Github-Release-Action@v5\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n with:\n title: Release ${{ github.ref_name }}\n tag: ${{ github.ref_name }}\n" }, { "path": ".github/workflows/tests.yml", "content": "name: Tests\non: pull_request\n\njobs:\n tests:\n runs-on: ubuntu-latest\n\n steps:\n - name: Checkout\n uses: actions/checkout@v4\n\n - name: Cache dependencies\n id: yarn-cache\n uses: actions/cache@v3\n with:\n path: |\n **/node_modules\n .yarn/install-state.gz\n key: ${{ runner.os }}-yarn-${{ hashFiles('yarn.lock') }}-${{ hashFiles('**/package.json', '!node_modules/**') }}\n restore-keys: |\n ${{ runner.os }}-yarn-${{ hashFiles('yarn.lock') }}\n ${{ runner.os }}-yarn-\n\n - name: Install dependencies\n run: yarn\n\n - name: Run tests\n run: yarn test\n" }, { "path": ".gitignore", "content": "node_modules/\n.vscode/\nbuild/\n.DS_Store" }, { "path": ".npmignore", "content": ".github\n__tests__\n.vscode\n.eslintrc.js\nindex.ts\njest.config.js\nLICENSE\nREADME.md\ntsconfig.json\ntypings.d.ts" }, { "path": ".prettierignore", "content": "build" }, { "path": ".prettierrc.json", "content": "{}\n" }, { "path": "CONTRIBUTING.md", "content": "If you add new functionality please make sure you add the appropiate tests and the tests are running.\n\nPlease also try to keep the style consistant, linter use is a must.\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2019 Oscar Franco\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "

Link Preview JS

\n\n
\n\n\n
\nnpm i link-preview-js\n
\n\n\n \n \n
\n \n \n \n
\n\n**Before creating an issue**\n\nIt's more than likely there is nothing wrong with the library:\n\n- It's very simple; fetch HTML, parse HTML, and search for OpenGraph HTML tags.\n- Unless HTML or the OpenGraph standard change, the library will not break\n- If the target website you are trying to preview redirects you to a login page **the preview will fail**, because it will parse the login page\n- If the target website does not have OpenGraph tags **the preview will most likely fail**, there are some fallbacks but in general, it will not work\n- **You cannot preview (fetch) another web page from YOUR web page. This is an intentional security feature of browsers called CORS**\n\n

DO NOT FETCH CONTENT DIRECTLY FROM A USERS DEVICE. ONLY RUN THIS IN YOUR SERVER AND SANDBOX IT IF YOU CAN

\nBrowsers block this via cors, but you might be clever like a fox and run this in React Native. This is a bad idea, you are exposing the device user to potentially malicious links\n\nIf you use this library and find it useful please consider [sponsoring me](https://github.com/sponsors/ospfranco), open source takes a lot of time and effort.\n\n# Link Preview\n\nAllows you to extract information from an HTTP URL/link (or parse an HTML string) and retrieve meta information such as title, description, images, videos, etc. via **OpenGraph** tags.\n\n## Discord\n\nJoin the Discord\n\nhttps://discord.gg/W9XmqCQCKP\n\n## GOTCHAs\n\n- You cannot request a different domain from your web app (Browsers block cross-origin-requests). If you don't know how _same-origin-policy_ works, [here is a good intro](https://dev.to/lydiahallie/cs-visualized-cors-5b8h), therefore **this library works on Node.js and certain mobile run-times (Cordova or React-Native)**.\n- **This library acts as if the user would visit the page, sites might re-direct you to sign-up pages, consent screens, etc.** You can try to change the user-agent header (try with `google-bot` or with `Twitterbot`), but you need to work around these issues yourself.\n\n## API\n\n`getLinkPreview`: you have to pass a string, doesn't matter if it is just a URL or a piece of text that contains a URL, the library will take care of parsing it and returning the info o the first valid HTTP(S) URL info it finds.\n\n`getPreviewFromContent`: useful for passing a pre-fetched Response object from an existing async/etc. call. Refer to the example below for required object values.\n\n```typescript\nimport { getLinkPreview, getPreviewFromContent } from \"link-preview-js\";\n\n// pass the link directly\ngetLinkPreview(\"https://www.youtube.com/watch?v=MejbOFk7H6c\").then((data) =>\n console.debug(data)\n);\n\n////////////////////////// OR //////////////////////////\n\n// pass a chunk of text\ngetLinkPreview(\n \"This is a text supposed to be parsed and the first link displayed https://www.youtube.com/watch?v=MejbOFk7H6c\"\n).then((data) => console.debug(data));\n\n////////////////////////// OR //////////////////////////\n\n// pass a pre-fetched response object\n// The passed response object should include, at minimum:\n// {\n// data: '...', // response content\n// headers: {\n// ...\n// // should include content-type\n// content-type: \"text/html; charset=ISO-8859-1\",\n// ...\n// },\n// url: 'https://domain.com/' // resolved url\n// }\nyourAjaxCall(url, (response) => {\n getPreviewFromContent(response).then((data) => console.debug(data));\n});\n```\n\n## Options\n\nAdditionally, you can pass an options object which should add more functionality to the parsing of the link\n\n| Property Name | Result |\n| -------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |\n| imagesPropertyType (**optional**) (ex: 'og') | Fetches images only with the specified property, `meta[property='${imagesPropertyType}:image']` |\n| headers (**optional**) (ex: { 'user-agent': 'googlebot', 'Accept-Language': 'en-US' }) | Add request headers to fetch call |\n| timeout (**optional**) (ex: 1000) | Timeout for the request to fail |\n| followRedirects (**optional**) (default 'error') | For security reasons, the library does not automatically follow redirects ('error' value), a malicious agent can exploit redirects to steal data, posible values: ('error', 'follow', 'manual') |\n| handleRedirects (**optional**) (with followRedirects 'manual') | When followRedirects is set to 'manual' you need to pass a function that validates if the redirectinon is secure, below you can find an example |\n| resolveDNSHost (**optional**) | Function that resolves the final address of the detected/parsed URL to prevent SSRF attacks |\n| onResponse (**optional**) | Function that handles the response object to allow for managing special cases |\n\n```javascript\ngetLinkPreview(\"https://www.youtube.com/watch?v=MejbOFk7H6c\", {\n imagesPropertyType: \"og\", // fetches only open-graph images\n headers: {\n \"user-agent\": \"googlebot\", // fetches with googlebot crawler user agent\n \"Accept-Language\": \"fr-CA\", // fetches site for French language\n // ...other optional HTTP request headers\n },\n timeout: 1000,\n}).then((data) => console.debug(data));\n```\n\n## SSRF Concerns\n\nDoing requests on behalf of your users or using user-provided URLs is dangerous. One of such attack is trying to fetch a domain that redirects to localhost so the users get the contents of your server (doesn't affect mobile runtimes). To mitigate this attack you can use the resolveDNSHost option:\n\n```ts\n// example how to use node's dns resolver\nconst dns = require(\"node:dns\");\ngetLinkPreview(\"http://maliciousLocalHostRedirection.com\", {\n resolveDNSHost: async (url: string) => {\n return new Promise((resolve, reject) => {\n const hostname = new URL(url).hostname;\n dns.lookup(hostname, (err, address, family) => {\n if (err) {\n reject(err);\n return;\n }\n\n resolve(address); // if address resolves to localhost or '127.0.0.1' library will throw an error\n });\n });\n },\n}).catch((e) => {\n // will throw a detected redirection to localhost\n});\n```\n\nThis might add some latency to your request but prevents loopback attacks.\n\n## Redirections\n\nSame to SSRF, following redirections is dangerous, the library errors by default when the response tries to redirect the user. There are however some simple redirections that are valid (e.g. HTTP to HTTPS) and you might want to allow them, you can do it via:\n\n```ts\nawait getLinkPreview(`http://google.com/`, {\n followRedirects: `manual`,\n handleRedirects: (baseURL: string, forwardedURL: string) => {\n const urlObj = new URL(baseURL);\n const forwardedURLObj = new URL(forwardedURL);\n if (\n forwardedURLObj.hostname === urlObj.hostname ||\n forwardedURLObj.hostname === \"www.\" + urlObj.hostname ||\n \"www.\" + forwardedURLObj.hostname === urlObj.hostname\n ) {\n return true;\n } else {\n return false;\n }\n },\n});\n```\n\n## onResponse\n\nIn some cases the website might be missing OpenGraph tags and you might want to provide your own custom logic to try to parse data. For example, if the library is unable to detect a description, you might want to use the text value of the first paragraph instead. This callback gives you access to the Cheerio (the library internally used to parse the HTML) instance, as well as the URL object so you could handle cases on a site-by-site basis, if you need to. This callback must return the modified response object\n\n```javascript\nawait getLinkPreview(`https://example.com/`, {\n onResponse: (response, doc, URL) => {\n if (URL.hostname == 'example.com') {\n response.siteName = 'Example Website';\n }\n\n if (!response.description) {\n response.description = doc('p').first().text();\n }\n \n return response;\n },\n});\n```\n\n## Response\n\nReturns a Promise that resolves with an object describing the provided link.\nThe info object returned varies depending on the content type (MIME type) returned\nin the HTTP response (see below for variations of response). Rejects with an error if the response can not be parsed or if there was no URL in the text provided.\n\n### Text/HTML URL\n\n```javascript\n{\n url: \"https://www.youtube.com/watch?v=MejbOFk7H6c\",\n title: \"OK Go - Needing/Getting - Official Video - YouTube\",\n siteName: \"YouTube\",\n description: \"Buy the video on iTunes: https://itunes.apple.com/us/album/needing-getting-bundle-ep/id508124847 See more about the guitars at: http://www.gretschguitars.com...\",\n images: [\"https://i.ytimg.com/vi/MejbOFk7H6c/maxresdefault.jpg\"],\n mediaType: \"video.other\",\n contentType: \"text/html\",\n charset: \"utf-8\"\n videos: [],\n favicons:[\"https://www.youtube.com/yts/img/favicon_32-vflOogEID.png\",\"https://www.youtube.com/yts/img/favicon_48-vflVjB_Qk.png\",\"https://www.youtube.com/yts/img/favicon_96-vflW9Ec0w.png\",\"https://www.youtube.com/yts/img/favicon_144-vfliLAfaB.png\",\"https://s.ytimg.com/yts/img/favicon-vfl8qSV2F.ico\"]\n}\n```\n\n### Image URL\n\n```javascript\n{\n url: \"https://media.npr.org/assets/img/2018/04/27/gettyimages-656523922nunes-4bb9a194ab2986834622983bb2f8fe57728a9e5f-s1100-c15.jpg\",\n mediaType: \"image\",\n contentType: \"image/jpeg\",\n favicons: [ \"https://media.npr.org/favicon.ico\" ]\n}\n```\n\n### Audio URL\n\n```javascript\n{\n url: \"https://ondemand.npr.org/anon.npr-mp3/npr/atc/2007/12/20071231_atc_13.mp3\",\n mediaType: \"audio\",\n contentType: \"audio/mpeg\",\n favicons: [ \"https://ondemand.npr.org/favicon.ico\" ]\n}\n```\n\n### Video URL\n\n```javascript\n{\n url: \"https://www.w3schools.com/html/mov_bbb.mp4\",\n mediaType: \"video\",\n contentType: \"video/mp4\",\n favicons: [ \"https://www.w3schools.com/favicon.ico\" ]\n}\n```\n\n### Application URL\n\n```javascript\n{\n url: \"https://assets.curtmfg.com/masterlibrary/56282/installsheet/CME_56282_INS.pdf\",\n mediaType: \"application\",\n contentType: \"application/pdf\",\n favicons: [ \"https://assets.curtmfg.com/favicon.ico\" ]\n}\n```\n\n## License\n\nMIT license\n" }, { "path": "__tests__/__snapshots__/index.spec.ts.snap", "content": "// Jest Snapshot v1, https://goo.gl/fbAQLP\n\nexports[`#getLinkPreview() no link in text should fail gracefully 1`] = `\"link-preview-js did not receive a valid a url or text\"`;\n\nexports[`#getLinkPreview() should handle empty strings gracefully 1`] = `\"link-preview-js did not receive a valid url or text\"`;\n\nexports[`#getLinkPreview() should handle malformed urls gracefully 1`] = `\"link-preview-js did not receive a valid a url or text\"`;\n" }, { "path": "__tests__/index.spec.ts", "content": "import { getLinkPreview, getPreviewFromContent } from \"../index\";\nimport prefetchedResponse from \"./sampleResponse.json\";\n\ndescribe(`#getLinkPreview()`, () => {\n it(`should extract link info from just URL`, async () => {\n const linkInfo: any = await getLinkPreview(\n `https://www.youtube.com/watch?v=wuClZjOdT30`,\n { headers: { \"Accept-Language\": `en-US` } }\n );\n\n expect(linkInfo.url).toEqual(`https://www.youtube.com/watch?v=wuClZjOdT30`);\n expect(linkInfo.siteName).toEqual(`YouTube`);\n expect(linkInfo.title).toEqual(`Geography Now! Germany`);\n expect(linkInfo.description).toBeTruthy();\n expect(linkInfo.mediaType).toEqual(`video.other`);\n expect(linkInfo.images.length).toEqual(1);\n expect(linkInfo.images[0]).toEqual(\n `https://i.ytimg.com/vi/wuClZjOdT30/maxresdefault.jpg`\n );\n expect(linkInfo.videos.length).toEqual(0);\n expect(linkInfo.favicons[0]).not.toBe(``);\n expect(linkInfo.contentType.toLowerCase()).toEqual(`text/html`);\n expect(linkInfo.charset?.toLowerCase()).toEqual(`utf-8`);\n });\n\n it(\"returns charset of website\", async () => {\n const linkInfo: any = await getLinkPreview(`https://www.pravda.com.ua`);\n\n expect(linkInfo.url).toEqual(`https://www.pravda.com.ua/`);\n expect(linkInfo.contentType.toLowerCase()).toEqual(`text/html`);\n expect(linkInfo.charset?.toLowerCase()).toEqual(`utf-8`);\n });\n\n xit(\"should extract author from news article\", async () => {\n const linkInfo: any = await getLinkPreview(\n `https://www.usatoday.com/story/special/contributor-content/2025/10/15/why-chaos-engineering-is-more-important-than-ever-in-the-ai-era/86712877007/`\n );\n\n expect(linkInfo.author).toEqual(`Matt Emma`);\n });\n\n it(`should extract link info from a URL with a newline`, async () => {\n const linkInfo: any = await getLinkPreview(\n `\n https://www.youtube.com/watch?v=wuClZjOdT30\n `,\n { headers: { \"Accept-Language\": `en-US` } }\n );\n\n expect(linkInfo.url).toEqual(`https://www.youtube.com/watch?v=wuClZjOdT30`);\n expect(linkInfo.title).toEqual(`Geography Now! Germany`);\n expect(linkInfo.siteName).toBeTruthy();\n expect(linkInfo.description).toBeTruthy();\n expect(linkInfo.mediaType).toEqual(`video.other`);\n expect(linkInfo.images.length).toEqual(1);\n expect(linkInfo.images[0]).toEqual(\n `https://i.ytimg.com/vi/wuClZjOdT30/maxresdefault.jpg`\n );\n expect(linkInfo.videos.length).toEqual(0);\n expect(linkInfo.favicons[0]).not.toBe(``);\n expect(linkInfo.contentType.toLowerCase()).toEqual(`text/html`);\n });\n\n it(`should extract link info from just text with a URL`, async () => {\n const linkInfo: any = await getLinkPreview(\n `This is some text blah blah https://www.youtube.com/watch?v=wuClZjOdT30 and more text`,\n { headers: { \"Accept-Language\": `en-US` } }\n );\n\n expect(linkInfo.url).toEqual(`https://www.youtube.com/watch?v=wuClZjOdT30`);\n expect(linkInfo.title).toEqual(`Geography Now! Germany`);\n expect(linkInfo.siteName).toEqual(`YouTube`);\n expect(linkInfo.description).toBeTruthy();\n expect(linkInfo.mediaType).toEqual(`video.other`);\n expect(linkInfo.images.length).toEqual(1);\n expect(linkInfo.images[0]).toEqual(\n `https://i.ytimg.com/vi/wuClZjOdT30/maxresdefault.jpg`\n );\n expect(linkInfo.videos.length).toEqual(0);\n expect(linkInfo.favicons[0]).toBeTruthy();\n expect(linkInfo.contentType.toLowerCase()).toEqual(`text/html`);\n });\n\n // it(`should make request with different languages`, async () => {\n // let linkInfo: any = await getLinkPreview(`https://www.wikipedia.org/`, {\n // headers: { \"Accept-Language\": `es` },\n // followRedirects: `follow`,\n // });\n // expect(linkInfo.title).toContain(`Wikipedia, la enciclopedia libre`);\n\n // linkInfo = await getLinkPreview(`https://www.wikipedia.org/`);\n // expect(linkInfo.title).toContain(`Wikipedia`);\n // });\n\n it(`should handle audio urls`, async () => {\n const linkInfo = await getLinkPreview(\n `https://ondemand.npr.org/anon.npr-mp3/npr/atc/2007/12/20071231_atc_13.mp3`\n );\n expect(linkInfo.url).toEqual(\n `https://ondemand.npr.org/anon.npr-mp3/npr/atc/2007/12/20071231_atc_13.mp3`\n );\n expect(linkInfo.mediaType).toEqual(`audio`);\n expect(linkInfo.contentType?.toLowerCase()).toEqual(`audio/mpeg`);\n expect(linkInfo.favicons[0]).toBeTruthy();\n });\n\n it(`should handle video urls`, async () => {\n const linkInfo = await getLinkPreview(\n `https://www.w3schools.com/html/mov_bbb.mp4`\n );\n\n expect(linkInfo.url).toEqual(`https://www.w3schools.com/html/mov_bbb.mp4`);\n expect(linkInfo.mediaType).toEqual(`video`);\n expect(linkInfo.contentType?.toLowerCase()).toEqual(`video/mp4`);\n expect(linkInfo.favicons[0]).toBeTruthy();\n });\n\n it(`should handle image urls`, async () => {\n const linkInfo = await getLinkPreview(\n `https://media.npr.org/assets/img/2018/04/27/gettyimages-656523922nunes-4bb9a194ab2986834622983bb2f8fe57728a9e5f-s1100-c15.jpg`\n );\n\n expect(linkInfo.url).toEqual(\n `https://media.npr.org/assets/img/2018/04/27/gettyimages-656523922nunes-4bb9a194ab2986834622983bb2f8fe57728a9e5f-s1100-c15.jpg`\n );\n expect(linkInfo.mediaType).toEqual(`image`);\n expect(linkInfo.contentType?.toLowerCase()).toEqual(`image/jpeg`);\n expect(linkInfo.favicons[0]).toBeTruthy();\n });\n\n it(`should handle unknown content type urls`, async () => {\n const linkInfo = await getLinkPreview(`https://mjml.io/try-it-live`);\n\n expect(linkInfo.url).toEqual(`https://mjml.io/try-it-live`);\n expect(linkInfo.mediaType).toEqual(`website`);\n });\n\n // This site changed? it is not returning application any more but rather website\n // it.skip(`should handle application urls`, async () => {\n // const linkInfo = await getLinkPreview(\n // `https://assets.curtmfg.com/masterlibrary/56282/installsheet/CME_56282_INS.pdf`\n // );\n\n // expect(linkInfo.url).toEqual(\n // `https://assets.curtmfg.com/masterlibrary/56282/installsheet/CME_56282_INS.pdf`\n // );\n // expect(linkInfo.mediaType).toEqual(`application`);\n // expect(linkInfo.contentType?.toLowerCase()).toEqual(`application/pdf`);\n // expect(linkInfo.favicons[0]).toBeTruthy();\n // });\n\n it(`no link in text should fail gracefully`, async () => {\n await expect(\n getLinkPreview(`no link`)\n ).rejects.toThrowErrorMatchingSnapshot();\n });\n\n it(`should handle malformed urls gracefully`, async () => {\n await expect(\n getLinkPreview(\n `this is a malformed link: ahttps://www.youtube.com/watch?v=wuClZjOdT30`\n )\n ).rejects.toThrowErrorMatchingSnapshot();\n });\n\n it(`should handle empty strings gracefully`, async () => {\n await expect(getLinkPreview(``)).rejects.toThrowErrorMatchingSnapshot();\n });\n\n it.skip(`should handle a proxy url option`, async () => {\n // origin header is required by cors-anywhere\n const linkInfo: any = await getLinkPreview(\n `https://www.youtube.com/watch?v=wuClZjOdT30`,\n {\n proxyUrl: `https://cors-anywhere.herokuapp.com/`,\n headers: {\n Origin: `http://localhost:8000`,\n \"Accept-Language\": `en-US`,\n },\n }\n );\n\n expect(linkInfo.url).toEqual(`https://www.youtube.com/watch?v=wuClZjOdT30`);\n expect(linkInfo.siteName).toEqual(`YouTube`);\n expect(linkInfo.title).toEqual(`Geography Now! Germany`);\n expect(linkInfo.description).toBeTruthy();\n expect(linkInfo.mediaType).toEqual(`video.other`);\n expect(linkInfo.images.length).toEqual(1);\n expect(linkInfo.images[0]).toEqual(\n `https://i.ytimg.com/vi/wuClZjOdT30/maxresdefault.jpg`\n );\n expect(linkInfo.videos.length).toEqual(0);\n expect(linkInfo.favicons[0]).not.toBe(``);\n expect(linkInfo.contentType.toLowerCase()).toEqual(`text/html`);\n });\n\n it(\"should timeout (default 3s) with infinite loading link\", async () => {\n try {\n await getLinkPreview(\n `https://www.gamestop.com/video-games/pc-gaming/components/cooling/products/hyper-212-rgb-black-edition-fan/185243.html?gclid=Cj0KCQjwraqHBhDsARIsAKuGZeECDlqkF2cxpcuS0xRxQmrv5BxFawWS_B51kiqehPf64_KlO0oyunsaAhn5EALw_wcB&gclsrc=aw.ds`\n );\n } catch (e: any) {\n expect(e.message).toEqual(\"Request timeout\");\n }\n });\n\n it(\"should timeout (custom 1s) with infinite loading link\", async () => {\n try {\n await getLinkPreview(\n `https://www.gamestop.com/video-games/pc-gaming/components/cooling/products/hyper-212-rgb-black-edition-fan/185243.html?gclid=Cj0KCQjwraqHBhDsARIsAKuGZeECDlqkF2cxpcuS0xRxQmrv5BxFawWS_B51kiqehPf64_KlO0oyunsaAhn5EALw_wcB&gclsrc=aw.ds`,\n {\n timeout: 1000,\n }\n );\n } catch (e: any) {\n expect(e.message).toEqual(\"Request timeout\");\n }\n });\n\n it(`should handle followRedirects option is error`, async () => {\n try {\n await getLinkPreview(`http://google.com/`, { followRedirects: `error` });\n } catch (e: any) {\n expect(e.message).toEqual(`fetch failed`);\n }\n });\n\n it(`should handle followRedirects option is manual but handleRedirects was not provided`, async () => {\n try {\n await getLinkPreview(`http://google.com/`, { followRedirects: `manual` });\n } catch (e: any) {\n expect(e.message).toEqual(\n `link-preview-js followRedirects is set to manual, but no handleRedirects function was provided`\n );\n }\n });\n\n it(`should handle followRedirects option is manual with handleRedirects function`, async () => {\n const response = await getLinkPreview(`http://google.com/`, {\n followRedirects: `manual`,\n handleRedirects: (baseURL: string, forwardedURL: string) => {\n if (forwardedURL !== `http://www.google.com/`) {\n return false;\n }\n return true;\n },\n });\n\n expect(response.contentType).toEqual(`text/html`);\n expect(response.url).toEqual(`http://www.google.com/`);\n expect(response.mediaType).toEqual(`website`);\n });\n\n it(\"should handle override response body using onResponse option\", async () => {\n let firstParagraphText;\n\n const res: any = await getLinkPreview(`https://www.example.com/`, {\n onResponse: (result, doc) => {\n firstParagraphText = doc(\"p\")\n .first()\n .text()\n .split(\"\\n\")\n .map((x) => x.trim())\n .join(\" \");\n result.siteName = `SiteName has been overridden`;\n result.description = firstParagraphText;\n\n return result;\n },\n });\n\n expect(res.siteName).toEqual(\"SiteName has been overridden\");\n expect(res.description).toEqual(firstParagraphText);\n });\n\n it(\"should handle video tags without type or secure_url tags\", async () => {\n const res: any = await getLinkPreview(\n `https://newpathtitle.com/falling-markets-how-to-stop-buyer-from-getting-out/`,\n { followRedirects: `follow` }\n );\n\n expect(res.siteName).toEqual(`New Path Title`);\n expect(res.title).toEqual(\n `Falling Markets: How To Stop A Buyer From Getting Out | New Path Title`\n );\n expect(res.description).toBeTruthy();\n expect(res.mediaType).toEqual(`article`);\n expect(res.images.length).toBeGreaterThan(0);\n expect(res.videos.length).toBeGreaterThan(0);\n expect(res.videos[0].url).toEqual(\n `https://www.youtube.com/embed/nqNXjxpAPkU`\n );\n expect(res.favicons.length).toBeGreaterThan(0);\n expect(res.contentType.toLowerCase()).toEqual(`text/html`);\n });\n});\n\ndescribe(`#getPreviewFromContent`, () => {\n it(`Basic parsing`, async () => {\n const linkInfo: any = await getPreviewFromContent(prefetchedResponse);\n\n expect(linkInfo.url).toEqual(`https://www.youtube.com/watch?v=wuClZjOdT30`);\n expect(linkInfo.siteName).toEqual(`YouTube`);\n expect(linkInfo.title).toEqual(`Geography Now! Germany`);\n expect(linkInfo.description).toBeTruthy();\n expect(linkInfo.mediaType).toEqual(`video.other`);\n expect(linkInfo.images.length).toEqual(1);\n expect(linkInfo.images[0]).toEqual(\n `https://i.ytimg.com/vi/wuClZjOdT30/maxresdefault.jpg`\n );\n expect(linkInfo.videos.length).toEqual(0);\n expect(linkInfo.favicons[0]).not.toBe(``);\n expect(linkInfo.contentType.toLowerCase()).toEqual(`text/html`);\n });\n});\n" }, { "path": "__tests__/sampleResponse.json", "content": "{\"url\":\"https://www.youtube.com/watch?v=wuClZjOdT30\",\"headers\":{\"alt-svc\":\"h3-29=\\\":443\\\"; ma=2592000,h3-T051=\\\":443\\\"; ma=2592000,h3-Q050=\\\":443\\\"; ma=2592000,h3-Q046=\\\":443\\\"; ma=2592000,h3-Q043=\\\":443\\\"; ma=2592000,quic=\\\":443\\\"; ma=2592000; v=\\\"46,43\\\"\",\"cache-control\":\"no-cache, no-store, max-age=0, must-revalidate\",\"connection\":\"close\",\"content-encoding\":\"gzip\",\"content-type\":\"text/html; charset=utf-8\",\"date\":\"Fri, 11 Dec 2020 19:36:56 GMT\",\"expires\":\"Mon, 01 Jan 1990 00:00:00 GMT\",\"p3p\":\"CP=\\\"This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info.\\\"\",\"pragma\":\"no-cache\",\"server\":\"ESF\",\"set-cookie\":\"YSC=jCz2JZmjRfE; Domain=.youtube.com; Path=/; Secure; HttpOnly; SameSite=none, VISITOR_INFO1_LIVE=oa3D7kCWj44; Domain=.youtube.com; Expires=Wed, 09-Jun-2021 19:36:56 GMT; Path=/; Secure; HttpOnly; SameSite=none\",\"strict-transport-security\":\"max-age=31536000\",\"transfer-encoding\":\"chunked\",\"x-content-type-options\":\"nosniff\",\"x-frame-options\":\"SAMEORIGIN\",\"x-xss-protection\":\"0\"},\"data\":\"Geography Now! Germany - YouTube
AboutPressCopyrightContact usCreatorsAdvertiseDevelopersImpressumNetzDG TransparenzberichtNetzDG ComplaintsTermsPrivacyPolicy & SafetyHow YouTube worksTest new features
© 2020 Google LLC
\"}" }, { "path": "bump-version.sh", "content": "#!/bin/bash\n\nset -ex\n\nnpm --no-git-tag-version version patch\n\ngit add .\n\ngit commit -m \"Bump version\"\n\n# git tag $PACKAGE_VERSION\n\ngit push" }, { "path": "constants.ts", "content": "export const CONSTANTS = {\n REGEX_VALID_URL: new RegExp(\n \"^\" +\n // protocol identifier\n \"(?:(?:https?|ftp)://)\" +\n // user:pass authentication\n \"(?:\\\\S+(?::\\\\S*)?@)?\" +\n \"(?:\" +\n // IP address exclusion\n // private & local networks\n \"(?!(?:10|127)(?:\\\\.\\\\d{1,3}){3})\" +\n \"(?!(?:169\\\\.254|192\\\\.168)(?:\\\\.\\\\d{1,3}){2})\" +\n \"(?!172\\\\.(?:1[6-9]|2\\\\d|3[0-1])(?:\\\\.\\\\d{1,3}){2})\" +\n // IP address dotted notation octets\n // excludes loopback network 0.0.0.0\n // excludes reserved space >= 224.0.0.0\n // excludes network & broacast addresses\n // (first & last IP address of each class)\n \"(?:[1-9]\\\\d?|1\\\\d\\\\d|2[01]\\\\d|22[0-3])\" +\n \"(?:\\\\.(?:1?\\\\d{1,2}|2[0-4]\\\\d|25[0-5])){2}\" +\n \"(?:\\\\.(?:[1-9]\\\\d?|1\\\\d\\\\d|2[0-4]\\\\d|25[0-4]))\" +\n \"|\" +\n // host name\n \"(?:(?:[a-z\\\\u00a1-\\\\uffff0-9]-*)*[a-z\\\\u00a1-\\\\uffff0-9]+)\" +\n // domain name\n \"(?:\\\\.(?:[a-z\\\\u00a1-\\\\uffff0-9]-*)*[a-z\\\\u00a1-\\\\uffff0-9]+)*\" +\n // TLD identifier\n \"(?:\\\\.(?:[a-z\\\\u00a1-\\\\uffff]{2,}))\" +\n // TLD may end with dot\n \"\\\\.?\" +\n \")\" +\n // port number\n \"(?::\\\\d{2,5})?\" +\n // resource path\n \"(?:[/?#]\\\\S*)?\" +\n \"$\",\n \"i\"\n ),\n\n REGEX_LOOPBACK: new RegExp(\n \"^\" +\n // Loopback: 127.0.0.0 - 127.255.255.255\n \"(?:127(?:\\\\.\\\\d{1,3}){3})\" +\n \"|\" +\n // Private Class A: 10.0.0.0 - 10.255.255.255\n \"(?:10(?:\\\\.\\\\d{1,3}){3})\" +\n \"|\" +\n // Private Class B: 172.16.0.0 - 172.31.255.255\n \"(?:172\\\\.(?:1[6-9]|2\\\\d|3[0-1])(?:\\\\.\\\\d{1,3}){2})\" +\n \"|\" +\n // Private Class C: 192.168.0.0 - 192.168.255.255\n \"(?:192\\\\.168(?:\\\\.\\\\d{1,3}){2})\" +\n \"|\" +\n // Link-local: 169.254.0.0 - 169.254.255.255\n \"(?:169\\\\.254(?:\\\\.\\\\d{1,3}){2})\" +\n \"|\" +\n // Documentation: 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24\n \"(?:192\\\\.0\\\\.2(?:\\\\.\\\\d{1,3}){1})\" +\n \"|\" +\n \"(?:198\\\\.51\\\\.100(?:\\\\.\\\\d{1,3}){1})\" +\n \"|\" +\n \"(?:203\\\\.0\\\\.113(?:\\\\.\\\\d{1,3}){1})\" +\n \"|\" +\n // Carrier-Grade NAT (CGNAT): 100.64.0.0 - 100.127.255.255\n \"(?:100\\\\.(?:6[4-9]|[7-9]\\\\d|1[0-1]\\\\d)(?:\\\\.\\\\d{1,3}){2})\" +\n \"$\",\n \"i\"\n ),\n\n REGEX_CONTENT_TYPE_IMAGE: new RegExp(\"image/.*\", \"i\"),\n\n REGEX_CONTENT_TYPE_AUDIO: new RegExp(\"audio/.*\", \"i\"),\n\n REGEX_CONTENT_TYPE_VIDEO: new RegExp(\"video/.*\", \"i\"),\n\n REGEX_CONTENT_TYPE_TEXT: new RegExp(\"text/.*\", \"i\"),\n\n REGEX_CONTENT_TYPE_APPLICATION: new RegExp(\"application/.*\", \"i\"),\n};\n" }, { "path": "index.ts", "content": "import cheerio from \"cheerio\";\nimport { CONSTANTS } from \"./constants\";\n\ninterface ILinkPreviewResponse {\n url: string;\n title: string;\n siteName: string | undefined;\n author: string | undefined;\n description: string | undefined;\n mediaType: string;\n contentType: string | undefined;\n images: string[];\n videos: IVideoType[];\n favicons: string[];\n}\n\ninterface IVideoType {\n url: string | undefined;\n secureUrl: string | null | undefined;\n type: string | null | undefined;\n width: string | undefined;\n height: string | undefined;\n}\n\ninterface ILinkPreviewOptions {\n headers?: Record;\n imagesPropertyType?: string;\n proxyUrl?: string;\n timeout?: number;\n followRedirects?: `follow` | `error` | `manual`;\n resolveDNSHost?: (url: string) => Promise;\n handleRedirects?: (baseURL: string, forwardedURL: string) => boolean;\n onResponse?: (\n response: ILinkPreviewResponse,\n doc: cheerio.Root,\n url?: URL,\n ) => ILinkPreviewResponse;\n}\n\ninterface IPreFetchedResource {\n headers: Record;\n status?: number;\n imagesPropertyType?: string;\n proxyUrl?: string;\n url: string;\n data: string;\n}\n\nfunction throwOnLoopback(address: string) {\n if (CONSTANTS.REGEX_LOOPBACK.test(address)) {\n throw new Error(\"SSRF request detected, trying to query host\");\n }\n}\n\nfunction metaTag(doc: cheerio.Root, type: string, attr: string) {\n const nodes = doc(`meta[${attr}='${type}']`);\n return nodes.length ? nodes : null;\n}\n\nfunction metaTagContent(doc: cheerio.Root, type: string, attr: string) {\n return doc(`meta[${attr}='${type}']`).attr(`content`);\n}\n\nfunction getTitle(doc: cheerio.Root) {\n let title =\n metaTagContent(doc, `og:title`, `property`) ||\n metaTagContent(doc, `og:title`, `name`);\n if (!title) {\n title = doc(`head > title`).text();\n }\n return title;\n}\n\nfunction getSiteName(doc: cheerio.Root) {\n const siteName =\n metaTagContent(doc, `og:site_name`, `property`) ||\n metaTagContent(doc, `og:site_name`, `name`);\n return siteName;\n}\n\nfunction getAuthor(doc: cheerio.Root) {\n const author =\n metaTagContent(doc, `author`, `name`) ||\n metaTagContent(doc, `article:author`, `property`);\n return author;\n}\n\nfunction getDescription(doc: cheerio.Root) {\n const description =\n metaTagContent(doc, `description`, `name`) ||\n metaTagContent(doc, `Description`, `name`) ||\n metaTagContent(doc, `og:description`, `property`);\n return description;\n}\n\nfunction getMediaType(doc: cheerio.Root) {\n const node = metaTag(doc, `medium`, `name`);\n if (node) {\n const content = node.attr(`content`);\n return content === `image` ? `photo` : content;\n }\n return (\n metaTagContent(doc, `og:type`, `property`) ||\n metaTagContent(doc, `og:type`, `name`)\n );\n}\n\nfunction getImages(\n doc: cheerio.Root,\n rootUrl: string,\n imagesPropertyType?: string,\n) {\n let images: string[] = [];\n let nodes: cheerio.Cheerio | null;\n let src: string | undefined;\n let dic: Record = {};\n\n const imagePropertyType = imagesPropertyType ?? `og`;\n nodes =\n metaTag(doc, `${imagePropertyType}:image`, `property`) ||\n metaTag(doc, `${imagePropertyType}:image`, `name`);\n\n if (nodes) {\n nodes.each((_: number, node: cheerio.Element) => {\n if (node.type === `tag`) {\n src = node.attribs.content;\n if (src) {\n src = new URL(src, rootUrl).href;\n images.push(src);\n }\n }\n });\n }\n\n if (images.length <= 0 && !imagesPropertyType) {\n src = doc(`link[rel=image_src]`).attr(`href`);\n if (src) {\n src = new URL(src, rootUrl).href;\n images = [src];\n } else {\n nodes = doc(`img`);\n\n if (nodes?.length) {\n dic = {};\n images = [];\n nodes.each((_: number, node: cheerio.Element) => {\n if (node.type === `tag`) src = node.attribs.src;\n if (src && !dic[src]) {\n dic[src] = true;\n // width = node.attribs.width;\n // height = node.attribs.height;\n images.push(new URL(src, rootUrl).href);\n }\n });\n }\n }\n }\n\n return images;\n}\n\nfunction getVideos(doc: cheerio.Root) {\n const videos = [];\n let nodeTypes;\n let nodeSecureUrls;\n let nodeType;\n let nodeSecureUrl;\n let video;\n let videoType;\n let videoSecureUrl;\n let width;\n let height;\n let videoObj;\n let index;\n\n const nodes =\n metaTag(doc, `og:video`, `property`) || metaTag(doc, `og:video`, `name`);\n\n if (nodes?.length) {\n nodeTypes =\n metaTag(doc, `og:video:type`, `property`) ||\n metaTag(doc, `og:video:type`, `name`);\n nodeSecureUrls =\n metaTag(doc, `og:video:secure_url`, `property`) ||\n metaTag(doc, `og:video:secure_url`, `name`);\n width =\n metaTagContent(doc, `og:video:width`, `property`) ||\n metaTagContent(doc, `og:video:width`, `name`);\n height =\n metaTagContent(doc, `og:video:height`, `property`) ||\n metaTagContent(doc, `og:video:height`, `name`);\n\n for (index = 0; index < nodes.length; index += 1) {\n const node = nodes[index];\n if (node.type === `tag`) video = node.attribs.content;\n\n nodeType = nodeTypes?.[index];\n if (nodeType?.type === `tag`) {\n videoType = nodeType ? nodeType.attribs.content : null;\n }\n\n nodeSecureUrl = nodeSecureUrls?.[index];\n if (nodeSecureUrl?.type === `tag`) {\n videoSecureUrl = nodeSecureUrl ? nodeSecureUrl.attribs.content : null;\n }\n\n videoObj = {\n url: video,\n secureUrl: videoSecureUrl,\n type: videoType,\n width,\n height,\n };\n if (videoType && videoType.indexOf(`video/`) === 0) {\n videos.splice(0, 0, videoObj);\n } else {\n videos.push(videoObj);\n }\n }\n }\n\n return videos;\n}\n\n// returns default favicon (//hostname/favicon.ico) for a url\nfunction getDefaultFavicon(rootUrl: string): string {\n return new URL(`/favicon.ico`, rootUrl).href;\n}\n\n// returns an array of URLs to favicon images\nfunction getFavicons(doc: cheerio.Root, rootUrl: string): string[] {\n const images = [];\n let nodes: cheerio.Cheerio | never[] = [];\n let src: string | undefined;\n\n const relSelectors = [\n `rel=icon`,\n `rel=\"shortcut icon\"`,\n `rel=apple-touch-icon`,\n ];\n\n relSelectors.forEach((relSelector) => {\n // look for all icon tags\n nodes = doc(`link[${relSelector}]`);\n\n // collect all images from icon tags\n if (nodes.length) {\n nodes.each((_: number, node: cheerio.Element) => {\n if (node.type === `tag`) src = node.attribs.href;\n if (src) {\n src = new URL(src, rootUrl).href;\n images.push(src);\n }\n });\n }\n });\n\n // if no icon images, use default favicon location\n if (images.length <= 0) {\n images.push(getDefaultFavicon(rootUrl));\n }\n\n return images;\n}\n\nfunction parseImageResponse(url: string, contentType: string) {\n return {\n url,\n mediaType: `image`,\n contentType,\n favicons: [getDefaultFavicon(url)],\n };\n}\n\nfunction parseAudioResponse(url: string, contentType: string) {\n return {\n url,\n mediaType: `audio`,\n contentType,\n favicons: [getDefaultFavicon(url)],\n };\n}\n\nfunction parseVideoResponse(url: string, contentType: string) {\n return {\n url,\n mediaType: `video`,\n contentType,\n favicons: [getDefaultFavicon(url)],\n };\n}\n\nfunction parseApplicationResponse(url: string, contentType: string) {\n return {\n url,\n mediaType: `application`,\n contentType,\n favicons: [getDefaultFavicon(url)],\n };\n}\n\nfunction parseTextResponse(\n body: string,\n url: string,\n options: ILinkPreviewOptions = {},\n contentType?: string,\n): ILinkPreviewResponse {\n const doc = cheerio.load(body);\n\n let response = {\n url,\n title: getTitle(doc),\n siteName: getSiteName(doc),\n description: getDescription(doc),\n author: getAuthor(doc),\n mediaType: getMediaType(doc) || `website`,\n contentType,\n images: getImages(doc, url, options.imagesPropertyType),\n videos: getVideos(doc),\n favicons: getFavicons(doc, url),\n };\n\n if (options?.onResponse && typeof options.onResponse !== `function`) {\n throw new Error(`link-preview-js onResponse option must be a function`);\n }\n\n if (options?.onResponse) {\n // send in a cloned response (to avoid mutation of original response reference)\n const clonedResponse = structuredClone(response);\n const urlObject = new URL(url);\n response = options.onResponse(clonedResponse, doc, urlObject);\n }\n\n return response;\n}\n\nfunction parseUnknownResponse(\n body: string,\n url: string,\n options: ILinkPreviewOptions = {},\n contentType?: string,\n) {\n return parseTextResponse(body, url, options, contentType);\n}\n\nfunction parseResponse(\n response: IPreFetchedResource,\n options?: ILinkPreviewOptions,\n) {\n try {\n // console.log(\"[link-preview-js] response\", response);\n let contentType = response.headers[`content-type`];\n let contentTypeTokens: string[] = [];\n let charset = null;\n\n if (!contentType) {\n return parseUnknownResponse(response.data, response.url, options);\n }\n\n if (contentType.includes(`;`)) {\n contentTypeTokens = contentType.split(`;`);\n contentType = contentTypeTokens[0];\n\n for (let token of contentTypeTokens) {\n if (token.indexOf(\"charset=\") !== -1) {\n charset = token.split(\"=\")[1];\n }\n }\n }\n\n // parse response depending on content type\n if (CONSTANTS.REGEX_CONTENT_TYPE_IMAGE.test(contentType)) {\n return { ...parseImageResponse(response.url, contentType), charset };\n }\n\n if (CONSTANTS.REGEX_CONTENT_TYPE_AUDIO.test(contentType)) {\n return { ...parseAudioResponse(response.url, contentType), charset };\n }\n\n if (CONSTANTS.REGEX_CONTENT_TYPE_VIDEO.test(contentType)) {\n return { ...parseVideoResponse(response.url, contentType), charset };\n }\n\n if (CONSTANTS.REGEX_CONTENT_TYPE_TEXT.test(contentType)) {\n return {\n ...parseTextResponse(response.data, response.url, options, contentType),\n charset,\n };\n }\n\n if (CONSTANTS.REGEX_CONTENT_TYPE_APPLICATION.test(contentType)) {\n return {\n ...parseApplicationResponse(response.url, contentType),\n charset,\n };\n }\n\n const htmlString = response.data;\n\n return {\n ...parseUnknownResponse(htmlString, response.url, options),\n charset,\n };\n } catch (e) {\n throw new Error(\n `link-preview-js could not fetch link information ${(\n e as any\n ).toString()}`,\n );\n }\n}\n\n/**\n * Parses the text, extracts the first link it finds and does a HTTP request\n * to fetch the website content, afterwards it tries to parse the internal HTML\n * and extract the information via meta tags\n * @param text string, text to be parsed\n * @param options ILinkPreviewOptions\n */\nexport async function getLinkPreview(\n text: string,\n options?: ILinkPreviewOptions,\n) {\n if (!text || typeof text !== `string`) {\n throw new Error(`link-preview-js did not receive a valid url or text`);\n }\n\n const detectedUrl = text\n .replace(/\\n/g, ` `)\n .split(` `)\n .find((token) => CONSTANTS.REGEX_VALID_URL.test(token));\n\n if (!detectedUrl) {\n throw new Error(`link-preview-js did not receive a valid a url or text`);\n }\n\n if (options?.followRedirects === `manual` && !options?.handleRedirects) {\n throw new Error(\n `link-preview-js followRedirects is set to manual, but no handleRedirects function was provided`,\n );\n }\n\n if (!!options?.resolveDNSHost) {\n const resolvedUrl = await options.resolveDNSHost(detectedUrl);\n\n throwOnLoopback(resolvedUrl);\n }\n\n const timeout = options?.timeout ?? 3000; // 3 second timeout default\n const controller = new AbortController();\n const timeoutCounter = setTimeout(() => controller.abort(), timeout);\n\n const fetchOptions = {\n headers: options?.headers ?? {},\n redirect: options?.followRedirects ?? `error`,\n signal: controller.signal,\n };\n\n const fetchUrl = options?.proxyUrl\n ? options.proxyUrl.concat(detectedUrl)\n : detectedUrl;\n\n let response = await fetch(fetchUrl, fetchOptions).catch((e) => {\n if (e.name === `AbortError`) {\n throw new Error(`Request timeout`);\n }\n\n clearTimeout(timeoutCounter);\n throw e;\n });\n\n if (\n response.status > 300 &&\n response.status < 309 &&\n fetchOptions.redirect === `manual` &&\n options?.handleRedirects\n ) {\n const locationHeader = response.headers.get(`location`) || ``;\n const isAbsoluteURI =\n locationHeader.startsWith(\"http://\") ||\n locationHeader.startsWith(\"https://\");\n\n // Resolve the URL, handling both absolute and relative URLs\n const forwardedUrl = isAbsoluteURI\n ? locationHeader\n : new URL(locationHeader, fetchUrl).href;\n\n if (!options.handleRedirects(fetchUrl, forwardedUrl)) {\n throw new Error(`link-preview-js could not handle redirect`);\n }\n\n if (!!options?.resolveDNSHost) {\n const resolvedUrl = await options.resolveDNSHost(forwardedUrl);\n\n throwOnLoopback(resolvedUrl);\n }\n\n response = await fetch(forwardedUrl, fetchOptions as any);\n }\n\n clearTimeout(timeoutCounter);\n\n const headers: Record = {};\n response.headers.forEach((header, key) => {\n headers[key] = header;\n });\n\n const normalizedResponse: IPreFetchedResource = {\n url: options?.proxyUrl\n ? response.url.replace(options.proxyUrl, ``)\n : response.url,\n headers,\n data: await response.text(),\n };\n\n return parseResponse(normalizedResponse, options);\n}\n\n/**\n * Skip the library fetching the website for you, instead pass a response object\n * from whatever source you get and use the internal parsing of the HTML to return\n * the necessary information\n * @param response Preview Response\n * @param options IPreviewLinkOptions\n */\nexport async function getPreviewFromContent(\n response: IPreFetchedResource,\n options?: ILinkPreviewOptions,\n) {\n if (!response || typeof response !== `object`) {\n throw new Error(`link-preview-js did not receive a valid response object`);\n }\n\n if (!response.url) {\n throw new Error(`link-preview-js did not receive a valid response object`);\n }\n\n return parseResponse(response, options);\n}\n" }, { "path": "jest.config.js", "content": "module.exports = {\n preset: `ts-jest`,\n testEnvironment: `node`,\n globals: { \"ts-jest\": { diagnostics: false } },\n testPathIgnorePatterns: [\"/build/\"],\n};\n" }, { "path": "mise.toml", "content": "[tools]\nnode = \"24\"\n" }, { "path": "package.json", "content": "{\n \"name\": \"link-preview-js\",\n \"version\": \"3.0.6\",\n \"description\": \"Javascript module to extract and fetch HTTP link information from blocks of text.\",\n \"main\": \"build/index.js\",\n \"exports\": {\n \".\": {\n \"require\": \"./build/index.js\",\n \"import\": \"./build/index.js\"\n },\n \"./package.json\": \"./package.json\"\n },\n \"types\": \"build/index.d.ts\",\n \"scripts\": {\n \"test\": \"jest\",\n \"test:ci\": \"jest --testLocationInResults --ci --outputFile=test_results.json --json\",\n \"build\": \"tsc\",\n \"bump\": \"./bump-version.sh\",\n \"prepublishOnly\": \"tsc\"\n },\n \"keywords\": [\n \"javascript\",\n \"link\",\n \"url\",\n \"http\",\n \"preview\"\n ],\n \"author\": \"Oscar Franco\",\n \"email\": \"ospfranco@protonmail.com\",\n \"license\": \"MIT\",\n \"repository\": \"https://github.com/ospfranco/link-preview-js\",\n \"dependencies\": {\n \"cheerio\": \"1.0.0-rc.11\"\n },\n \"files\": [\n \"build\"\n ],\n \"devDependencies\": {\n \"@skypack/package-check\": \"^0.2.2\",\n \"@types/cheerio\": \"0.22.24\",\n \"@types/jest\": \"^28.1.4\",\n \"jest\": \"^28.1.2\",\n \"prettier\": \"2.7.1\",\n \"ts-jest\": \"^28.0.5\",\n \"typescript\": \"^4.7.4\"\n },\n \"packageManager\": \"yarn@1.22.22+sha1.ac34549e6aa8e7ead463a7407e1c7390f61a6610\",\n \"engines\": {\n \"node\": \">=18\"\n }\n}\n" }, { "path": "tsconfig.json", "content": "{\n \"compilerOptions\": {\n \"outDir\": \"build\",\n \"allowSyntheticDefaultImports\": true,\n \"esModuleInterop\": true,\n \"moduleResolution\": \"node\",\n \"declaration\": true,\n \"target\": \"es5\",\n \"skipLibCheck\": true,\n \"strict\": true,\n \"resolveJsonModule\": true\n },\n \"exclude\": [\"node_modules\", \"build\"]\n}\n" } ]