Web Scraping Policy

1. Purpose and scope

This policy sets out how Hudson Labs approaches web scraping (also referred to as web crawling or automated data collection). It applies to all Hudson Labs personnel and to any automated collection of data from public websites and public data sources carried out by or on behalf of Hudson Labs — whether in the ordinary course of operating our platform or as part of a one-off Deep Dive research project. It does not apply to data that Hudson Labs licenses from third-party providers or receives through provider APIs or SFTP, which is governed by the applicable provider agreements.

2. Definitions

• Web scraping. The automated retrieval of information from websites or online data sources, including crawling, parsing, and programmatic collection.
• Public data source / public website. A source, or a portion of a website, that is accessible to the general public without a private login, paywall, or other access control.
• Limited scraping. Low-volume, narrowly targeted retrieval of specific public data points (for example, a small set of company reference fields), as distinct from systematic or bulk collection.
• More-than-limited scraping. Any collection beyond the limited category — broader in scope, higher in volume, or systematic — which triggers a detailed terms-of-service review before proceeding.
• Deep Dive. A one-off, project-specific research exercise that may involve broad public web searches and targeted retrieval (by scraping or public API) from miscellaneous public sources. Deep Dives are not conducted on an ongoing basis.

3. Core principles

Hudson Labs conducts web scraping only within the following limits:

• We scrape only public data sources and the public parts of websites. We do not scrape non-public data.
• We do not scrape behind private logins or paywalls.
• We identify ourselves where required, and we do not disguise or misrepresent our identity to gain access.
• We never swarm a site, and we do not attempt to override, bypass, evade, or defeat CAPTCHAs, firewalls, or other access controls.
• We never scrape with the intent to recreate a source's dataset or to replicate a complete website.
• Where we perform more than limited scraping, we review the source's terms of service in detail and confirm compliance before proceeding. If we cannot confirm compliance, we do not scrape the source.

4. What we scrape

Normal platform use

• Company reference details from SEC.gov — for example, sector, subsector, ticker, company identifiers, and company changes.
• Investor relations (IR) websites — to identify new events and disclosures and incorporate them into our systems where they are otherwise missing.

Deep Dives only (one-off, not ongoing)

• Broad-based public web searches and targeted retrieval — by scraping or via public APIs — of subsets of data from miscellaneous public sources. Examples include JASDEC (Japan Securities Depository Center), FRED (Federal Reserve Economic Data), the Federal Register, corporate websites, and public news sources. Compliance for each Deep Dive is evaluated at the time of the project.

What we do not scrape

• Any content behind a private login, account, or paywall.
• Any non-public area of a website.
• Anything collected for the purpose of reconstructing a source's dataset or a complete website.
• Sources whose terms do not permit our intended activity (see Section 5).

5. Compliance and legal review

Hudson Labs applies a tiered review:

• Limited scraping in the ordinary course. We operate under the standing principles in this policy and confirm, before collecting from a source, that the data is public and that our activity is consistent with the site's terms and any published access requirements (such as rate limits and identification requirements).
• More-than-limited scraping. We conduct a detailed review of the source's terms of service or use and confirm compliance before proceeding. Where compliance cannot be confirmed, we do not scrape.
• Deep Dives. Compliance is evaluated on a per-project basis at the time of the Deep Dive, because these are one-off and source-specific.

Worked examples

• SEC.gov — reviewed; we scrape in strict compliance with SEC.gov's terms and published access requirements.
• SEDAR / SEDAR+ — reviewed; we have determined that scraping is not compliant with its terms, and therefore we do not scrape it.

6. Operational safeguards

• Volume and rate. Our scraping is targeted and low-volume. We rate-limit and throttle requests, honor published rate limits and robots directives where applicable, and avoid patterns that could burden a site. If we detect that our activity could overwhelm a target, we back off or pause. We never swarm.
• Identification and traceability. We identify ourselves where required — for example, through a descriptive User-Agent and, where appropriate, contact information — so that a site operator can attribute requests to Hudson Labs and reach us if needed. We do not spoof User-Agents or otherwise conceal our identity. We identify using our User-Agent string and scraping contact address, e.g., info@hudson-labs.com.
• Access controls. Encountering a CAPTCHA, firewall, login, paywall, or other barrier is treated as a signal that automated access is not intended. We stop; we do not attempt to bypass or defeat the control.

7. Frequently Asked Questions

8. Governance and review

• Ownership. This policy is owned by the Hudson Labs Privacy and Security Officer, who is responsible for its application, for approving more-than-limited scraping and Deep Dive compliance evaluations, and for maintaining supporting documentation.
• Review. This policy is reviewed at least annually and updated as our practices or a source's terms change.
• Contact. Questions about this policy or about Hudson Labs' scraping activity may be directed to info@hudson-labs.com.