Website Content Crawler: Markdown for AI (Half Price)

Crawl a site and get clean Markdown for LLMs, RAG and vector databases. Respects robots.txt.

Open on Apify Store

How to run

1. Open website-content-crawler-half-price on Apify (free Apify account works for a trial run).
2. Click Try for free / create a run.
3. Keep or edit the sample input below, then Start.
4. Download the dataset as JSON or CSV when the run succeeds.

Sample input

{
  "startUrls": [
    {
      "url": "https://www.python-httpx.org/"
    }
  ]
}

What you get

{ "startUrls": [{ "url": "https://www.python-httpx.org/" }], "maxPagesPerSite": 50, "maxDepth": 3,
  "useSitemap": true, "stayUnderStartPath": true, "outputMarkdown": true, "minWords": 20 }

FAQ

This guide describes Unadhi’s public Apify actor. Pricing on Apify is set in the Console Monetization tab; this page does not change prices or payouts. Never paste personal data you are not allowed to process.

• Do I need proxies? Follow each actor README; our contact and jobs actors prefer official public APIs or polite HTTP without residential proxies where possible.
• Is this legal advice? No. Accessibility and privacy tools do not certify compliance.