Apify → stratless

Apify’s Website Content Crawler returns clean text for a list of URLs. Crawl your set, then map it.

import { ApifyClient } from 'apify-client'
const apify = new ApifyClient({ token: process.env.APIFY_TOKEN })

const KEY = process.env.STRATLESS_API_KEY
const set = [
  { name: 'Pinecone', url: 'https://pinecone.io' },
  { name: 'Qdrant',   url: 'https://qdrant.tech' },
  { name: 'Weaviate', url: 'https://weaviate.io' },
]

// 1 — crawl each page to text
const run = await apify.actor('apify/website-content-crawler').call({
  startUrls: set.map((s) => ({ url: s.url })),
  maxCrawlPages: set.length,
})
const { items } = await apify.dataset(run.defaultDatasetId).listItems()
const pages = set.map((s) => items.find((i) => i.url?.startsWith(s.url))?.text ?? '')

// 2 — analyze the set → a competitive field map
const res = await fetch('https://api.stratless.com/v1/analyze', {
  method: 'POST',
  headers: { 'x-api-key': KEY, 'content-type': 'application/json' },
  body: JSON.stringify({
    texts: set.map((s, i) => ({ name: s.name, text: pages[i] })),
  }),
})
const fieldMap = await res.json()

See The analyze API for the full response shape.