Skip to content
2 min read · 318 words

Interface: ScrapperLinksParams

Defined in: batteries/tools/scrapper/index.ts:80

Model-facing params for /api/links.

Extends

Properties

PropertyTypeDescriptionInherited fromDefined in
cache?booleanReturn a cached result when available instead of re-scraping.ScrapperCommonParams.cachebatteries/tools/scrapper/index.ts:50
device?stringEmulated device, e.g. Desktop Chrome. Overrides individual viewport/UA settings.ScrapperCommonParams.devicebatteries/tools/scrapper/index.ts:64
extra_http_headers?stringExtra headers the SCRAPER's browser sends to the TARGET site, K:v;K2:v2 (NOT instance auth).ScrapperCommonParams.extra_http_headersbatteries/tools/scrapper/index.ts:68
incognito?booleanRun in an incognito browser context (no persisted browsing data). Default true upstream.ScrapperCommonParams.incognitobatteries/tools/scrapper/index.ts:54
proxy_server?stringUpstream proxy, e.g. http://host:3128 or socks5://host:1080.ScrapperCommonParams.proxy_serverbatteries/tools/scrapper/index.ts:70
screenshot?booleanCapture a screenshot; the result carries a screenshotUri.ScrapperCommonParams.screenshotbatteries/tools/scrapper/index.ts:52
scroll_down?numberScroll down N pixels for lazy-loading pages. Requires a positive sleep.ScrapperCommonParams.scroll_downbatteries/tools/scrapper/index.ts:62
sleep?numberWait this many ms after load before parsing.ScrapperCommonParams.sleepbatteries/tools/scrapper/index.ts:60
text_len_threshold?numberMedian link-text length threshold for the link parser.-batteries/tools/scrapper/index.ts:82
timeout?numberBrowser navigation timeout in ms (0 disables). Distinct from the tool's own fetch timeout.ScrapperCommonParams.timeoutbatteries/tools/scrapper/index.ts:56
user_agent?stringExplicit user-agent (prefer device).ScrapperCommonParams.user_agentbatteries/tools/scrapper/index.ts:66
wait_until?"load" | "domcontentloaded" | "networkidle" | "commit"When navigation is considered finished.ScrapperCommonParams.wait_untilbatteries/tools/scrapper/index.ts:58
words_threshold?numberMedian words-per-link threshold for the link parser.-batteries/tools/scrapper/index.ts:84