Browse agent skills

A lightweight, Open Source robots.txt parser written in JavaScript

Web Crawler for Node.js (Redis)

A web crawler that builds a tree of URLs

Crawls the DHT Network and publishes infohashes using ØMQ.

Fisheries price crawler for wpi.kkp.go.id

This tool helps you to generate lighthouse reports for provided sitemap URL & compare them effortlessly.

lei-crawler is a ai & automation skill indexed by Skillful.sh. Package: lei-crawler. GitHub stars: 7.

SEO Solution for MEAN.JS applications which forwards crawlers requests to a compiled HTML copy using PhantomJS.

fork from headless-chrome-crawler and update puppeteer to the latest version

No description available

Identifies bots/crawlers

casperjs server for scraping, with it you can simple write the script file , it can be executed in the browser and response you the result you expect. it can load ajax content as you are using browser.

[](https://badge.fury.io/js/plugin-books-pro)

No description available

An utility to crawl generic objects paths

URL crawler for analysing web content

Unofficial API for zhihu (https://www.zhihu.com)

Lightsabing Node/PhantomJS crawler. Crawl almost everything, including AJAX content.

spamlet is an efficient and simple crawler for playwright

Crawl data or download files from website with custom rules

node-web-crawler ================

crawlercore is a ai & automation skill indexed by Skillful.sh. Package: crawlercore. GitHub stars: 1.

BotWall SDK for site protection and bot crawling

This module is propose to scrapy website pages and extract information from doms which selected by jQuery-like selectors

Web crawling using Puppeteer

Crawler4nodejs is an open source web crawler for Node.js which provides a simple interface for crawling the Web.

Plugin of @ljw1412/web-crawler.Replace default request function with axios.

A simple web crawler that generates a sitemap

linkedin scraper for 2020 website

A crawler crawling rental information, based Nodejs