Browse agent skills

No description available

Plugin for bauer-crawler to scrape content.

A prerender middleware for express. This middleware serves cached prerendered pages for crawlers or if no cached page exists for a requested page it will cache a page using PhantomJS and respond with that newly cached page.

a video downloader that crawls the web

Plugin for bauer-crawler to convert PDF into text.

Scrapy Framework implemented by nodejs.

A Mercator crawler with url-frontier, url-set, meta-data fetcher, etc. inspired by the Stanford Information Retrieval book

Pull athlete information straight from the Parkrun website

a module to interact with the crawler tables stored in hbase

auto-web-crawler is a ai & automation skill indexed by Skillful.sh. Package: auto-web-crawler. GitHub stars: 4.

A painless Node.js web crawler that simply works

Web Crawler in Node.js to spider dynamically whole websites.

Crawl your CSS/SCSS or HTML files for img URL's and store the crawled image URL's in a local JSON file.

crawlerlibrary is a ai & automation skill indexed by Skillful.sh. Package: crawlerlibrary.

Crawl all urls of a domain

microcrawler-crawler-all is a ai & automation skill indexed by Skillful.sh. Package: microcrawler-crawler-all.

This library provides support for traversing objects and their values while providing information on the traversal state, pathing to target values, and the ability to manipulate said pathing to easily move to related values.

A NodeJS web crawler that can be deployed to multiple machines and writes page data to a Firebase database.

Module comparing two objects for key parity.

Scalable, extensible, web crawler framework.

Web site crawler using cheerio to scan HTML documents

Plugin for bauer-crawler to extract values from JSON data.

Simple node.js SEO content crawler for SPAs with cache

Yet another Node crawler.

crawler-common is a ai & automation skill indexed by Skillful.sh. Package: crawler-common.

Simple facebook posts crawler

The server crawler for Rabbit Server List

pika-web-crawler-test is a testing & review skill indexed by Skillful.sh. Package: pika-web-crawler-test. GitHub stars: 2.

A node library to crawler imdb and parse to json

typescript dcinside crawler