Browse agent skills

Automated crawling and messaging for LinkedIn

No description available

Plugin for bauer-crawler to scrape content.

A prerender middleware for express. This middleware serves cached prerendered pages for crawlers or if no cached page exists for a requested page it will cache a page using PhantomJS and respond with that newly cached page.

Forked from https://github.com/github/lightcrawler

a video downloader that crawls the web

A Typescript library to fetch data from GitHub projects and write them to JSON files for the HocDoc parser.

Plugin for bauer-crawler to convert PDF into text.

Scrapy Framework implemented by nodejs.

A Mercator crawler with url-frontier, url-set, meta-data fetcher, etc. inspired by the Stanford Information Retrieval book

Config based news crawler using Google Puppeteer

Pull athlete information straight from the Parkrun website

a module to interact with the crawler tables stored in hbase

auto-web-crawler is a ai & automation skill indexed by Skillful.sh. Package: auto-web-crawler. GitHub stars: 4.

A painless Node.js web crawler that simply works

Web Crawler in Node.js to spider dynamically whole websites.

https://user-images.githubusercontent.com/74648201/216847852-66d3cab4-4f1e-4125-a54f-95e96f1b95e7.mp4

Crawl your CSS/SCSS or HTML files for img URL's and store the crawled image URL's in a local JSON file.

crawlerlibrary is a ai & automation skill indexed by Skillful.sh. Package: crawlerlibrary.

Crawl all urls of a domain

microcrawler-crawler-all is a ai & automation skill indexed by Skillful.sh. Package: microcrawler-crawler-all.

This library provides support for traversing objects and their values while providing information on the traversal state, pathing to target values, and the ability to manipulate said pathing to easily move to related values.

A NodeJS web crawler that can be deployed to multiple machines and writes page data to a Firebase database.

Module comparing two objects for key parity.

Scalable, extensible, web crawler framework.

Web site crawler using cheerio to scan HTML documents

Plugin for bauer-crawler to extract values from JSON data.

Simple node.js SEO content crawler for SPAs with cache

Yet another Node crawler.

crawler-common is a ai & automation skill indexed by Skillful.sh. Package: crawler-common.