跳至主要内容
小龙虾小龙虾AI
🤖

Apify Ultimate Scraper

Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, Google Maps, Google Search, Google Trends, Booking.com, and TripAdvisor. Use for lead generation, brand monitoring, competitor analysis, influencer discovery, trend research, content analytics, audience analysis, or any data extraction task.

下载850
星标6
版本1.0.1
数据分析
安全通过
⚙️脚本

技能说明


name: apify-ultimate-scraper description: Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, Google Maps, Google Search, Google Trends, Booking.com, and TripAdvisor. Use for lead generation, brand monitoring, competitor analysis, influencer discovery, trend research, content analytics, audience analysis, or any data extraction task. version: 1.0.8 source: https://github.com/apify/agent-skills homepage: https://apify.com metadata: openclaw: requires: env: - APIFY_TOKEN bins: - node - mcpc primaryEnv: APIFY_TOKEN install: - kind: node package: "@apify/mcpc" bins: [mcpc]

Universal Web Scraper

AI-driven data extraction from 55+ Actors across all major platforms. This skill automatically selects the best Actor for your task.

Prerequisites

  • APIFY_TOKEN configured in OpenClaw settings
  • Node.js 20.6+
  • mcpc CLI (auto-installed via skill metadata)

Input Sanitization Rules

Before substituting any value into a bash command:

  • ACTOR_ID: Must be either a technical name (owner/actor-name — alphanumeric, hyphens, dots, one slash) or a raw ID (exactly 17 alphanumeric characters, e.g., oeiQgfg5fsmIJB7Cn). Reject values containing shell metacharacters (; | & $ ` ( ) { } < > ! \n).
  • SEARCH_KEYWORDS: Plain text words only. Reject shell metacharacters.
  • JSON_INPUT: Must be valid JSON. Must not contain single quotes (use escaped double quotes). Validate structure before use.
  • Output filenames: Must match YYYY-MM-DD_descriptive-name.{csv,json}. No path separators (/, ..), no spaces, no metacharacters.

Workflow

Copy this checklist and track progress:

Task Progress:
- [ ] Step 1: Understand user goal and select Actor
- [ ] Step 2: Fetch Actor schema via mcpc
- [ ] Step 3: Ask user preferences (format, filename)
- [ ] Step 4: Run the scraper script
- [ ] Step 5: Summarize results and offer follow-ups

Step 1: Understand User Goal and Select Actor

First, understand what the user wants to achieve. Then select the best Actor from the options below.

Instagram Actors (12)

Actor IDBest For
apify/instagram-profile-scraperProfile data, follower counts, bio info
apify/instagram-post-scraperIndividual post details, engagement metrics
apify/instagram-comment-scraperComment extraction, sentiment analysis
apify/instagram-hashtag-scraperHashtag content, trending topics
apify/instagram-hashtag-statsHashtag performance metrics
apify/instagram-reel-scraperReels content and metrics
apify/instagram-search-scraperSearch users, places, hashtags
apify/instagram-tagged-scraperPosts tagged with specific accounts
apify/instagram-followers-count-scraperFollower count tracking
apify/instagram-scraperComprehensive Instagram data
apify/instagram-api-scraperAPI-based Instagram access
apify/export-instagram-comments-postsBulk comment/post export

Facebook Actors (14)

Actor IDBest For
apify/facebook-pages-scraperPage data, metrics, contact info
apify/facebook-page-contact-informationEmails, phones, addresses from pages
apify/facebook-posts-scraperPost content and engagement
apify/facebook-comments-scraperComment extraction
apify/facebook-likes-scraperReaction analysis
apify/facebook-reviews-scraperPage reviews
apify/facebook-groups-scraperGroup content and members
apify/facebook-events-scraperEvent data
apify/facebook-ads-scraperAd creative and targeting
apify/facebook-search-scraperSearch results
apify/facebook-reels-scraperReels content
apify/facebook-photos-scraperPhoto extraction
apify/facebook-marketplace-scraperMarketplace listings
apify/facebook-followers-following-scraperFollower/following lists

TikTok Actors (14)

Actor IDBest For
clockworks/tiktok-scraperComprehensive TikTok data
clockworks/free-tiktok-scraperFree TikTok extraction
clockworks/tiktok-profile-scraperProfile data
clockworks/tiktok-video-scraperVideo details and metrics
clockworks/tiktok-comments-scraperComment extraction
clockworks/tiktok-followers-scraperFollower lists
clockworks/tiktok-user-search-scraperFind users by keywords
clockworks/tiktok-hashtag-scraperHashtag content
clockworks/tiktok-sound-scraperTrending sounds
clockworks/tiktok-ads-scraperAd content
clockworks/tiktok-discover-scraperDiscover page content
clockworks/tiktok-explore-scraperExplore content
clockworks/tiktok-trends-scraperTrending content
clockworks/tiktok-live-scraperLive stream data

YouTube Actors (5)

Actor IDBest For
streamers/youtube-scraperVideo data and metrics
streamers/youtube-channel-scraperChannel information
streamers/youtube-comments-scraperComment extraction
streamers/youtube-shorts-scraperShorts content
streamers/youtube-video-scraper-by-hashtagVideos by hashtag

Google Maps Actors (4)

Actor IDBest For
compass/crawler-google-placesBusiness listings, ratings, contact info
compass/google-maps-extractorDetailed business data
compass/Google-Maps-Reviews-ScraperReview extraction
poidata/google-maps-email-extractorEmail discovery from listings

Other Actors (6)

Actor IDBest For
apify/google-search-scraperGoogle search results
apify/google-trends-scraperGoogle Trends data
voyager/booking-scraperBooking.com hotel data
voyager/booking-reviews-scraperBooking.com reviews
maxcopell/tripadvisor-reviewsTripAdvisor reviews
vdrmota/contact-info-scraperContact enrichment from URLs

Actor Selection by Use Case

Use CasePrimary Actors
Lead Generationcompass/crawler-google-places, poidata/google-maps-email-extractor, vdrmota/contact-info-scraper
Influencer Discoveryapify/instagram-profile-scraper, clockworks/tiktok-profile-scraper, streamers/youtube-channel-scraper
Brand Monitoringapify/instagram-tagged-scraper, apify/instagram-hashtag-scraper, compass/Google-Maps-Reviews-Scraper
Competitor Analysisapify/facebook-pages-scraper, apify/facebook-ads-scraper, apify/instagram-profile-scraper
Content Analyticsapify/instagram-post-scraper, clockworks/tiktok-scraper, streamers/youtube-scraper
Trend Researchapify/google-trends-scraper, clockworks/tiktok-trends-scraper, apify/instagram-hashtag-stats
Review Analysiscompass/Google-Maps-Reviews-Scraper, voyager/booking-reviews-scraper, maxcopell/tripadvisor-reviews
Audience Analysisapify/instagram-followers-count-scraper, clockworks/tiktok-followers-scraper, apify/facebook-followers-following-scraper

Multi-Actor Workflows

For complex tasks, chain multiple Actors:

WorkflowStep 1Step 2
Lead enrichmentcompass/crawler-google-placesvdrmota/contact-info-scraper
Influencer vettingapify/instagram-profile-scraperapify/instagram-comment-scraper
Competitor deep-diveapify/facebook-pages-scraperapify/facebook-posts-scraper
Local business analysiscompass/crawler-google-placescompass/Google-Maps-Reviews-Scraper

Can't Find a Suitable Actor?

If none of the Actors above match the user's request, search the Apify Store directly:

mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call search-actors keywords:="SEARCH_KEYWORDS" limit:=10 offset:=0 category:="" | jq -r '.content[0].text'

Replace SEARCH_KEYWORDS with 1-3 simple terms (e.g., "LinkedIn profiles", "Amazon products", "Twitter").

Step 2: Fetch Actor Schema

Fetch the Actor's input schema and details dynamically using mcpc:

mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call fetch-actor-details actor:="ACTOR_ID" | jq -r ".content"

Replace ACTOR_ID with the selected Actor (e.g., compass/crawler-google-places).

This returns:

  • Actor description and README
  • Required and optional input parameters
  • Output fields (if available)

Step 3: Ask User Preferences

Before running, ask:

  1. Output format:
    • Quick answer - Display top few results in chat (no file saved)
    • CSV - Full export with all fields
    • JSON - Full export in JSON format
  2. Number of results: Based on character of use case

Step 4: Run the Script

Quick answer (display in chat, no file):

node {baseDir}/reference/scripts/run_actor.js \
  --actor 'ACTOR_ID' \
  --input 'JSON_INPUT'

CSV:

node {baseDir}/reference/scripts/run_actor.js \
  --actor 'ACTOR_ID' \
  --input 'JSON_INPUT' \
  --output 'YYYY-MM-DD_OUTPUT_FILE.csv' \
  --format csv

JSON:

node {baseDir}/reference/scripts/run_actor.js \
  --actor 'ACTOR_ID' \
  --input 'JSON_INPUT' \
  --output 'YYYY-MM-DD_OUTPUT_FILE.json' \
  --format json

Step 5: Summarize Results and Offer Follow-ups

After completion, report:

  • Number of results found
  • File location and name
  • Key fields available
  • Suggested follow-up workflows based on results:
If User GotSuggest Next
Business listingsEnrich with vdrmota/contact-info-scraper or get reviews
Influencer profilesAnalyze engagement with comment scrapers
Competitor pagesDeep-dive with post/ad scrapers
Trend dataValidate with platform-specific hashtag scrapers

Security & Data Privacy

This skill instructs the agent to select an Apify Actor, fetch its schema (via mcpc), and run scrapers. The included script communicates only with api.apify.com and writes outputs to files under the current working directory; it does not access unrelated system files or other environment variables.

Apify Actors only scrape publicly available data and do not collect private or personally identifiable information beyond what is openly accessible on the target platforms. For additional security assurance, you can check an Actor's permission level by querying https://api.apify.com/v2/acts/:actorId — an Actor with LIMITED_PERMISSIONS operates in a restricted sandbox, while FULL_PERMISSIONS indicates broader system access. For full details, see Apify's General Terms and Conditions.

Error Handling

APIFY_TOKEN not found - Ask user to configure APIFY_TOKEN in OpenClaw settings mcpc not found - Run npm install -g @apify/mcpc Actor not found - Check Actor ID spelling Run FAILED - Ask user to check Apify console link in error output Timeout - Reduce input size or increase --timeout

如何使用「Apify Ultimate Scraper」?

  1. 打开小龙虾AI(Web 或 iOS App)
  2. 点击上方「立即使用」按钮,或在对话框中输入任务描述
  3. 小龙虾AI 会自动匹配并调用「Apify Ultimate Scraper」技能完成任务
  4. 结果即时呈现,支持继续对话优化

相关技能