---

name: scrask-bot version: 3.0.0 description: > When the user sends a screenshot via Telegram, parse it using Gemini (fast, default) with automatic Claude fallback when confidence is low. Saves results to Google Calendar (events) or Google Tasks (reminders and tasks). Saves silently if confidence is high; asks for confirmation if uncertain. author: your-name metadata: openclaw: emoji: "🦞" primaryEnv: GEMINI_API_KEY requires: env: - GOOGLE_CREDENTIALS - GEMINI_API_KEY # required for auto and gemini modes # - ANTHROPIC_API_KEY # optional — enables Claude fallback in auto mode bins: - python3 config: vision_provider: type: string description: > Vision model provider. 'auto' = Gemini first, Claude fallback if any item confidence < fallback_threshold. 'gemini' = Gemini only. 'claude' = Claude only. default: auto fallback_threshold: type: number description: "Confidence floor for auto mode. If any item is below this, Claude reruns the parse." default: 0.60 timezone: type: string description: "User's IANA timezone. Used when none is detected in the screenshot." default: "UTC" confidence_threshold: type: number description: "0.0–1.0. Items below this score ask for confirmation. Items above save silently." default: 0.75 reminder_minutes_before: type: integer description: "Popup reminder lead time in minutes for Google Calendar events." default: 30

Scrask Bot

Overview

This skill activates when the user sends a screenshot via Telegram. It uses vision AI to extract actionable information from the image, then:

• High confidence (≥ 0.75): Saves immediately and replies with a brief confirmation.
• Low confidence (< 0.75): Shows a structured preview in Telegram and asks for confirmation before saving.

Provider behaviour (auto mode, default):

Set vision_provider to "claude" or "gemini" to lock a specific provider.

Output destinations (AI-decided by content type):

---

Trigger Conditions

Activate when:

1. The user sends a message in Telegram that contains an image attachment
2. The image appears to be a screenshot — not a photo of a person, place, or physical object
3. No other skill has already claimed the image

Do not activate for:

• Photos of people, places, food, scenery
• Screenshots of code, errors, or UI bugs (leave for other skills)
• Images the user explicitly asks to edit, describe, or analyze for another purpose

---

Step-by-Step Instructions

Step 1: Acknowledge Immediately

Reply in Telegram right away so the user knows the skill is working:

"📸 Got it — analyzing your screenshot..."

Do not make the user wait silently.

---

Step 2: Run the Parser

python3 ~/.openclaw/skills/scrask-bot/scripts/scrask_bot.py \
  --image-path "<path-to-temp-image>" \
  --provider "$CONFIG_VISION_PROVIDER" \
  --timezone "$CONFIG_TIMEZONE" \
  --google-credentials "$GOOGLE_CREDENTIALS"

The script auto-resolves the API key from ANTHROPIC_API_KEY or GEMINI_API_KEY
depending on the provider — no need to pass it explicitly.

The script returns a JSON object with:

• success — whether parsing worked
• no_actionable_content — true if nothing found
• results[] — one entry per detected item, each with confidence, type, destination, needs_confirmation, action_taken
• telegram_reply — the pre-formatted message to send back to the user

---

Step 3: Handle the Output

If no_actionable_content is true: Reply: "🤷 I couldn't find any event, reminder, or task info in that screenshot. Could you describe what you'd like to add?"

If success is true: Send the telegram_reply value directly back to the user in Telegram. The script has already:

• Saved high-confidence items silently
• Formatted confirmation prompts for low-confidence items

Do not rephrase or reformat the telegram_reply — send it as-is.

---

Step 4: Handle Confirmation Responses

If the script returned items with needs_confirmation: true, wait for the user's reply.

"yes" or "save" or "add": Re-run the script for that specific item with confirmed=true, or use the calendar_create / tasks_create tools directly with the extracted fields.

"edit": Ask what to change, update the relevant field, then save.

"skip" or "no": Reply: "Got it, skipped ✓"

---

Step 5: Confirm Saves

For items saved silently (high confidence), the telegram_reply from the script already contains the confirmation message. Examples of what the user will see:

• 📅 Added to Calendar: **Team Standup** — 2026-03-01 at 09:00
• 🔔 Added to Tasks: **Pay electricity bill** (due 2026-02-28)
• ✅ Added to Tasks: **Review PR for Arjun**

---

Edge Cases

---

Configuration

{
  "skills": {
    "entries": {
      "scrask-bot": {
        "enabled": true,
        "env": {
          "GEMINI_API_KEY": "AIza-your-gemini-key",
          "ANTHROPIC_API_KEY": "sk-ant-your-key-here",
          "GOOGLE_CREDENTIALS": "/home/user/.openclaw/google-creds.json"
        },
        "config": {
          "vision_provider": "auto",
          "fallback_threshold": 0.60,
          "timezone": "Asia/Kolkata",
          "confidence_threshold": 0.75,
          "reminder_minutes_before": 30
        }
      }
    }
  }
}

---

Permissions Required

• image:read — to access the screenshot from Telegram
• network:outbound — to call Anthropic API and Google APIs
• telegram:reply — to send confirmation messages back to the user