Start Web Scraping
Initiates a web scraping task using predefined parameters
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.START_SCRAPING])
Stop Web Scraping
Halts an ongoing web scraping task
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.STOP_SCRAPING])
Create New Scraper
Sets up a new web scraper with specified configurations
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.CREATE_SCRAPER])
Delete Scraper
Removes an existing web scraper from the system
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.DELETE_SCRAPER])
Update Scraper Configuration
Modifies the settings of an existing web scraper
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.UPDATE_SCRAPER])
Export Scraped Data
Exports collected data from a scraping task in a specified format
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.EXPORT_DATA])
Schedule Scraping Task
Sets up a recurring or one-time scheduled scraping task
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.SCHEDULE_TASK])
Pause Scheduled Task
Temporarily halts a scheduled scraping task
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.PAUSE_SCHEDULE])
Resume Scheduled Task
Resumes a paused scheduled scraping task
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.RESUME_SCHEDULE])
Add Proxy
Adds a new proxy server to the scraping configuration
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.ADD_PROXY])
Remove Proxy
Removes a proxy server from the scraping configuration
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.REMOVE_PROXY])
Set User Agent
Configures the user agent string for web scraping requests
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.SET_USER_AGENT])
Enable JavaScript Rendering
Activates JavaScript rendering for dynamic web page scraping
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.ENABLE_JS])
Disable JavaScript Rendering
Deactivates JavaScript rendering for web scraping
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.DISABLE_JS])
Set Request Timeout
Configures the timeout duration for web scraping requests
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.SET_TIMEOUT])
Scraping Completed
Triggered when a web scraping task finishes successfully
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.SCRAPING_COMPLETE])
Scraping Failed
Triggered when a web scraping task encounters an error
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.SCRAPING_FAILED])
Data Threshold Reached
Triggered when scraped data volume reaches a specified threshold
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.DATA_THRESHOLD])
New Data Available
Triggered when new data is scraped and ready for processing
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.NEW_DATA])
Proxy Error
Triggered when there's an issue with the proxy server during scraping
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.PROXY_ERROR])
Rate Limit Reached
Triggered when the scraper hits a rate limit on the target website
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.RATE_LIMIT])
Scheduled Task Started
Triggered when a scheduled scraping task begins execution
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.SCHEDULE_START])
Scheduled Task Completed
Triggered when a scheduled scraping task finishes execution
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.SCHEDULE_COMPLETE])
CAPTCHA Detected
Triggered when a CAPTCHA is encountered during web scraping
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.CAPTCHA_DETECTED])
Website Structure Changed
Triggered when the target website's structure differs from expected
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.STRUCTURE_CHANGE])
Low Storage Space
Triggered when available storage for scraped data is running low
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.LOW_STORAGE])
New Pattern Detected
Triggered when a new data pattern is identified in scraped content
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.NEW_PATTERN])
Scraping Progress Update
Triggered at regular intervals to report scraping task progress
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.PROGRESS_UPDATE])
Duplicate Content Detected
Triggered when identical content is scraped multiple times
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.DUPLICATE_CONTENT])
API Usage Limit Warning
Triggered when API usage is approaching the allocated limit
from composio_crewai import ComposioToolSet, Action
tool_set = ComposioToolSet()
tools = tool_set.get_tools(actions=[Action.API_LIMIT_WARNING])