The agentic_mouse_action method performs mouse actions at specific screen coordinates captured during a recording. If the action fails at those coordinates (e.g., due to page layout changes), it falls back to the Operator agent using a natural language query.
This method is primarily used by agents generated through Imitation Learning, where click coordinates are captured during recording. For most manual automations, prefer agentic_selector() which targets elements by CSS selectors. Method Signature
async def agentic_mouse_action(
self,
*,
action: AgenticMouseAction,
recorded_click: RecordedClick,
fallback_operator_query: str,
resize_window: bool = True,
timeout: int | None = 60,
) -> None
Parameters
action
AgenticMouseAction
required
The mouse action to perform. Supported actions:{"type": "click"} # Left click
{"type": "right_click"} # Right click
{"type": "double_click"} # Double click
{"type": "fill", "text": "hello", "press_enter": False} # Type text at the location
{"type": "scroll", "horizontal": 0, "vertical": 300} # Scroll by pixel offset
The coordinates and viewport dimensions from when the action was originally recorded.{
"x": 450, # X coordinate in pixels
"y": 320, # Y coordinate in pixels
"viewport": {
"width": 1920, # Viewport width when recorded
"height": 1080 # Viewport height when recorded
}
}
Natural language instruction for the Operator agent to use if the coordinate-based action fails.fallback_operator_query="click the Submit button at the bottom of the form"
Whether to resize the browser window to match the recorded viewport dimensions before performing the action. This improves accuracy when the page layout is responsive.
Maximum time in seconds to wait for the operation to complete.
Return Value
This method returns None and completes when the action has been performed successfully.
Example
import asyncio
from narada import Narada
async def main():
async with Narada() as narada:
window = await narada.open_and_initialize_browser_window()
await window.go_to_url(url="https://www.google.com")
# Click at recorded coordinates, with Operator fallback
await window.agentic_mouse_action(
action={"type": "click"},
recorded_click={
"x": 600,
"y": 450,
"viewport": {"width": 1440, "height": 900}
},
fallback_operator_query='click the "Google Search" button'
)
# Type into a field at recorded coordinates
await window.agentic_mouse_action(
action={"type": "fill", "text": "Narada AI", "press_enter": True},
recorded_click={
"x": 600,
"y": 350,
"viewport": {"width": 1440, "height": 900}
},
fallback_operator_query='type "Narada AI" in the search box and press Enter'
)
if __name__ == "__main__":
asyncio.run(main())
When to Use This vs agentic_selector
| agentic_mouse_action | agentic_selector |
|---|
| Targets by | Screen coordinates | CSS selectors / element attributes |
| Best for | Replaying recorded actions | Programmatic element targeting |
| Resilience | Depends on page layout stability | Depends on element attribute stability |
| Typical use | Generated by Imitation Learning | Written manually or in custom agents |
Both methods fall back to the Operator agent when their primary targeting strategy fails, ensuring robust automation even when pages change.
