Sherlock CLI Documentation

Welcome to the Sherlock CLI documentation. Sherlock is a persona-driven testing tool that simulates real user behavior to detect signup flow issues before they impact your users.

Installation

# Install Sherlock CLI using pip
pip install sherlock-personas

# Or install from source
git clone https://github.com/your-org/sherlock.git
cd sherlock
pip install -e .

Quick Start

Run your first Sherlock test in 60 seconds:

# Basic signup flow test
sherlock run

# Test with a specific persona
sherlock run --persona confused_first_time_user

# Test on a specific device
sherlock run --device iphone13

# Test with network throttling
sherlock run --network 3g

Commands

sherlock run

Execute a persona-driven test run.

Usage:

sherlock run [options]

Options:

• --persona <name> - Select a user persona (default: standard_user)
• --device <device> - Specify target device (default: desktop)
• --network <speed> - Set network conditions (default: 4g)
• --url <url> - Target URL to test (default: from config)
• --headless - Run in headless mode (default: true)
• --debug - Enable debug mode with verbose logging
• --record - Record session video and screenshots
• --slack <webhook> - Slack webhook URL for alerts

Example:

sherlock run \
  --persona power_user \
  --device iphone13 \
  --network 3g \
  --url https://app.example.com/signup

sherlock init

Initialize a new Sherlock configuration in your project.

Usage:

sherlock init

This creates a sherlock.config.json file with default settings.

sherlock list

List available personas, devices, and network profiles.

Usage:

# List all available options
sherlock list

# List personas only
sherlock list --personas

# List devices only
sherlock list --devices

# List network profiles
sherlock list --networks

sherlock config

Manage Sherlock configuration.

Usage:

# View current configuration
sherlock config view

# Set a configuration value
sherlock config set <key> <value>

# Reset to defaults
sherlock config reset

Personas

Sherlock includes built-in personas that simulate different user behaviors:

Standard User

ID: standard_user

• Familiar with web applications
• Normal reading speed
• Minimal mistakes
• Good network literacy

Confused First-Time User

ID: confused_first_time_user

• First time using the app
• Slower interaction speed
• May click wrong buttons
• Hesitates before submitting forms
• Re-reads instructions

Power User

ID: power_user

• Very fast interactions
• Uses keyboard shortcuts
• Minimal mouse usage
• Expects instant responses

Mobile-First User

ID: mobile_first_user

• Primarily uses mobile devices
• Thumb-based navigation
• Expects mobile-optimized UI
• Impatient with slow loading

Accessibility-Focused User

ID: accessibility_focused

• Uses screen readers
• Keyboard-only navigation
• Needs high contrast
• Sensitive to motion

Devices

Test across different devices and viewports:

Network Conditions

Simulate various network speeds:

Configuration

Create a sherlock.config.json in your project root:

{
  "baseUrl": "https://app.example.com",
  "timeout": 30000,
  "retries": 3,
  "defaultPersona": "standard_user",
  "defaultDevice": "desktop",
  "defaultNetwork": "4g",
  "slack": {
    "webhookUrl": "https://hooks.slack.com/services/YOUR/WEBHOOK/URL",
    "channel": "#eng-alerts",
    "mentionOnCritical": ["@engineering"]
  },
  "alerts": {
    "enabled": true,
    "severityLevels": ["P0", "P1", "P2", "P3"],
    "minSeverity": "P2"
  },
  "recording": {
    "enabled": true,
    "videoFormat": "mp4",
    "screenshotOnFailure": true,
    "retainSuccessful": false
  },
  "diagnosis": {
    "enabled": true,
    "aiProvider": "bedrock",
    "model": "anthropic.claude-3-sonnet"
  }
}

Alert Integration

Slack Integration

Configure Slack alerts to get notified instantly when issues are detected:

sherlock run --slack https://hooks.slack.com/services/YOUR/WEBHOOK/URL

Alert Format:

🚨 Signup Flow Blocked [P0]

Issue: Submit button unresponsive on iPhone 13 with 3G network
Root Cause: JavaScript event listener not attached due to slow bundle load
Impact: 100% of signup attempts failing on slower connections
Device: iPhone 13
Network: 3G
Persona: confused_first_time_user

View Full Diagnosis →

Custom Webhooks

Send alerts to any webhook endpoint:

{
  "webhooks": [
    {
      "url": "https://api.example.com/alerts",
      "headers": {
        "Authorization": "Bearer YOUR_TOKEN"
      },
      "severityFilter": ["P0", "P1"]
    }
  ]
}

Examples

Example 1: Basic Signup Flow Test

sherlock run \
  --url https://app.example.com/signup \
  --persona standard_user

Example 2: Mobile Performance Test

sherlock run \
  --persona mobile_first_user \
  --device iphone13 \
  --network 3g \
  --record

Example 3: Accessibility Audit

sherlock run \
  --persona accessibility_focused \
  --device desktop \
  --debug

Example 4: Power User Flow

sherlock run \
  --persona power_user \
  --device laptop \
  --network 5g

Example 5: Comprehensive Test Suite

# Test multiple personas in sequence
for persona in standard_user confused_first_time_user power_user; do
  sherlock run \
    --persona $persona \
    --device iphone13 \
    --network 3g \
    --record
done

Example 6: CI/CD Integration

# .github/workflows/sherlock.yml
name: Sherlock E2E Tests

on:
  push:
    branches: [main, staging]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install Sherlock
        run: pip install sherlock-personas

      - name: Run Signup Tests
        run: |
          sherlock run \
            --persona confused_first_time_user \
            --device iphone13 \
            --network 3g \
            --slack ${{ secrets.SLACK_WEBHOOK }}

Troubleshooting

Common Issues

Issue: "Command not found: sherlock"

# Solution: Install using pip and ensure it's in your PATH
pip install sherlock-personas

# Verify installation
sherlock --version

Issue: "Timeout waiting for element"

# Solution: Increase timeout in config
sherlock config set timeout 60000

Issue: "Network throttling not working"

# Solution: Ensure you have proper permissions
# Run with sudo if needed (not recommended for production)

Support

• Documentation: https://docs.sherlock.dev
• GitHub Issues: https://github.com/sherlock/cli/issues
• Discord Community: https://discord.gg/sherlock
• Email: support@sherlock.dev

License

MIT License - see LICENSE file for details

---

Built with ❤️ by the Sherlock team