linear-coding-agent/prompts/app_spec_tavily_mcp.txt

<project_specification>
  <project_name>ikario - Tavily MCP Integration for Internet Access</project_name>

  <overview>
    This specification adds Tavily search capabilities via MCP (Model Context Protocol) to give Ikario
    internet access for real-time web searches. Tavily provides high-quality search results optimized
    for AI agents, making it ideal for research, fact-checking, and accessing current information.

    This integration adds a new MCP server connection to the existing architecture (alongside the
    ikario-memory MCP server) and exposes Tavily search tools to Ikario during conversations.

    All changes are additive and backward-compatible. Existing functionality remains unchanged.
  </overview>

  <architecture_design>
    <mcp_integration>
      Tavily MCP Server Connection:
      - Uses @modelcontextprotocol/sdk Client to connect to Tavily MCP server
      - Connection can be stdio-based (local MCP server) or HTTP-based (remote)
      - Tavily MCP server provides search tools that are exposed to Claude via Tool Use API
      - Backend routes handle tool execution and return results to Claude
    </mcp_integration>

    <benefits>
      - Real-time internet access for Ikario
      - High-quality search results optimized for LLMs
      - Fact-checking and verification capabilities
      - Access to current events and news
      - Research assistance with cited sources
      - Seamless integration with existing memory tools
    </benefits>
  </architecture_design>

  <technology_stack>
    <mcp_server>
      <name>Tavily MCP Server</name>
      <protocol>Model Context Protocol (MCP)</protocol>
      <connection>stdio or HTTP transport</connection>
      <sdk>@modelcontextprotocol/sdk</sdk>
      <api_key>Tavily API key (from https://tavily.com)</api_key>
    </mcp_server>
    <backend>
      <runtime>Node.js with Express (existing)</runtime>
      <mcp_client>MCP Client for Tavily server connection</mcp_client>
      <tool_executor>Existing toolExecutor service extended with Tavily tools</tool_executor>
    </backend>
    <api_endpoints>
      <tavily_routes>GET/POST /api/tavily/* for Tavily-specific operations</tavily_routes>
      <existing_routes>Existing /api/claude/chat routes support Tavily tools automatically</existing_routes>
    </api_endpoints>
  </technology_stack>

  <prerequisites>
    <environment_setup>
      - Tavily API key obtained from https://tavily.com (free tier available)
      - API key stored in environment variable TAVILY_API_KEY or configuration file
      - MCP SDK already installed (@modelcontextprotocol/sdk exists for ikario-memory)
      - Tavily MCP server installed (npm package or Python package)
    </environment_setup>
    <configuration>
      - Add Tavily MCP server config to server/.claude_settings.json or similar
      - Configure connection parameters (stdio vs HTTP)
      - Set API key securely
    </configuration>
  </prerequisites>

  <core_features>
    <feature_1>
      <title>Tavily MCP Client Setup</title>
      <description>
        Create MCP client connection to Tavily search server. This is similar to the existing
        ikario-memory MCP client but connects to Tavily instead.

        Implementation:
        - Create server/services/tavilyMcpClient.js
        - Initialize MCP client with Tavily server connection
        - Handle connection lifecycle (connect, disconnect, reconnect)
        - Implement health checks and connection status
        - Export client instance and helper functions

        Configuration:
        - Read Tavily API key from environment or config file
        - Configure transport (stdio or HTTP)
        - Set connection timeout and retry logic
        - Log connection status for debugging

        Error Handling:
        - Graceful degradation if Tavily is unavailable
        - Connection retry with exponential backoff
        - Clear error messages for configuration issues
      </description>
      <priority>1</priority>
      <category>backend</category>
      <test_steps>
        1. Verify MCP client can connect to Tavily server on startup
        2. Test connection health check endpoint returns correct status
        3. Verify graceful handling when Tavily API key is missing
        4. Test reconnection logic when connection drops
        5. Verify connection status is logged correctly
        6. Test that server starts even if Tavily is unavailable
      </test_steps>
    </feature_1>

    <feature_2>
      <title>Tavily Tool Configuration</title>
      <description>
        Configure Tavily search tools to be available to Claude during conversations.
        This integrates with the existing tool system (like memory tools).

        Implementation:
        - Create server/config/tavilyTools.js
        - Define tool schemas for Tavily search capabilities
        - Integrate with existing toolExecutor service
        - Add Tavily tools to system prompt alongside memory tools

        Tavily Tools to Expose:
        - tavily_search: General web search with AI-optimized results
          - Parameters: query (string), max_results (number), search_depth (basic/advanced)
          - Returns: Array of search results with title, url, content, score

        - tavily_search_news: News-specific search for current events
          - Parameters: query (string), max_results (number), days (number)
          - Returns: Recent news articles with metadata

        Tool Schema:
        - Follow Claude Tool Use API format
        - Clear descriptions for each tool
        - Well-defined input schemas with validation
        - Proper error handling in tool execution
      </description>
      <priority>1</priority>
      <category>backend</category>
      <test_steps>
        1. Verify Tavily tools are listed in available tools
        2. Test tool schema validation with valid inputs
        3. Test tool schema validation rejects invalid inputs
        4. Verify tools appear in Claude's system prompt
        5. Test that tool descriptions are clear and accurate
        6. Verify tools can be called without errors
      </test_steps>
    </feature_2>

    <feature_3>
      <title>Tavily Tool Executor Integration</title>
      <description>
        Integrate Tavily tools into the existing toolExecutor service so Claude can
        use them during conversations.

        Implementation:
        - Extend server/services/toolExecutor.js to handle Tavily tools
        - Add tool detection for tavily_search and tavily_search_news
        - Implement tool execution logic using Tavily MCP client
        - Format Tavily results for Claude consumption
        - Handle errors and timeouts gracefully

        Tool Execution Flow:
        1. Claude requests tool use (e.g., tavily_search)
        2. toolExecutor detects Tavily tool request
        3. Call Tavily MCP client with tool parameters
        4. Receive and format search results
        5. Return formatted results to Claude
        6. Claude incorporates results into response

        Result Formatting:
        - Convert Tavily results to Claude-friendly format
        - Include source URLs for citation
        - Add relevance scores
        - Truncate content if too long
        - Handle empty results gracefully
      </description>
      <priority>1</priority>
      <category>backend</category>
      <test_steps>
        1. Test tavily_search tool execution with valid query
        2. Verify results are properly formatted
        3. Test tavily_search_news tool execution
        4. Verify error handling when Tavily API fails
        5. Test timeout handling for slow searches
        6. Verify results include proper citations and URLs
        7. Test with empty search results
        8. Test with very long search queries
      </test_steps>
    </feature_3>

    <feature_4>
      <title>System Prompt Enhancement for Internet Access</title>
      <description>
        Update the system prompt to inform Ikario about internet access capabilities.
        This should be added alongside existing memory tools instructions.

        Implementation:
        - Update MEMORY_SYSTEM_PROMPT in server/routes/messages.js and claude.js
        - Add Tavily tools documentation
        - Provide usage guidelines for when to search the internet
        - Include examples of good search queries

        Prompt Addition:
        "## Internet Access via Tavily

        Tu as accès à internet en temps réel via deux outils de recherche :

        1. tavily_search : Recherche web générale optimisée pour l'IA
           - Utilise pour : rechercher des informations actuelles, vérifier des faits,
             trouver des sources fiables
           - Paramètres : query (ta question), max_results (nombre de résultats, défaut: 5),
             search_depth ('basic' ou 'advanced')
           - Retourne : Résultats avec titre, URL, contenu et score de pertinence

        2. tavily_search_news : Recherche d'actualités récentes
           - Utilise pour : événements actuels, nouvelles, actualités
           - Paramètres : query, max_results, days (nombre de jours en arrière, défaut: 7)

        Quand utiliser la recherche internet :
        - Quand l'utilisateur demande des informations récentes ou actuelles
        - Pour vérifier des faits ou données que tu n'es pas sûr de connaître
        - Quand ta base de connaissances est trop ancienne (après janvier 2025)
        - Pour trouver des sources et citations spécifiques
        - Pour des requêtes nécessitant des données en temps réel

        N'utilise PAS la recherche pour :
        - Des questions sur ta propre identité ou capacités
        - Des concepts généraux que tu connais déjà bien
        - Des questions purement créatives ou d'opinion

        Utilise ces outils de façon autonome selon les besoins de la conversation.
        Cite toujours tes sources quand tu utilises des informations de Tavily."
      </description>
      <priority>2</priority>
      <category>backend</category>
      <test_steps>
        1. Verify system prompt includes Tavily instructions
        2. Test that Claude understands when to use Tavily search
        3. Verify Claude cites sources from Tavily results
        4. Test that Claude uses appropriate search queries
        5. Verify Claude chooses between tavily_search and tavily_search_news correctly
        6. Test that Claude doesn't over-use search for simple questions
      </test_steps>
    </feature_4>

    <feature_5>
      <title>Tavily Status API Endpoint</title>
      <description>
        Create API endpoint to check Tavily MCP connection status and search capabilities.
        Similar to /api/memory/status endpoint.

        Implementation:
        - Create GET /api/tavily/status endpoint
        - Return connection status, available tools, and configuration
        - Create GET /api/tavily/health endpoint for health checks
        - Add Tavily status to existing /api/memory/stats (rename to /api/tools/stats)

        Response Format:
        {
          "success": true,
          "data": {
            "connected": true,
            "message": "Tavily MCP server is connected",
            "tools": ["tavily_search", "tavily_search_news"],
            "apiKeyConfigured": true,
            "transport": "stdio"
          }
        }
      </description>
      <priority>2</priority>
      <category>backend</category>
      <test_steps>
        1. Test GET /api/tavily/status returns correct status
        2. Verify status shows "connected" when Tavily is available
        3. Verify status shows "disconnected" when Tavily is unavailable
        4. Test health endpoint returns proper status code
        5. Verify tools list is accurate
        6. Test with missing API key shows proper error
      </test_steps>
    </feature_5>

    <feature_6>
      <title>Frontend UI Indicator for Internet Access</title>
      <description>
        Add visual indicator in the UI to show when Ikario has internet access via Tavily.
        This can be displayed alongside the existing memory status indicator.

        Implementation:
        - Add Tavily status indicator in header or sidebar
        - Show online/offline status for Tavily connection
        - Optional: Show when Tavily is being used during a conversation
        - Optional: Add tooltip explaining internet access capabilities

        Visual Design:
        - Globe or wifi icon to represent internet access
        - Green when connected, gray when disconnected
        - Subtle animation when search is in progress
        - Tooltip: "Internet access via Tavily" or similar

        Integration:
        - Use existing useMemory hook pattern or create useTavily hook
        - Poll /api/tavily/status periodically (every 60s)
        - Update status in real-time during searches
      </description>
      <priority>3</priority>
      <category>frontend</category>
      <test_steps>
        1. Verify internet access indicator appears in UI
        2. Test status updates when Tavily connects/disconnects
        3. Verify tooltip shows correct information
        4. Test that indicator shows activity during searches
        5. Verify status polling doesn't impact performance
        6. Test with Tavily disabled shows offline status
      </test_steps>
    </feature_6>

    <feature_7>
      <title>Manual Search UI (Optional Enhancement)</title>
      <description>
        Optional: Add manual search interface to allow users to trigger Tavily searches directly,
        similar to the memory search panel.

        Implementation:
        - Add "Internet Search" panel in sidebar (alongside Memory panel)
        - Search input for manual Tavily queries
        - Display search results with title, snippet, URL
        - Click to insert results into conversation
        - Filter by search type (general vs news)

        This is OPTIONAL and lower priority. The primary use case is autonomous search by Claude.
      </description>
      <priority>4</priority>
      <category>frontend</category>
      <test_steps>
        1. Verify search panel appears in sidebar
        2. Test manual search returns results
        3. Verify results display properly with links
        4. Test inserting results into conversation
        5. Test news search filter works correctly
        6. Verify search history is saved (optional)
      </test_steps>
    </feature_7>

    <feature_8>
      <title>Configuration and Settings</title>
      <description>
        Add Tavily configuration options to settings and environment.

        Implementation:
        - Add TAVILY_API_KEY to environment variables
        - Add Tavily settings to .claude_settings.json or similar config file
        - Create server/config/tavilyConfig.js for configuration management
        - Document configuration options in README

        Configuration Options:
        - API key
        - Max results per search (default: 5)
        - Search depth (basic/advanced)
        - Timeout duration
        - Enable/disable Tavily globally
        - Rate limiting settings

        Security:
        - API key should NOT be exposed to frontend
        - Use environment variable or secure config file
        - Validate API key on startup
        - Log warnings if API key is missing
      </description>
      <priority>2</priority>
      <category>backend</category>
      <test_steps>
        1. Verify API key is read from environment variable
        2. Test fallback to config file if env var not set
        3. Verify API key validation on startup
        4. Test configuration options are applied correctly
        5. Verify API key is never exposed in API responses
        6. Test enabling/disabling Tavily via config
      </test_steps>
    </feature_8>

    <feature_9>
      <title>Error Handling and Rate Limiting</title>
      <description>
        Implement robust error handling and rate limiting for Tavily API calls.

        Implementation:
        - Detect and handle Tavily API errors (rate limits, invalid API key, etc.)
        - Implement client-side rate limiting to avoid hitting Tavily limits
        - Cache search results for duplicate queries (optional)
        - Provide clear error messages to Claude when searches fail

        Error Types:
        - 401: Invalid API key
        - 429: Rate limit exceeded
        - 500: Tavily server error
        - Timeout: Search took too long
        - Network: Connection failed

        Rate Limiting:
        - Track searches per minute/hour
        - Queue requests if limit reached
        - Return cached results for duplicate queries within 5 minutes
        - Log rate limit warnings
      </description>
      <priority>2</priority>
      <category>backend</category>
      <test_steps>
        1. Test error handling for invalid API key
        2. Verify rate limit detection and handling
        3. Test timeout handling for slow searches
        4. Verify error messages are clear to Claude
        5. Test rate limiting prevents API abuse
        6. Verify caching works for duplicate queries
      </test_steps>
    </feature_9>

    <feature_10>
      <title>Documentation and README Updates</title>
      <description>
        Update project documentation to explain Tavily integration.

        Implementation:
        - Update main README.md with Tavily setup instructions
        - Add TAVILY_SETUP.md with detailed configuration guide
        - Document API endpoints in README
        - Add examples of using Tavily with Ikario
        - Document troubleshooting steps

        Documentation Sections:
        - Prerequisites (Tavily API key)
        - Installation steps
        - Configuration options
        - Testing Tavily connection
        - Example conversations using internet search
        - Troubleshooting common issues
        - API reference for Tavily endpoints
      </description>
      <priority>3</priority>
      <category>documentation</category>
      <test_steps>
        1. Verify README has Tavily setup section
        2. Test that setup instructions are clear and complete
        3. Verify all configuration options are documented
        4. Test examples work as described
        5. Verify troubleshooting section covers common issues
      </test_steps>
    </feature_10>
  </core_features>

  <implementation_notes>
    <order>
      Recommended implementation order:
      1. Feature 1 (MCP Client Setup) - Foundation
      2. Feature 2 (Tool Configuration) - Core functionality
      3. Feature 3 (Tool Executor Integration) - Core functionality
      4. Feature 8 (Configuration) - Required for testing
      5. Feature 4 (System Prompt) - Makes tools accessible to Claude
      6. Feature 9 (Error Handling) - Production readiness
      7. Feature 5 (Status API) - Monitoring
      8. Feature 10 (Documentation) - User onboarding
      9. Feature 6 (UI Indicator) - Nice to have
      10. Feature 7 (Manual Search UI) - Optional enhancement
    </order>

    <testing>
      After implementing features 1-5, you should be able to:
      - Ask Ikario: "Quelle est l'actualité aujourd'hui ?"
      - Ask Ikario: "Recherche des informations sur [topic actuel]"
      - Ask Ikario: "Vérifie cette information : [claim]"

      Ikario should autonomously use Tavily search and cite sources.
    </testing>

    <compatibility>
      - This specification is fully compatible with existing ikario-memory MCP integration
      - Ikario will have both memory tools AND internet search tools
      - Tools can be used together in the same conversation
      - No conflicts expected between tool systems
    </compatibility>
  </implementation_notes>

  <safety_requirements>
    <critical>
      - DO NOT expose Tavily API key to frontend or in API responses
      - DO NOT modify existing MCP memory integration
      - DO NOT break existing conversation functionality
      - Tavily should gracefully degrade if unavailable (don't crash the app)
      - Implement proper rate limiting to avoid API abuse
      - Validate all user inputs before passing to Tavily
      - Sanitize search results before displaying (XSS prevention)
      - Log all Tavily API calls for monitoring and debugging
    </critical>
  </safety_requirements>

  <success_metrics>
    - Ikario can successfully perform internet searches when asked
    - Search results are relevant and well-formatted
    - Sources are properly cited
    - Tavily integration doesn't slow down conversations
    - Error handling is robust and user-friendly
    - Configuration is straightforward
    - Documentation is clear and complete
  </success_metrics>
</project_specification>