feat: add file system search tool for improved codebase navigation

[mikeyobrien]

Implements a new fs_search tool that provides two modes for searching files:

• Name mode: Search for files and directories by glob patterns
• Content mode: Search within file contents using regex patterns with optional context lines

The tool respects standard ignore patterns (.git, node_modules, etc.) by default and includes comprehensive test coverage. This enhancement improves the developer experience by enabling efficient file and content discovery within the codebase.

Issue #, if available:

Description of changes:

Summary

This PR introduces a new fs_search tool that significantly enhances the developer experience by providing efficient file and content discovery capabilities within the codebase. The tool offers two distinct search modes:

• Name mode: Search for files and directories using glob patterns
• Content mode: Search within file contents using regex patterns with configurable context lines

Key Features

🔍 Dual Search Modes

• Name search: Quickly find files matching patterns like *.rs, **/*test*, etc.
• Content search: Find code snippets, function definitions, or any text pattern within files

⚡ Performance Optimized

• Parallel file processing for fast searches
• Respects standard ignore patterns (.git, node_modules, etc.) by default
• Configurable search depth limits

🎯 Developer-Friendly

• Returns results sorted by relevance and modification time
• Provides context lines around matches for better understanding
• Clear, structured output format

Implementation Details

The tool is implemented in Rust and integrates seamlessly with the existing tool infrastructure:

• fs_search.rs: Core implementation with comprehensive error handling
• Updated tool_manager.rs to register the new tool
• Added tool metadata to tool_index.json
• Includes extensive test coverage for both search modes

Testing

The implementation includes comprehensive tests covering:

• Basic file name searches
• Content searches with regex patterns
• Context line functionality
• Edge cases and error handling
• Performance with large file sets

Example Usage

# Search for all test files
fs_search --mode name --pattern "**/*test*.rs"

# Find all occurrences of a function with context
fs_search --mode content --pattern "fn process.*\(" --context 3

# Search for TypeScript files in specific directories
fs_search --mode name --pattern "src/**/*.ts"

Impact

This tool addresses a common developer need for efficient codebase navigation, especially useful for:

• Finding implementation files quickly
• Locating specific code patterns across the codebase
• Understanding code context without opening multiple files
• Improving overall development velocity

Tests:

fs-search-demo.mp4

Error Handling UX

fs-search-error-handling.mp4

fs_search Tool Evaluation Report

This report documents a comprehensive evaluation of the fs_search tool in Amazon Q CLI, testing its various capabilities and modes.

Tool Overview

The fs_search tool provides two primary search modes:

• name: Search for files and directories by name using glob patterns
• content: Search within file contents using regex patterns

Key features include:

• Recursive directory traversal
• File filtering
• Context lines around matches
• File size limits
• Response size limits
• Default ignore patterns
• UTF-8 validation for binary files

Implementation Details

The fs_search tool is implemented in Rust with the following key components:

• FsSearch Enum: The main entry point with two variants:

  • Name(FsSearchName): For file name pattern searches
  • Content(FsSearchContent): For file content searches

• Resource Limits:

  • Maximum response size: 30KB
  • Default maximum file size: 50MB
  • Maximum directory depth: 100
  • Maximum context lines: 10

• Default Ignored Directories:

  • Common directories like .git, node_modules, target, etc.

Test Scenarios

The following test scenarios were executed to evaluate the tool's functionality:

Test 1: Search for Files by Name Pattern

Command:

fs_search(mode="name", pattern="*.md", path=".")

Results:

• Successfully found 21 Markdown files in the repository
• Properly excluded hidden directories like .git
• Output was well-formatted with relative paths

Test 2: Search for Content in a Specific File

Command:

fs_search(mode="content", pattern="Amazon Q", path="README.md")

Results:

• Successfully found 6 matches in the README.md file
• Displayed line numbers with each match
• Properly handled case-insensitive matching

Test 3: Search with Context Lines

Command:

fs_search(mode="content", pattern="fs_search", file_path="*.md", path=".", context_before=2, context_after=2)

Results:

• Successfully found matches across 9 Markdown files
• Properly displayed context lines before and after matches
• Context lines were clearly labeled with [context] prefix
• Match lines were clearly labeled with [match] prefix

Test 4: Search for Specific File Types

Command:

fs_search(mode="name", pattern="*.rs", path="crates")

Results:

• Successfully found 1650 Rust files in the crates directory
• Properly traversed subdirectories
• Handled large result sets appropriately

Test 5: Search for Regex Patterns in Content

Command:

fs_search(mode="content", pattern="fn\\s+main", path="crates/cli/src", context_before=1, context_after=3)

Results:

• Successfully found regex pattern matches
• Properly handled escape sequences in the regex pattern
• Displayed appropriate context lines

Test 6: Search with File Filtering

Command:

fs_search(mode="content", pattern="FsSearch", path="crates/cli/src/cli/chat/tools", file_path="*.rs")

Results:

• Successfully filtered search to only Rust files
• Found matches in 3 different files
• Properly handled the file_path parameter

Test 7: Search with Include Ignored Parameter

Commands:

fs_search(mode="name", pattern="*.git*", path=".", include_ignored=true)
fs_search(mode="name", pattern="*.git*", path=".")

Results:

• With include_ignored=true: Found 4 files/directories (.git, .github, .gitignore, crates/cli/.gitignore)
• Without include_ignored: Found 0 files/directories
• Properly respected the include_ignored parameter

Test 8: Search for Files with a Specific Extension

Command:

fs_search(mode="name", pattern="*.toml", path=".")

Results:

• Successfully found 13 TOML files in the repository
• Properly traversed the directory structure

Test 9: Search for Specific Content in a File

Command:

fs_search(pattern="fs_search", mode="content", path="test_report.md")

Results:

• Successfully found matches in the test_report.md file
• Properly displayed line numbers and context

Observations and Findings

1. Performance: The tool performs efficiently even with large directory structures (1650+ Rust files).

2. Usability:

   • Clear error messages when paths don't exist or patterns are invalid
   • Well-formatted output with line numbers and context
   • Intuitive parameter naming

3. Functionality:

   • Both name and content modes work as expected
   • Context lines feature works correctly
   • File filtering works properly
   • Include_ignored parameter functions correctly

4. Robustness:

   • Handles large result sets appropriately
   • Properly processes regex patterns
   • Respects resource limits

5. Integration:

   • Tool is properly registered in the Amazon Q CLI system
   • Properly handles parameters passed from the CLI

Conclusion

The fs_search tool is a robust and versatile utility for searching files and their contents within the Amazon Q CLI environment. It provides functionality comparable to native tools like find, grep, and ripgrep, but with a unified interface and integration with the CLI's chat capabilities.

The tool successfully handles a wide range of search scenarios, from simple file name pattern matching to complex regex-based content searches with context lines. Its performance is satisfactory even with large directory structures, and it provides clear, well-formatted output.

The implementation includes appropriate resource limits and error handling to ensure stability and prevent resource exhaustion. The default ignore patterns help focus searches on relevant files while allowing users to override this behavior when needed.

Overall, the fs_search tool is a valuable addition to the Amazon Q CLI toolkit, enabling users to efficiently locate files and content within their development environment.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.