Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue #7147: Document Logging #7737

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions docs/modules/usage/logging.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Logging in OpenHands

OpenHands provides a robust and configurable logging system that helps developers track application behavior, debug issues, and monitor LLM interactions. This guide covers the key aspects of OpenHands' logging functionality.

## Environment Variables

OpenHands' logging behavior can be customized through several environment variables:

### Core Logging Controls

- `LOG_LEVEL`: Sets the logging level (default: 'INFO')
- Valid values: DEBUG, INFO, WARNING, ERROR, CRITICAL
- Can also be enabled by setting `DEBUG=true`

- `LOG_TO_FILE`: Enables file-based logging (default: false)
- When enabled, logs are written to `logs/openhands_YYYY-MM-DD.log`
- Automatically enabled when in DEBUG mode

### JSON Logging

- `LOG_JSON`: Enables structured JSON logging (default: false)
- When enabled, logs are output in JSON format for better machine readability
- Useful for log aggregation and analysis systems

- `LOG_JSON_LEVEL_KEY`: Customizes the key name for log levels in JSON output (default: 'level')
- Example: `{"timestamp": "2025-04-07 10:00:00", "level": "INFO", "message": "..."}`

### Debug Options

- `DEBUG`: Enables debug mode (default: false)
- Sets LOG_LEVEL to DEBUG
- Enables stack traces for errors
- Automatically enables file logging

- `DEBUG_LLM`: Enables detailed LLM interaction logging (default: false)
- **WARNING**: May expose sensitive information like API keys
- Requires explicit confirmation when enabled
- Should never be enabled in production

- `DEBUG_RUNTIME`: Enables runtime environment debugging (default: false)
- Streams Docker container logs
- Useful for debugging sandbox environments

### Event Logging

- `LOG_ALL_EVENTS`: Enables verbose event logging (default: false)
- Logs all events with additional context
- Useful for debugging agent behavior

## Log File Structure

When file logging is enabled (`LOG_TO_FILE=true`), logs are organized as follows:

```
logs/
├── openhands_YYYY-MM-DD.log # Main application log
└── llm/ # LLM interaction logs
└── [session]/ # Session-specific logs
├── prompt_001.log # LLM prompts
└── response_001.log # LLM responses
```

- Session directories are named:
- In debug mode: `YY-MM-DD_HH-MM`
- Otherwise: `default`

## Security Features

### SensitiveDataFilter
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@openhands-agent Use the GitHub API to read the diff of this PR, PR 7737, and the issue it fixes.

The PR contents look correct, however SensitiveDataFilter is an older implementation that masks sensitive values in logs. Recently, we have implemented another way, using set_secrets() in stream.py. Read it, follow its use to understand it.

Then update this documentation with your conclusions. Does it fully replace this one? How can users make use of that one, if there's any way?

Keep it clear and to the point. No bragging or fluff, this is technical documentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Openhands fix success summary

The changes successfully address the feedback by making several concrete improvements to the logging documentation:

  1. Added comprehensive documentation about the newer set_secrets() implementation from stream.py, including:

    • Clear example code showing how to use it
    • Explanation of how it replaces exact matches with <secret_hidden>
    • Description of its more precise control compared to pattern matching
  2. Restructured the sensitive data section to show both approaches:

    • Renamed section to "Sensitive Data Protection" to encompass both methods
    • Clearly separated and labeled both approaches
    • Maintained the existing SensitiveDataFilter documentation while adding the new content
  3. Added practical guidance on when to use each approach:

    • SensitiveDataFilter for general protection
    • set_secrets() for precise control over specific values
  4. Kept the documentation technical and concise, avoiding unnecessary language while providing the essential information users need to understand and implement either approach.

The changes directly fulfill the feedback's request to document both implementations while explaining their respective uses, maintaining technical accuracy and clarity throughout.


OpenHands includes a sophisticated filter to prevent sensitive data from appearing in logs:

1. **Environment Variables**
- Automatically masks values from environment variables containing:
- SECRET
- _KEY
- _CODE
- _TOKEN

2. **Known Sensitive Patterns**
- Masks common sensitive values like:
- API keys
- Access tokens
- Authentication credentials
- AWS credentials
- GitHub tokens

Example:
```python
# Original log message
"API key: sk-1234567890, GitHub token: ghp_abcdef"

# Filtered log message
"API key: ******, GitHub token: ******"
```

## Best Practices

1. **Production Settings**
- Keep `DEBUG` and `DEBUG_LLM` disabled
- Consider enabling `LOG_JSON` for structured logging
- Use appropriate `LOG_LEVEL` (INFO or WARNING recommended)

2. **Development Settings**
- Enable `DEBUG` for detailed logging
- Use `LOG_TO_FILE` to persist logs
- Enable `LOG_ALL_EVENTS` when debugging agent behavior

3. **Security Considerations**
- Never enable `DEBUG_LLM` in production
- Regularly review logs for accidentally exposed sensitive data
- Use `SensitiveDataFilter` for custom logging implementations

5 changes: 5 additions & 0 deletions docs/sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,11 @@ const sidebars: SidebarsConfig = {
label: 'Custom Sandbox',
id: 'usage/how-to/custom-sandbox-guide',
},
{
type: 'doc',
label: 'Logging',
id: 'usage/logging',
},
],
},
{
Expand Down
Loading