diff --git a/README.md b/README.md index f34c3eb..e992592 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,394 @@ -# loglens-cli +# LogLens CLI -A CLI tool that parses, analyzes, and provides intelligent summaries of log files \ No newline at end of file +A powerful CLI tool for parsing, analyzing, and providing intelligent summaries of log files. LogLens detects error patterns, highlights anomalies, and outputs human-readable reports with severity classifications and suggested fixes. + +## Features + +- **Multi-format Log Parsing**: Support for JSON, syslog (RFC 3164/5424), and Apache/Nginx logs +- **Error Pattern Detection**: Built-in library of 20+ patterns for common errors (exceptions, stack traces, connection failures) +- **Severity Classification**: Automatic classification into Critical, Error, Warning, Info, and Debug levels +- **Real-time Analysis**: Pipe support for live log streaming with `--follow` mode +- **Human-readable Reports**: Beautiful table and text outputs with color coding +- **Unix Tool Integration**: Works seamlessly with grep, sed, and other command-line tools + +## Installation + +### From Source + +```bash +git clone https://7000pct.gitea.bloupla.net/7000pctAUTO/loglens-cli.git +cd loglens-cli +pip install -e . +``` + +### Using pip + +```bash +pip install loglens-cli +``` + +### Development Installation + +```bash +pip install -e ".[dev]" +``` + +## Quick Start + +### Analyze a log file + +```bash +# Auto-detect format and analyze +loglens analyze app.log + +# Specify format explicitly +loglens analyze app.log --format json +loglens analyze app.log --format syslog +loglens analyze app.log --format apache + +# Show detailed output +loglens analyze app.log --verbose +``` + +### Watch logs in real-time + +```bash +# Follow log file updates +loglens watch /var/log/app.log + +# Follow with auto-refresh +loglens watch /var/log/app.log --interval 2 +``` + +### Generate summary reports + +```bash +# Generate summary report +loglens report app.log + +# Generate report with severity breakdown +loglens report app.log --severity + +# JSON output for programmatic use +loglens report app.log --json +``` + +### Check detected patterns + +```bash +# List all available patterns +loglens patterns + +# Search for specific patterns +loglens patterns --search "connection" + +# Show pattern details +loglens patterns --show-exceptions +``` + +### Pipe support + +```bash +# Process logs from stdin +tail -f app.log | loglens analyze - + +# Chain with grep +grep "ERROR" app.log | loglens analyze - + +# Complex pipeline +cat logs/*.log | loglens analyze - +``` + +## Usage + +### Commands + +#### analyze + +Parse and analyze log files. + +```bash +loglens analyze [OPTIONS] FILE +``` + +Options: +- `--format, -f TEXT`: Log format (auto, json, syslog, apache) +- `--output, -o TEXT`: Output format (table, json, text) +- `--verbose, -v`: Show detailed analysis +- `--json`: Output as JSON + +#### watch + +Monitor a log file in real-time. + +```bash +loglens watch [OPTIONS] FILE +``` + +Options: +- `--interval SECONDS`: Refresh interval (default: 1.0) +- `--format TEXT`: Log format (auto, json, syslog, apache) + +#### report + +Generate a summary report. + +```bash +loglens report [OPTIONS] FILE +``` + +Options: +- `--severity`: Include severity breakdown +- `--output TEXT`: Output format (table, json, text) +- `--json`: Output as JSON + +#### patterns + +Show available error patterns. + +```bash +loglens patterns [OPTIONS] +``` + +Options: +- `--search TEXT`: Search patterns by name +- `--show-exceptions`: Show exception patterns +- `--show-http`: Show HTTP error patterns +- `--json`: Output as JSON + +#### info + +Show tool information and configuration. + +```bash +loglens info +``` + +### Examples + +#### Analyze JSON logs + +```bash +$ loglens analyze application.json --format json +╭──────────────────────────────────────────────────────────────────────────────╮ +│ Log Analysis Summary │ +├──────────────────────────────────────────────────────────────────────────────┤ +│ Total Lines: 150 │ +│ Parsed: 148 (98.7%) │ +│ Errors Found: 12 │ +│ Severity: 2 Critical, 5 Error, 3 Warning, 2 Info │ +╰──────────────────────────────────────────────────────────────────────────────╯ + +Top Errors: + [CRITICAL] Database connection failure: 3 occurrences + [ERROR] NullPointerException: 2 occurrences + [WARNING] Deprecated API usage: 2 occurrences +``` + +#### Generate report with severity + +```bash +$ loglens report server.log --severity --output table +Severity Breakdown: + CRITICAL: ████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 2 + ERROR: ██████████████████████████████ 15 + WARNING: ██████████████████████████░░░░ 10 + INFO: ██████████████████████████████ 18 + DEBUG: ██████████████░░░░░░░░░░░░░░░░░ 6 +``` + +#### Real-time monitoring + +```bash +$ loglens watch /var/log/app.log --interval 2 +Watching: /var/log/app.log +Press Ctrl+C to stop... + +Last 5 lines: +[2024-01-15 10:30:45] ERROR: Database connection failed +[2024-01-15 10:30:46] WARNING: High memory usage detected +[2024-01-15 10:30:47] INFO: Connection pool status: healthy +[2024-01-15 10:30:48] ERROR: Request timeout: /api/users +[2024-01-15 10:30:49] CRITICAL: Service unresponsive +``` + +## Configuration + +### Environment Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `LOGLEVEL` | INFO | Default log level for output | +| `LOGLENSMASK` | false | Enable PII masking in logs | +| `LOGLENSCOLOR` | auto | Color output: auto, always, never | + +### Configuration File + +Create `~/.loglens/config.yaml`: + +```yaml +default_format: auto +output_format: table +color: auto +severity_rules: + critical: + - "Out of memory" + - "Service unresponsive" + error: + - "ERROR" + - "Exception" + warning: + - "WARNING" + - "Deprecated" +``` + +### Custom Patterns + +Add custom patterns in `~/.loglens/patterns.yaml`: + +```yaml +patterns: + - name: "Custom Error" + regex: "CUSTOM_ERROR:.*" + severity: error + suggestion: "Check custom error documentation" +``` + +## Supported Log Formats + +### JSON Logs + +```json +{ + "timestamp": "2024-01-15T10:30:00Z", + "level": "ERROR", + "message": "Connection refused to database", + "logger": "main", + "service": "api" +} +``` + +### Syslog (RFC 3164/5424) + +``` +Jan 15 10:30:00 server-01 app[1234]: ERROR: Connection refused +Jan 15 10:30:01 server-01 systemd[1]: Started Application Service +``` + +### Apache/Nginx + +``` +192.168.1.1 - - [15/Jan/2024:10:30:00 +0000] "GET /api/users HTTP/1.1" 200 1234 +192.168.1.2 - - [15/Jan/2024:10:30:01 +0000] "POST /api/login HTTP/1.1" 401 567 +``` + +## Development + +### Setup + +```bash +git clone https://7000pct.gitea.bloupla.net/7000pctAUTO/loglens-cli.git +cd loglens-cli +pip install -e ".[dev]" +``` + +### Running Tests + +```bash +# Run all tests +pytest tests/ -v + +# Run with coverage +pytest tests/ --cov=loglens --cov-report=html + +# Run specific test suite +pytest tests/unit/ -v +pytest tests/integration/ -v +``` + +### Linting + +```bash +# Ruff linting +ruff check loglens/ + +# Type checking +mypy loglens/ --ignore-missing-imports +``` + +### Code Formatting + +```bash +# Black formatting +black loglens/ tests/ + +# Import sorting +isort loglens/ tests/ +``` + +## Project Structure + +``` +loglens-cli/ +├── loglens/ +│ ├── __init__.py +│ ├── __main__.py +│ ├── cli/ +│ │ ├── __init__.py +│ │ ├── main.py +│ │ └── commands.py +│ ├── parsers/ +│ │ ├── __init__.py +│ │ ├── base.py +│ │ ├── json_parser.py +│ │ ├── syslog_parser.py +│ │ ├── apache_parser.py +│ │ └── factory.py +│ ├── analyzers/ +│ │ ├── __init__.py +│ │ ├── patterns.py +│ │ ├── severity.py +│ │ └── analyzer.py +│ ├── formatters/ +│ │ ├── __init__.py +│ │ ├── base.py +│ │ ├── table_formatter.py +│ │ ├── json_formatter.py +│ │ └── text_formatter.py +│ └── config.yaml +├── tests/ +│ ├── __init__.py +│ ├── conftest.py +│ ├── unit/ +│ │ ├── __init__.py +│ │ ├── test_parsers.py +│ │ ├── test_analyzer.py +│ │ └── test_cli.py +│ ├── integration/ +│ │ ├── __init__.py +│ │ └── test_end_to_end.py +│ └── fixtures/ +├── pyproject.toml +├── README.md +├── LICENSE +└── .env.example +``` + +## Error Patterns + +LogLens includes patterns for: + +- **Exceptions**: Python, Java, Node.js stack traces +- **HTTP Errors**: 5xx server errors, connection failures +- **Database Errors**: Connection refused, query timeouts +- **System Errors**: Out of memory, disk space, permissions +- **Security Issues**: Authentication failures, access denied +- **Performance**: High latency, slow queries, timeouts + +## Contributing + +Contributions are welcome! Please read the contributing guidelines before submitting PRs. + +## License + +MIT License - see LICENSE file for details.