Initial commit: LogLens CLI with parsers, analyzers, and CLI

2026-02-02 08:00:27 +00:00
parent 0a2564b8ee
commit 1e569d55ed
1 changed files with 393 additions and 2 deletions
--- a/README.md
+++ b/README.md
@@ -1,3 +1,394 @@
-# loglens-cli
+# LogLens CLI

-A CLI tool that parses, analyzes, and provides intelligent summaries of log files
+A powerful CLI tool for parsing, analyzing, and providing intelligent summaries of log files. LogLens detects error patterns, highlights anomalies, and outputs human-readable reports with severity classifications and suggested fixes.
+
+## Features
+
+- **Multi-format Log Parsing**: Support for JSON, syslog (RFC 3164/5424), and Apache/Nginx logs
+- **Error Pattern Detection**: Built-in library of 20+ patterns for common errors (exceptions, stack traces, connection failures)
+- **Severity Classification**: Automatic classification into Critical, Error, Warning, Info, and Debug levels
+- **Real-time Analysis**: Pipe support for live log streaming with `--follow` mode
+- **Human-readable Reports**: Beautiful table and text outputs with color coding
+- **Unix Tool Integration**: Works seamlessly with grep, sed, and other command-line tools
+
+## Installation
+
+### From Source
+
+```bash
+git clone https://7000pct.gitea.bloupla.net/7000pctAUTO/loglens-cli.git
+cd loglens-cli
+pip install -e .
+```
+
+### Using pip
+
+```bash
+pip install loglens-cli
+```
+
+### Development Installation
+
+```bash
+pip install -e ".[dev]"
+```
+
+## Quick Start
+
+### Analyze a log file
+
+```bash
+# Auto-detect format and analyze
+loglens analyze app.log
+
+# Specify format explicitly
+loglens analyze app.log --format json
+loglens analyze app.log --format syslog
+loglens analyze app.log --format apache
+
+# Show detailed output
+loglens analyze app.log --verbose
+```
+
+### Watch logs in real-time
+
+```bash
+# Follow log file updates
+loglens watch /var/log/app.log
+
+# Follow with auto-refresh
+loglens watch /var/log/app.log --interval 2
+```
+
+### Generate summary reports
+
+```bash
+# Generate summary report
+loglens report app.log
+
+# Generate report with severity breakdown
+loglens report app.log --severity
+
+# JSON output for programmatic use
+loglens report app.log --json
+```
+
+### Check detected patterns
+
+```bash
+# List all available patterns
+loglens patterns
+
+# Search for specific patterns
+loglens patterns --search "connection"
+
+# Show pattern details
+loglens patterns --show-exceptions
+```
+
+### Pipe support
+
+```bash
+# Process logs from stdin
+tail -f app.log | loglens analyze -
+
+# Chain with grep
+grep "ERROR" app.log | loglens analyze -
+
+# Complex pipeline
+cat logs/*.log | loglens analyze -
+```
+
+## Usage
+
+### Commands
+
+#### analyze
+
+Parse and analyze log files.
+
+```bash
+loglens analyze [OPTIONS] FILE
+```
+
+Options:
+- `--format, -f TEXT`: Log format (auto, json, syslog, apache)
+- `--output, -o TEXT`: Output format (table, json, text)
+- `--verbose, -v`: Show detailed analysis
+- `--json`: Output as JSON
+
+#### watch
+
+Monitor a log file in real-time.
+
+```bash
+loglens watch [OPTIONS] FILE
+```
+
+Options:
+- `--interval SECONDS`: Refresh interval (default: 1.0)
+- `--format TEXT`: Log format (auto, json, syslog, apache)
+
+#### report
+
+Generate a summary report.
+
+```bash
+loglens report [OPTIONS] FILE
+```
+
+Options:
+- `--severity`: Include severity breakdown
+- `--output TEXT`: Output format (table, json, text)
+- `--json`: Output as JSON
+
+#### patterns
+
+Show available error patterns.
+
+```bash
+loglens patterns [OPTIONS]
+```
+
+Options:
+- `--search TEXT`: Search patterns by name
+- `--show-exceptions`: Show exception patterns
+- `--show-http`: Show HTTP error patterns
+- `--json`: Output as JSON
+
+#### info
+
+Show tool information and configuration.
+
+```bash
+loglens info
+```
+
+### Examples
+
+#### Analyze JSON logs
+
+```bash
+$ loglens analyze application.json --format json
+╭──────────────────────────────────────────────────────────────────────────────╮
+│ Log Analysis Summary                                                         │
+├──────────────────────────────────────────────────────────────────────────────┤
+│ Total Lines:    150                                                          │
+│ Parsed:         148 (98.7%)                                                  │
+│ Errors Found:   12                                                           │
+│ Severity:       2 Critical, 5 Error, 3 Warning, 2 Info                       │
+╰──────────────────────────────────────────────────────────────────────────────╯
+
+Top Errors:
+  [CRITICAL] Database connection failure: 3 occurrences
+  [ERROR] NullPointerException: 2 occurrences
+  [WARNING] Deprecated API usage: 2 occurrences
+```
+
+#### Generate report with severity
+
+```bash
+$ loglens report server.log --severity --output table
+Severity Breakdown:
+  CRITICAL:  ████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  2
+  ERROR:     ██████████████████████████████  15
+  WARNING:   ██████████████████████████░░░░  10
+  INFO:      ██████████████████████████████  18
+  DEBUG:     ██████████████░░░░░░░░░░░░░░░░░  6
+```
+
+#### Real-time monitoring
+
+```bash
+$ loglens watch /var/log/app.log --interval 2
+Watching: /var/log/app.log
+Press Ctrl+C to stop...
+
+Last 5 lines:
+[2024-01-15 10:30:45] ERROR: Database connection failed
+[2024-01-15 10:30:46] WARNING: High memory usage detected
+[2024-01-15 10:30:47] INFO: Connection pool status: healthy
+[2024-01-15 10:30:48] ERROR: Request timeout: /api/users
+[2024-01-15 10:30:49] CRITICAL: Service unresponsive
+```
+
+## Configuration
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `LOGLEVEL` | INFO | Default log level for output |
+| `LOGLENSMASK` | false | Enable PII masking in logs |
+| `LOGLENSCOLOR` | auto | Color output: auto, always, never |
+
+### Configuration File
+
+Create `~/.loglens/config.yaml`:
+
+```yaml
+default_format: auto
+output_format: table
+color: auto
+severity_rules:
+  critical:
+    - "Out of memory"
+    - "Service unresponsive"
+  error:
+    - "ERROR"
+    - "Exception"
+  warning:
+    - "WARNING"
+    - "Deprecated"
+```
+
+### Custom Patterns
+
+Add custom patterns in `~/.loglens/patterns.yaml`:
+
+```yaml
+patterns:
+  - name: "Custom Error"
+    regex: "CUSTOM_ERROR:.*"
+    severity: error
+    suggestion: "Check custom error documentation"
+```
+
+## Supported Log Formats
+
+### JSON Logs
+
+```json
+{
+  "timestamp": "2024-01-15T10:30:00Z",
+  "level": "ERROR",
+  "message": "Connection refused to database",
+  "logger": "main",
+  "service": "api"
+}
+```
+
+### Syslog (RFC 3164/5424)
+
+```
+Jan 15 10:30:00 server-01 app[1234]: ERROR: Connection refused
+Jan 15 10:30:01 server-01 systemd[1]: Started Application Service
+```
+
+### Apache/Nginx
+
+```
+192.168.1.1 - - [15/Jan/2024:10:30:00 +0000] "GET /api/users HTTP/1.1" 200 1234
+192.168.1.2 - - [15/Jan/2024:10:30:01 +0000] "POST /api/login HTTP/1.1" 401 567
+```
+
+## Development
+
+### Setup
+
+```bash
+git clone https://7000pct.gitea.bloupla.net/7000pctAUTO/loglens-cli.git
+cd loglens-cli
+pip install -e ".[dev]"
+```
+
+### Running Tests
+
+```bash
+# Run all tests
+pytest tests/ -v
+
+# Run with coverage
+pytest tests/ --cov=loglens --cov-report=html
+
+# Run specific test suite
+pytest tests/unit/ -v
+pytest tests/integration/ -v
+```
+
+### Linting
+
+```bash
+# Ruff linting
+ruff check loglens/
+
+# Type checking
+mypy loglens/ --ignore-missing-imports
+```
+
+### Code Formatting
+
+```bash
+# Black formatting
+black loglens/ tests/
+
+# Import sorting
+isort loglens/ tests/
+```
+
+## Project Structure
+
+```
+loglens-cli/
+├── loglens/
+│   ├── __init__.py
+│   ├── __main__.py
+│   ├── cli/
+│   │   ├── __init__.py
+│   │   ├── main.py
+│   │   └── commands.py
+│   ├── parsers/
+│   │   ├── __init__.py
+│   │   ├── base.py
+│   │   ├── json_parser.py
+│   │   ├── syslog_parser.py
+│   │   ├── apache_parser.py
+│   │   └── factory.py
+│   ├── analyzers/
+│   │   ├── __init__.py
+│   │   ├── patterns.py
+│   │   ├── severity.py
+│   │   └── analyzer.py
+│   ├── formatters/
+│   │   ├── __init__.py
+│   │   ├── base.py
+│   │   ├── table_formatter.py
+│   │   ├── json_formatter.py
+│   │   └── text_formatter.py
+│   └── config.yaml
+├── tests/
+│   ├── __init__.py
+│   ├── conftest.py
+│   ├── unit/
+│   │   ├── __init__.py
+│   │   ├── test_parsers.py
+│   │   ├── test_analyzer.py
+│   │   └── test_cli.py
+│   ├── integration/
+│   │   ├── __init__.py
+│   │   └── test_end_to_end.py
+│   └── fixtures/
+├── pyproject.toml
+├── README.md
+├── LICENSE
+└── .env.example
+```
+
+## Error Patterns
+
+LogLens includes patterns for:
+
+- **Exceptions**: Python, Java, Node.js stack traces
+- **HTTP Errors**: 5xx server errors, connection failures
+- **Database Errors**: Connection refused, query timeouts
+- **System Errors**: Out of memory, disk space, permissions
+- **Security Issues**: Authentication failures, access denied
+- **Performance**: High latency, slow queries, timeouts
+
+## Contributing
+
+Contributions are welcome! Please read the contributing guidelines before submitting PRs.
+
+## License
+
+MIT License - see LICENSE file for details.