200 lines
3.8 KiB
Markdown
200 lines
3.8 KiB
Markdown
# DataForge CLI
|
|
|
|
A CLI tool that converts and validates data formats (JSON, YAML, TOML) with schema validation using JSON Schema. Features include format conversion, schema validation, type checking, batch processing, and quiet mode for scripting.
|
|
|
|
## Features
|
|
|
|
- **Format Conversion**: Convert between JSON, YAML, and TOML formats
|
|
- **Schema Validation**: Validate data against JSON Schema (draft-07, draft-2019-09)
|
|
- **Type Checking**: Infer and validate data types
|
|
- **Batch Processing**: Process multiple files with pattern matching
|
|
- **Quiet Mode**: Minimal output for scripting and CI/CD integration
|
|
|
|
## Installation
|
|
|
|
### From Source
|
|
|
|
```bash
|
|
git clone https://github.com/dataforge/dataforge-cli
|
|
cd dataforge-cli
|
|
pip install -e .
|
|
```
|
|
|
|
### Using pipx
|
|
|
|
```bash
|
|
pipx install dataforge-cli
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
### Convert a file from JSON to YAML
|
|
|
|
```bash
|
|
dataforge convert input.json --to yaml -o output.yaml
|
|
```
|
|
|
|
### Convert from YAML to TOML
|
|
|
|
```bash
|
|
dataforge convert config.yaml --to toml -o config.toml
|
|
```
|
|
|
|
### Validate against a schema
|
|
|
|
```bash
|
|
dataforge validate data.json --schema schema.json
|
|
```
|
|
|
|
### Type checking with schema inference
|
|
|
|
```bash
|
|
dataforge typecheck config.json --infer
|
|
```
|
|
|
|
## Commands
|
|
|
|
### convert
|
|
|
|
Convert a file from one format to another.
|
|
|
|
```bash
|
|
dataforge convert INPUT_FILE OUTPUT_FILE --from FORMAT --to FORMAT
|
|
```
|
|
|
|
Options:
|
|
- `--from, -f`: Input format (json, yaml, toml)
|
|
- `--to, -t`: Output format (required, json, yaml, toml)
|
|
- `--indent, -i`: Indentation spaces (0 for compact)
|
|
- `--quiet, -q`: Minimal output
|
|
|
|
### validate
|
|
|
|
Validate a file against a JSON Schema.
|
|
|
|
```bash
|
|
dataforge validate INPUT_FILE --schema SCHEMA_FILE
|
|
```
|
|
|
|
Options:
|
|
- `--schema, -s`: Path to JSON Schema file
|
|
- `--strict`: Strict validation mode
|
|
- `--quiet, -q`: Minimal output
|
|
|
|
### typecheck
|
|
|
|
Check types in a data file.
|
|
|
|
```bash
|
|
dataforge typecheck INPUT_FILE
|
|
```
|
|
|
|
Options:
|
|
- `--infer`: Infer schema from data
|
|
- `--quiet, -q`: Minimal output
|
|
|
|
### batch-convert
|
|
|
|
Convert multiple files.
|
|
|
|
```bash
|
|
dataforge batch-convert --from FORMAT --to FORMAT --pattern "*.json" --directory DIR
|
|
```
|
|
|
|
Options:
|
|
- `--from, -f`: Input format (json, yaml, toml)
|
|
- `--to, -t`: Output format (required)
|
|
- `--output-dir, -o`: Output directory
|
|
- `--pattern, -p`: File pattern for batch processing
|
|
- `--recursive, -r`: Search recursively
|
|
- `--quiet, -q`: Minimal output
|
|
|
|
### batch-validate
|
|
|
|
Validate multiple files against a schema.
|
|
|
|
```bash
|
|
dataforge batch-validate --schema SCHEMA FILE1 FILE2 ...
|
|
```
|
|
|
|
Options:
|
|
- `--schema, -s`: Path to JSON Schema file (required)
|
|
- `--pattern, -p`: File pattern for batch processing
|
|
- `--recursive, -r`: Search recursively
|
|
- `--quiet, -q`: Minimal output
|
|
|
|
## Examples
|
|
|
|
### Converting with stdin/stdout
|
|
|
|
```bash
|
|
cat config.json | dataforge convert - --to yaml
|
|
```
|
|
|
|
### Validation in CI/CD
|
|
|
|
```bash
|
|
dataforge validate config.json --schema schema.json --quiet
|
|
if [ $? -ne 0 ]; then
|
|
echo "Validation failed!"
|
|
exit 1
|
|
fi
|
|
```
|
|
|
|
### Batch conversion
|
|
|
|
```bash
|
|
dataforge batch-convert --from json --to yaml --pattern "*.json" --output-dir converted/
|
|
```
|
|
|
|
### Schema inference
|
|
|
|
```bash
|
|
dataforge typecheck data.json --infer > schema.json
|
|
```
|
|
|
|
## Exit Codes
|
|
|
|
| Code | Description |
|
|
|------|-------------|
|
|
| 0 | Success / Validation passed |
|
|
| 1 | Error / Validation failed |
|
|
|
|
## Configuration
|
|
|
|
No configuration file required. All options are passed via command-line arguments.
|
|
|
|
## Supported Formats
|
|
|
|
| Format | Extension | Read | Write |
|
|
|--------|-----------|------|-------|
|
|
| JSON | .json | Yes | Yes |
|
|
| YAML | .yaml, .yml | Yes | Yes |
|
|
| TOML | .toml | Yes | Yes |
|
|
|
|
## Development
|
|
|
|
### Setup
|
|
|
|
```bash
|
|
git clone https://github.com/dataforge/dataforge-cli
|
|
cd dataforge-cli
|
|
pip install -e ".[dev]"
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
pytest tests/ -v --cov=dataforge
|
|
```
|
|
|
|
### Linting
|
|
|
|
```bash
|
|
ruff check dataforge tests
|
|
```
|
|
|
|
## License
|
|
|
|
MIT License - see [LICENSE](LICENSE) for details.
|