# PromptForge

A CLI tool for versioning, testing, and sharing AI prompts across different LLM providers. Treat prompts as code with git integration, A/B testing, templating, and a shared prompt registry.

## Features

- **Version Control**: Track prompt changes with Git, create branches for A/B testing variations
- **Multi-Provider Support**: Unified API for OpenAI, Anthropic, and Ollama
- **Prompt Templating**: Jinja2-based variable substitution with type validation
- **A/B Testing Framework**: Compare prompt variations with statistical analysis
- **Output Validation**: Validate LLM responses against JSON schemas or regex patterns
- **Prompt Registry**: Share prompts via local and remote registries

## Installation

```bash
pip install promptforge
```

Or from source:

```bash
git clone https://github.com/yourusername/promptforge.git
cd promptforge
pip install -e .
```

## Quick Start

1. Initialize a new PromptForge project:

```bash
pf init
```

2. Create your first prompt:

```bash
pf prompt create "Summarizer" -c "Summarize the following text: {{text}}"
```

3. Run the prompt:

```bash
pf run Summarizer -v text="Your long text here..."
```

## Configuration

Create a `configs/promptforge.yaml` file:

```yaml
providers:
  openai:
    api_key: ${OPENAI_API_KEY}
    model: gpt-4
    temperature: 0.7
  anthropic:
    api_key: ${ANTHROPIC_API_KEY}
    model: claude-3-sonnet-20240229
  ollama:
    base_url: http://localhost:11434
    model: llama2

defaults:
  provider: openai
  output_format: text
```

## Creating Prompts

Prompts are YAML files with front matter:

```yaml
---
name: Code Explainer
description: Explain code snippets
version: "1.0.0"
provider: openai
tags: [coding, education]
variables:
  - name: language
    type: choice
    required: true
    choices: [python, javascript, rust, go]
  - name: code
    type: string
    required: true
validation:
  - type: regex
    pattern: "(def|function|fn|func)"
---
Explain this {{language}} code:

{{code}}

Focus on:
- What the code does
- Key functions/classes used
- Any potential improvements
```

## Running Prompts

```bash
# Run with variables
pf run "Code Explainer" -v language=python -v code="def hello(): print('world')"

# Use a different provider
pf run "Code Explainer" -p anthropic -v language=rust -v code="..."

# Output as JSON
pf run "Code Explainer" -o json -v ...
```

## Version Control

```bash
# Create a version commit
pf version create "Added validation rules"

# View history
pf version history

# Create a branch for A/B testing
pf version branch test-variation-a

# List all branches
pf version list
```

## A/B Testing

Compare prompt variations:

```bash
# Test a single prompt
pf test "Code Explainer" --iterations 5

# Compare multiple prompts
pf test "Prompt A" "Prompt B" --iterations 3

# Run in parallel
pf test "Prompt A" "Prompt B" --parallel
```

## Output Validation

Add validation rules to your prompts:

```yaml
validation:
  - type: regex
    pattern: "^\\d+\\. .+"
    message: "Response must be numbered list"
  
  - type: json
    schema:
      type: object
      properties:
        summary:
          type: string
          minLength: 10
        keywords:
          type: array
          items:
            type: string
```

## Prompt Registry

### Local Registry

```bash
# List local prompts
pf registry list

# Add prompt to registry
pf registry add "Code Explainer" --author "Your Name"

# Search registry
pf registry search "python"
```

### Remote Registry

```bash
# Pull a prompt from remote
pf registry pull <entry_id>

# Publish your prompt
pf registry publish "Code Explainer"
```

## API Reference

### Core Classes

- `Prompt`: Main prompt model with YAML serialization
- `TemplateEngine`: Jinja2-based template rendering
- `GitManager`: Git integration for version control
- `ProviderBase`: Abstract interface for LLM providers

### Providers

- `OpenAIProvider`: OpenAI GPT models
- `AnthropicProvider`: Anthropic Claude models
- `OllamaProvider`: Local Ollama models

### Testing

- `ABTest`: A/B test runner
- `Validator`: Response validation framework
- `MetricsCollector`: Metrics aggregation

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests: `pytest tests/ -v`
5. Submit a pull request

## License

MIT License - see LICENSE file for details.