On-Premises Deployment¶

Deploy bytefreezer services on your infrastructure while using the managed control plane.

Overview¶

On-premises deployment allows you to run data processing components in your environment while the control plane remains managed by ByteFreezer:

Location	Components
Your Infrastructure	Proxy, Receiver, Piper, Packer, Query
ByteFreezer Managed	Control Plane, UI

This hybrid model provides:

Data sovereignty: Raw data stays in your environment
Compliance: Data never leaves your network
Central management: UI and configuration via managed control
Simplified operations: No need to manage control plane infrastructure

Architecture¶

┌─────────────────────────────────────────────────────────┐
│                  Your Infrastructure                     │
│                                                          │
│   ┌─────────┐     ┌──────────┐     ┌─────────┐          │
│   │  Proxy  │────▶│ Receiver │────▶│  Piper  │          │
│   └─────────┘     └──────────┘     └────┬────┘          │
│                                         │               │
│                        ┌────────────────┼───────────┐   │
│                        ▼                ▼           │   │
│                   ┌─────────┐     ┌─────────┐       │   │
│                   │ Packer  │     │  Query  │       │   │
│                   └────┬────┘     └────┬────┘       │   │
│                        │               │            │   │
│                        ▼               ▼            │   │
│                   ┌─────────────────────────────┐   │   │
│                   │     Your S3 Storage         │   │   │
│                   │     (MinIO, AWS S3, etc)    │   │   │
│                   └─────────────────────────────┘   │   │
│                                                     │   │
└─────────────────────────────────────────────────────┘   │
                           │                              │
                           │ Service Key Authentication   │
                           ▼                              │
┌─────────────────────────────────────────────────────────┘
│           ByteFreezer Managed Control
│
│   ┌─────────────────────────────────────────────┐
│   │          Control Plane                       │
│   │   - Configuration management                 │
│   │   - Dataset definitions                      │
│   │   - Transformation rules                     │
│   │   - User management                          │
│   └─────────────────────────────────────────────┘
│
│   ┌─────────────────────────────────────────────┐
│   │          Web UI (app.bytefreezer.com)       │
│   └─────────────────────────────────────────────┘
│
└──────────────────────────────────────────────────────────

Prerequisites¶

ByteFreezer Account: Sign up at app.bytefreezer.com
Infrastructure: Linux servers or Kubernetes cluster
S3 Storage: MinIO, AWS S3, or compatible object storage
Network: Outbound HTTPS (443) to control.bytefreezer.com

Step 1: Generate a Service Key¶

Service keys authenticate your on-prem services with the managed control plane.

Log into app.bytefreezer.com
Navigate to API Keys in the sidebar
Click Generate New Key
Select Service as the key type
Enter a descriptive name (e.g., "production-datacenter-1")
Copy the key immediately - it will only be shown once
Store the key securely (e.g., HashiCorp Vault, AWS Secrets Manager)

Key Security

The service key provides full API access for your account. Treat it like a password:

Never commit to version control
Use environment variables or secrets management
Rotate keys periodically
Revoke immediately if compromised

Step 2: Configure Storage¶

Deploy S3-compatible storage in your environment. MinIO is recommended for on-prem:

# Using Docker
docker run -d \
  --name minio \
  -p 9000:9000 \
  -p 9001:9001 \
  -v /data/minio:/data \
  -e MINIO_ROOT_USER=admin \
  -e MINIO_ROOT_PASSWORD=changeme123 \
  minio/minio server /data --console-address ":9001"

Create a bucket for bytefreezer data:

mc alias set local http://localhost:9000 admin changeme123
mc mb local/bytefreezer-data

Step 3: Deploy Services¶

Environment Variables¶

All services require these environment variables:

# Control plane connection
export BYTEFREEZER_CONTROL_URL="https://control.bytefreezer.com"
export BYTEFREEZER_SERVICE_KEY="your-service-key-here"

# S3 storage (shared by receiver, piper, packer, query)
export S3_ENDPOINT="http://minio:9000"
export S3_BUCKET="bytefreezer-data"
export S3_ACCESS_KEY="admin"
export S3_SECRET_KEY="changeme123"
export S3_REGION="us-east-1"

Receiver¶

Receives data from proxies and stores to S3:

# receiver-config.yaml
server:
  port: 8081

control:
  url: ${BYTEFREEZER_CONTROL_URL}
  service_key: ${BYTEFREEZER_SERVICE_KEY}

storage:
  type: s3
  endpoint: ${S3_ENDPOINT}
  bucket: ${S3_BUCKET}
  access_key: ${S3_ACCESS_KEY}
  secret_key: ${S3_SECRET_KEY}
  region: ${S3_REGION}

./bytefreezer-receiver --config receiver-config.yaml

Piper¶

Processes raw data and applies transformations:

# piper-config.yaml
control:
  url: ${BYTEFREEZER_CONTROL_URL}
  service_key: ${BYTEFREEZER_SERVICE_KEY}

storage:
  type: s3
  endpoint: ${S3_ENDPOINT}
  bucket: ${S3_BUCKET}
  access_key: ${S3_ACCESS_KEY}
  secret_key: ${S3_SECRET_KEY}
  region: ${S3_REGION}

./bytefreezer-piper --config piper-config.yaml

Packer¶

Compresses and archives processed data:

# packer-config.yaml
control:
  url: ${BYTEFREEZER_CONTROL_URL}
  service_key: ${BYTEFREEZER_SERVICE_KEY}

storage:
  type: s3
  endpoint: ${S3_ENDPOINT}
  bucket: ${S3_BUCKET}
  access_key: ${S3_ACCESS_KEY}
  secret_key: ${S3_SECRET_KEY}
  region: ${S3_REGION}

./bytefreezer-packer --config packer-config.yaml

Proxy¶

Collects data and forwards to receiver:

# proxy-config.yaml
server:
  port: 8082

receiver:
  url: "http://receiver:8081"

control:
  url: ${BYTEFREEZER_CONTROL_URL}
  service_key: ${BYTEFREEZER_SERVICE_KEY}

./bytefreezer-proxy --config proxy-config.yaml

Query Service (Optional)¶

Enables local querying of stored data:

# query-config.yaml
server:
  port: 8084

control:
  url: ${BYTEFREEZER_CONTROL_URL}
  service_key: ${BYTEFREEZER_SERVICE_KEY}

storage:
  type: s3
  endpoint: ${S3_ENDPOINT}
  bucket: ${S3_BUCKET}
  access_key: ${S3_ACCESS_KEY}
  secret_key: ${S3_SECRET_KEY}
  region: ${S3_REGION}

./bytefreezer-query --config query-config.yaml

Step 4: Verify Connectivity¶

Test that services can reach the control plane:

# Test service key authentication
curl -s -w "\nHTTP: %{http_code}\n" \
  -H "Authorization: Bearer $BYTEFREEZER_SERVICE_KEY" \
  https://control.bytefreezer.com/api/v1/tenants

Expected output: JSON list of tenants and HTTP: 200

Docker Compose Example¶

Full stack deployment using Docker Compose:

# docker-compose.yaml
version: '3.8'

services:
  minio:
    image: minio/minio
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio_data:/data
    environment:
      MINIO_ROOT_USER: admin
      MINIO_ROOT_PASSWORD: changeme123
    command: server /data --console-address ":9001"

  receiver:
    image: bytefreezer/receiver:latest
    ports:
      - "8081:8081"
    environment:
      BYTEFREEZER_CONTROL_URL: https://control.bytefreezer.com
      BYTEFREEZER_SERVICE_KEY: ${BYTEFREEZER_SERVICE_KEY}
      S3_ENDPOINT: http://minio:9000
      S3_BUCKET: bytefreezer-data
      S3_ACCESS_KEY: admin
      S3_SECRET_KEY: changeme123
    depends_on:
      - minio

  piper:
    image: bytefreezer/piper:latest
    environment:
      BYTEFREEZER_CONTROL_URL: https://control.bytefreezer.com
      BYTEFREEZER_SERVICE_KEY: ${BYTEFREEZER_SERVICE_KEY}
      S3_ENDPOINT: http://minio:9000
      S3_BUCKET: bytefreezer-data
      S3_ACCESS_KEY: admin
      S3_SECRET_KEY: changeme123
    depends_on:
      - minio

  packer:
    image: bytefreezer/packer:latest
    environment:
      BYTEFREEZER_CONTROL_URL: https://control.bytefreezer.com
      BYTEFREEZER_SERVICE_KEY: ${BYTEFREEZER_SERVICE_KEY}
      S3_ENDPOINT: http://minio:9000
      S3_BUCKET: bytefreezer-data
      S3_ACCESS_KEY: admin
      S3_SECRET_KEY: changeme123
    depends_on:
      - minio

  proxy:
    image: bytefreezer/proxy:latest
    ports:
      - "8082:8082"
    environment:
      RECEIVER_URL: http://receiver:8081
      BYTEFREEZER_CONTROL_URL: https://control.bytefreezer.com
      BYTEFREEZER_SERVICE_KEY: ${BYTEFREEZER_SERVICE_KEY}
    depends_on:
      - receiver

volumes:
  minio_data:

Start with:

export BYTEFREEZER_SERVICE_KEY="your-key-here"
docker-compose up -d

Managing Service Keys¶

Viewing Keys¶

In the UI, navigate to API Keys to see all keys. Service keys are marked with a "Service" type badge and show:

Key name
Key prefix (first 8 characters for identification)
Creation date
Last used timestamp

Revoking Keys¶

To revoke a compromised or unused key:

Navigate to API Keys
Find the service key in the list
Click Revoke next to the key
Confirm revocation

Immediate Effect

Revoked keys stop working immediately. All services using that key will lose access to the control plane.

Key Rotation¶

Best practice is to rotate service keys periodically:

Generate a new service key
Update all on-prem service configurations
Restart services to pick up new key
Verify connectivity
Revoke the old key

Troubleshooting¶

"401 Unauthorized" errors¶

Verify service key is correct and not revoked
Check key is passed in Authorization: Bearer <key> header
Ensure BYTEFREEZER_SERVICE_KEY environment variable is set

"Connection refused" to control plane¶

Verify outbound HTTPS (443) is allowed to control.bytefreezer.com
Check for proxy/firewall blocking the connection
Test with: curl -v https://control.bytefreezer.com/health

Services not processing data¶

Check logs for authentication errors
Verify S3 credentials and bucket access
Confirm control plane connectivity from each service

Next Steps¶

Data Model - Understand tenants and datasets
Transformations - Configure data processing
API Reference - Full API documentation