Getting Started¶

Welcome to ByteFreezer! This guide will help you understand the core concepts and get up and running quickly.

Core Concepts¶

ByteFreezer is built around a few key concepts:

Data Hierarchy¶

ByteFreezer organizes data in a hierarchical structure:

Account (Organization)
  └── Tenant (Data Source / Environment)
        └── Dataset (Data Stream / Collection)
              └── Events (Individual Records)

• Accounts represent organizations or customers
• Tenants are logical partitions for different data sources or environments
• Datasets are individual data streams with their own schemas and transformations

Data Flow¶

1. Ingestion: Data arrives via UDP proxy or HTTP webhooks
2. Storage: Raw data is stored in S3 for durability
3. Transformation: Piper processes data through filter pipelines
4. Compression: Packer optimizes data for long-term storage

Next Steps¶

• Understand the Data Model - Learn about accounts, tenants, and datasets
• Quick Start Guide - Set up your first data pipeline
• Transformations - Build data transformation pipelines