This directory contains detailed documentation for the core features of the Pangolin platform.
Basic data operations and lifecycle management.
- Entities: Understanding the Pangolin core models.
- Asset Management: Handling Tables, Views, and other assets.
- Generic Assets: Cataloging ML Models, Videos, and Files.
- Modern Table Formats: Support for Delta Lake, Hudi, and Paimon.
- Warehouse Management: Configuring storage backends.
- Time Travel: Querying historical data states.
- Tag Management: Versioning with tags.
Git-like workflows for your data lake.
- Branch Management: Creating and managing branches.
- Merge Operations: Workflow for 3-way merges.
- Merge Conflicts: Principles of data conflict resolution.
Governance and authentication mechanisms.
- Multi-Tenancy: Isolation principles and tenant management.
- RBAC: Role-Based Access Control system.
- Credential Vending: Cloud IAM and STS integration.
- Security Concepts: Concept guide for vending and signing.
Finding data and tracking changes.
- Business Catalog: Data discovery portal and business metadata.
- Audit Logging: Comprehensive action tracking and compliance.
Connecting to external systems.
- Federated Catalogs: Connecting to remote Iceberg catalogs.
- Catalog Management: Managing local and federated catalogs.
- Service Users: API keys for programmatic access.
Using Pangolin with other tools.
- PyIceberg Integration: Guide for Python users.
Optimizing and maintaining your data lake.
- Maintenance Operations: Snapshot management and orphan file cleanup.
| Feature | Category | Status |
|---|---|---|
| Multi-Tenancy | System | ✅ Implemented |
| Branching | Versioning | ✅ Implemented |
| 3-Way Merge | Versioning | ✅ Implemented |
| RBAC | Security | ✅ Implemented |
| Audit Logging | Security | ✅ Implemented |
| Business Catalog | Discovery | ✅ Implemented |
| Credential Vending (Static) | Security | ✅ Implemented |
| Credential Vending (STS) | Security | ✅ Implemented |
| Federated Catalogs | Integration | ✅ Implemented |
| View Management | Data | ✅ Implemented |