Zarr Vectors Spec

Draft specification for storing large-scale N-dimensional vector data — point clouds, skeletons, streamlines, meshes, tracks-over-time — on top of Zarr v3. Released to spark conversation.

Specification

• 1. Introduction
  • 1.1 Background and Motivation
  • 1.2 Purpose and Goals
  • 1.3 Design Principles
  • 1.4 Relationship to Other Formats
  • 1.5 Key Features
  • 1.6 Target Audiences
  • 1.7 Document Structure
  • 1.8 Conventions
• 2. Scope and Goals
  • 2.1 In Scope
  • 2.2 Out of Scope
  • 2.3 Example Use Cases
• 3. Terminology and Concepts
  • 3.1 Core Terms
  • 3.2 Geometry Terms
  • 3.3 Storage Terms
  • 3.4 Indexing Terms
  • 3.5 Convention Terms
  • 3.6 Capability Tokens
• 4. Data Model
  • 4.1 Hierarchical Organization
  • 4.2 Spatial Index Model
  • 4.3 Vertex Model
  • 4.4 Object Model
  • 4.5 Group Model
  • 4.6 Link Model
• 5. Zarr Store Structure
  • 5.1 Root Structure
  • 5.2 Zarr Version Requirements
  • 5.3 Store Backend Requirements
  • 5.4 Naming Conventions
• 6. Spatial Indexing
  • 6.1 Spatial Index Definition
  • 6.2 Spatial Chunk Addressing
  • 6.3 Spatial Query Semantics
  • 6.4 Boundary Conditions
  • 6.5 Alternative Indexing Strategies
  • 6.6 Per-Level Chunk Shape (v0.7)
• 7. Core Arrays
  • 7.1 Vertex Positions
  • 7.2 Vertex Attributes
  • 7.3 Vertex Fragments
  • 7.4 Groups
  • 7.5 Vertex Links
  • 7.6 Object Index
  • 7.7 Cross-Chunk Links
  • 7.8 Link Attributes
  • 7.9 Cross-Chunk Link Attributes
  • 7.10 Object Attributes
  • 7.11 Fragment Attributes
• 8. Metadata
  • 8.1 Metadata Structure
  • 8.2 Root-Level Metadata
  • 8.3 Resolution-Level Metadata
  • 8.4 Array-Level Metadata
  • 8.5 Object-Level Metadata
  • 8.6 Group-Level Metadata
  • 8.7 Point-Level Metadata
  • 8.8 Coordinate Reference System (CRS)
• 9. Multi-Resolution Support
  • 9.1 Resolution Level Structure
  • 9.2 Downsampling Strategies
  • 9.3 Spatial Chunk Scaling (v0.7)
  • 9.4 Level-of-Detail Selection
  • 9.5 Consistency Across Levels
  • 9.6 Multiscale Link Arrays — Optional
• 10. Cross-Chunk Linking
  • 10.1 Problem Statement
  • 10.2 Strategy 1: Boundary Deduplication
  • 10.3 Strategy 2: Explicit Cross-Chunk Links
  • 10.4 Strategy Selection
  • 10.5 Object Index for Cross-Chunk Objects
  • 10.6 Consistency Guarantees
  • 10.7 Cross-Pyramid-Level Cross-Chunk Links — Optional
• 11. Compression and Encoding
  • 11.1 Compression Overview
  • 11.2 Draco Encoding
  • 11.3 Standard Compression Codecs
  • 11.4 Compression Strategy
  • 11.5 Encoding Metadata
• 12. Geometry Types
  • 12.1 Point Clouds
  • 12.2 Meshes
  • 12.3 Skeletons
  • 12.4 Streamlines / Polylines
  • 12.5 Custom Geometries
• 13. Conformance and Validation
  • 13.1 Conformance Levels
  • 13.2 Validation Rules
  • 13.3 Validation Tools
  • 13.4 Compatibility
• 14. Examples
  • 14.1 Mouse Brain Nuclei: Point Cloud with Multi-Resolution and Regions
  • 14.2 Mesh with Multi-Resolution (v0.7 chunk-scale growth)
  • 14.3 Skeleton with Cross-Chunk Objects
  • 14.4 2-D Polylines with Attributes
  • 14.5 Time-Series Point Cloud (XYZT)
  • 14.6 Multiplexed FISH: Gene Measurements and Cell Types
  • 14.7 Individual Gene Detections in mFISH (Spots → Cells)
  • 14.8 Simple DTI: Small Volume (TRX-Aligned)
  • 14.9 DTI Streamlines: Large Volume with Chunking and Segment Reuse
  • 14.10 Large-Scale Distributed Write
  • Summary
• 15. Appendices
  • Appendix A: JSON Schema Definitions
  • Appendix B: Zarr Implementation Details
  • Appendix C: Coordinate Reference Systems
  • Appendix D: Compression Codec Reference
  • Appendix E: Downsampling Algorithms
  • Appendix F: Query Patterns
  • Appendix G: Migration Guide
  • Appendix H: Performance Considerations
  • Appendix I: Extensibility
  • Appendix J: References
  • Appendix K: Change Log
  • Appendix L: Mapping from Neuroglancer Precomputed Annotations