The zero-copy architecture, with data in its original location, preserves time series and doesn't modify the original data in any way
Zero-Copy Data Storage Architecture
CloudQuant's Data Liberator uses a zero-copy architecture where data remains in its original storage locations rather than being moved or duplicated. The system connects directly to existing databases, data stores, or file systems across various platforms including:
- Cloud storage (AWS S3, Azure Blob Storage, Google Cloud Storage)
- Databases (SQL Server, Oracle, PostgreSQL, MySQL, MariaDB, Snowflake)
- File systems (SFTP, local file systems, network shares)
- Real-time market data feeds
The Zero-Copy Limitation and Time Series Advantage
Why You Can't Modify Data in Zero-Copy Locations through Liberator: Since the data remains in its original location and the system provides read-only access without copying, you cannot directly modify existing records. This is by design - the zero-copy approach preserves data integrity and eliminates the risks associated with data duplication or migration.
The Time Series Solution: However, this limitation becomes an advantage when working with time series data because:
-
Append-Only Nature: Time series data is naturally append-only. Instead of modifying existing records, you add new records with timestamps to the end of the dataset.
-
Complete Historical View: When you append a new record or file, you maintain the entire historical timeline of changes. Each modification becomes a new data point with its own timestamp, preserving the complete evolution of the data.
-
Temporal Queries: CloudQuant's platform is purpose-built for financial data with point-in-time accuracy and temporal queries, allowing you to:
- Query data as it existed at any specific point in time
- See the progression of values over time
- Track all modifications as part of the time series
-
Real-Time Updates: New data can be continuously appended to the end of the time series, whether it's market data feeds, trading records, or analytical results, without disrupting the existing historical data.
This approach gives you both the efficiency of zero-copy access and the comprehensive audit trail that time series data requires, making it particularly valuable for financial applications where historical accuracy and the ability to track changes over time are crucial.