TimeBase is a high-performance streaming time-series database developed by EPAM Real-Time Computing Lab (formerly Deltix).
TimeBase was designed for very fast data aggregation and retrieval of massive volumes of high-frequency financial market data. The same TimeBase technology excels at processing any time-series data: financial markets (MBO/ITCH), IoT (MQTT), software metrics and signals, real-time events, logging, etc. TimeBase runs standalone or in a cluster, processes millions of messages per second per core, stores terabytes of data, and offers microsecond latencies.
TimeBase combines multiple solutions into a single package:
- Persistent message broker
- Message-oriented time-series database
- Schema-based data modeling and serialization framework
Key TimeBase Differentiators
- Unified streaming API for both historical and live time-series data.
- High performance: system may be configured to stream data with microsecond latencies or read/write millions of messages per second on each data producer and consumer.
- Low latency: when streaming live data, TimeBase can serve real-time consumers from memory rather than disk, which allows for a significant latency reduction.
- Complex message structure: TimeBase can store complex message structures that reflect data in your business domain (no need for intermediate DTO objects).
- Schema-based database with embedded data serialization and modeling framework allowing for better visibility and data migration. Smooth transition from rapid data prototyping to production solution.
- The row-based design offers better latency and throughput for streaming use cases comparing with column-based databases.
- Data replication framework: use multiple out-of-the-box integrations or open multi-language API to create custom integrations.
- Aggregation of massive volumes of heterogeneous time-series data history or real-time from multiple sources with superior latency and throughput.
- Reliable data storage for heterogeneous time-series data.
- Rapid retrieval/streaming of time-series data both history and real-time. TimeBase has a sophisticated time-series engine, capable of efficient on-the-fly merging of multiple data streams with arbitrary temporal characteristics into a unified query response.
- Live data streaming is provided by a simultaneous work of readers and writers.
- Framework for data processing and enrichment (foundation for building normalization and validation frameworks).
- Statistical models and machine learning: warm-up mode (initialization with historical data), parameter optimization, online forecasting, recurring learning (on-the-fly adjustment with the up-to-date parameters).
TimeBase Benefits Overview
|Integrated Messaging and Persistence
|TimeBase works equally well with historical and real-time data. It uniquely combines message distribution and persistence functions. Messages distributed from publishers to consumers can be automatically saved to the database for later replay or analysis.
|The switch between backtesting and production is transparent to the API users.
Production deployments can automatically save data (including both markets feeds and control messages) for warm-up and backtesting.
|Heterogeneous Platform Support
|TimeBase can be accessed from Java or any Microsoft .NET language.
|Users are free to use the tools and languages that are familiar and most appropriate for their environment.
|Rich Type System
|TimeBase provides a rich arsenal of data encodings to represent many data types, including (but not limited to) decimal numbers, IEEE floats, text, integer numbers, small alphanumeric codes, enumerations, and true/false values.
|The native representation of the user's data model.
Data compression and transmission performance.
|Out-of-the-box Native Object Binding
|TimeBase comes with a diverse well-documented API, with identical support for Java and .NET. In particular, dynamic generation of code for binding language-native objects to data (without losing performance) is built-in.
|Short learning curve.
Programs written against TimeBase API are clean, simple, and efficient.
|Support for equities, options, indexes, futures, bonds, ETFs, currencies, and custom objects.
|Market Vendor Integration
|Supports connectors to major data vendors: Bloomberg, Reuters, QuantHouse plus many more venues. The list is constantly growing.
|Users can start collecting tick data in minutes after the software is installed.
|Native Time-Series Support
|TimeBase is built from the ground up to support time-series concepts. All data is automatically associated with a timestamp (with millisecond precision). The query engine, API, and management tools are all optimized for time-series processing.
|TimeBase provides high performance and usability for aggregating and querying time-series data.
|Polymorphic Object-Oriented Data Model
|Ability to store large volumes of heterogeneously structured messages. The message structure is defined by Object-Oriented Design methods and supports inheritance.
|Ability to handle time-series data of different types: news, sentiment, sensory, etc.
|Low Search Latency combined with Extreme Read Throughput
|TimeBase gives the user the ability to quickly locate and retrieve required data, reaching the speeds of over 1 million messages per second per core on low-end desktop hardware.
|Users can write regular programs to process data outside of the database system using development tools and environments of their choice.
|TimeBase's unique design allows it to function, when necessary, on (relatively) low-end hardware with minimal consumption of RAM, while maintaining a reasonable level of performance.
|It runs well on laptops and low-end servers and leaves system resources for other applications and user's data processing programs (usually trading algorithms).
Deployments can start small and grow with time.
|Built-In ETL Tools
|TimeBase comes with tools to import CSV and MS Excel files.
|Users can upload proprietary data using out-of-the-box tools.