Why TimeBase

Why TimeBase

TimeBase is a high-performance streaming time-series database developed by EPAM Real Time Computing Lab (formerly Deltix).

TimeBase was designed for very fast data aggregation and retrieval of massive volumes of high frequency financial market data. The same TimeBase technology excels at processing any time-series data: financial markets (MBO/ITCH), IoT (MQTT), software metrics and signalsreal-time eventslogging etc. TimeBase runs standalone or in a cluster, processes millions of messages per second per core, stores terabytes of data, and offers microsecond latencies.

TimeBase combines multiple solutions into single package:

Key TimeBase Differentiators

  • Unified streaming API for both historical and live time-series data.
  • High performance: system may be configured to stream data with microsecond latencies or read/write millions of messages per second on each data producer and consumer.
  • Low latency: when streaming live data, TimeBase can serve real-time consumers from memory rather than disk, which allows for a significant latency reduction.
  • Complex message structure: TimeBase can store complex message structures that reflect data in your business domain (no need for intermediate DTO objects).
  • Schema-based database with embedded data serialization and modeling framework allowing for better visibility and data migration. Smooth transition from rapid data prototyping to production solution.
  • Row-based design offers better latency and throughput for streaming use cases comparing with column-based databases.

Typical Tasks

  • Data replication framework: use multiple out-of-the box integrations or open multi-language API to create custom integrations.
  • Aggregation of massive volumes of heterogeneous time-series data history or real-time from multiple sources with superior latency and throughput.
  • Reliable data storage for heterogeneous time-series data.
  • Rapid retrieval/streaming of time-series data both history and real-time. TimeBase has a sophisticated time-series engine, capable of efficient on-the-fly merging of multiple data streams with arbitrary temporal characteristics into a unified query response.
  • Live data streaming provided by a simultaneous work of readers and writers.
  • Framework for data processing and enrichment (foundation for building normalization and validation frameworks).
  • Statistical models and machine learning: warm-up mode (initialization with historical data), parameter optimization, online forecasting, recurring learning (on the fly adjustment with the up-to-date parameters).

TimeBase Benefits Overview

FeatureDescriptionBenefit
Integrated Messaging and Persistence TimeBase works equally well with historical and real-time data. It uniquely combines message distribution and persistence functions. Messages distributed from publishers to consumers can be automatically saved to the database for later replay or analysis. The switch between backtesting and production is transparent to API user.
Production deployments can automatically save data (including both market feeds and control messages) for warm-up and backtesting.
Heterogeneous Platform Support TimeBase can be accessed from Java or any Microsoft .NET language. Users are free to use the tools and languages that are familiar and most appropriate for their environment.
Client Libraries
Rich Type System TimeBase provides a rich arsenal of data encodings to represent many data types, including (but not limited to) decimal numbers, IEEE floats, text, integer numbers, small alphanumeric codes, enumerations and true/false values. Native representation of user's data model.
Data compression and transmission performance.
Schema Annotations
Out-of-the-box Native Object Binding TimeBase comes with a diverse well-documented API, with identical support for Java and .NET. In particular, dynamic generation of code for binding language-native objects to data (without losing performance) is built-in. Short learning curve.
Programs written against TimeBase API are clean, simple and efficient.
Schema Annotations
Asset Classes Support for equities, options, indexes, futures, bonds, ETFs, currencies and custom objects.
Market Vendor Integration Supports connectors to major data vendors: Bloomberg, Reuters, QuantHouse plus many more venues. The list is constantly growing. User can start collecting tick data in minutes after the software is installed.
Native Time-Series Support TimeBase is built from the ground up to support time-series concepts. All data is automatically associated with a timestamp (with millisecond precision). The query engine, API and management tools are all optimized for time-series processing. TimeBase provides high performance and usability for aggregating and querying time-series data.
Polymorphic Object-Oriented Data Model Ability to store large time-series of heterogeneously structured messages. Message structure is defined by Object-Oriented Design methods and supports inheritance.
TimeBase Structure
Ability to handle time-series data of different types: news, sentiment, sensory etc.
Low Search Latency combined with Extreme Read Throughput TimeBase gives the user the ability to quickly locate and retrieve required data, reaching the speeds of over 1 million messages per second per core on low-end desktop hardware. User can write regular programs to process data outside of the database system using development tools and environments of their choice.
TimeBase Requirements
Unmatched Downscaling TimeBase unique design allows it to function, when necessary, on (relatively) low-end hardware with minimal consumption of RAM, while maintaining reasonable level of performance.
SW/HW Requirements
It runs well on laptops and low-end servers, and leaves system resources for other applications and user's data processing programs (usually trading algorithms).
Deployments can start small and grow with time.
Built-In ETL Tools TimeBase comes with tools to import CSV and MS Excel files. User can upload proprietary data using out-of-the-box tools.