Cloud migration to a historical tick data shop with automated fulfillment
DataHex Data Management and Data Shop, tick data
Challenge
Hard drives, couriers, and eight-week extractions
With rising demand for tick data history, Morningstar set out to make access faster and easier, modernizing its tick data infrastructure, migrating to the cloud, and adding new tools. It was moving off legacy on-site storage with a single-threaded process that required data copied to hard drives and shipped by courier.
The migration covered over 2.5 PB of level 1 and 2 market data, 50 million instruments, 200+ trading venues, and around 99% of global equities, with data back to 2003.
Solution
A full-service cloud migration, with a shop front
A full-service cloud migration to a native AWS serverless environment on DataHex Data Management, ingesting, curating, and managing market data from global exchanges.
A client interface and shop front for direct login and purchase, and mapping tools for easy adoption of major instrument codes.
What DataHex delivered
- Full-service cloud migration To a native AWS serverless environment on DataHex Data Management.
- Ingestion and curation from global exchanges Market data managed centrally rather than shipped on hard drives.
- Client interface and shop front Direct login and purchase, replacing a courier-based process.
- Instrument code mapping tools Easy adoption of major instrument codes.
Results
From eight weeks to under two hours
- A scalable multi-petabyte operation
- High availability and resilience with access via API, React GUI, FTP, and AWS S3
- A typical extraction cut from 8 weeks to under 2 hours
- Lower barriers to adoption through instrument mapping
Access channels
Talk to us
Ready to build your own data marketplace?
We will walk your team through what a launch looks like, from catalog and licensing to delivery and managed service.