+
+
+
+
+
+
+
+ + + 404: + + Page not found +
++ Sorry, but the page you were trying to view does not exist. + +
+ + Return to homepage + +
+
+
+
+
+
++ Sorry, but the page you were trying to view does not exist. + +
+ + Return to homepage + +M3 consists of 3 simple components, all purposefully designed for scale, reliability and efficiency.
+
+ 
+ Ingest and streaming aggregation of metrics with dynamic configuration
+
+ Purpose built time series database (M3DB) for better performance, reliability and consistent results
+
+ 
+ Real time, highly performant and optimized query engine
+
+ 
+ 
+ 
+ 
+ 
+ In 2014, following several years of battling a legacy metrics platform based on Graphite and WhisperDB, the Observability team at Uber began developing an open source metrics platform that would become known as M3. The goal of the project was to provide teams with a highly available and centralized metrics platform that could be easily integrated with existing tooling.
+ + + +
+ 
+ 
+ 
+ The first iteration of the storage layer featured Cassandra and ElasticSearch; however, as the ride-hailing business entered its hyper growth phase, the firefighting that had once plagued the team during the Graphite and WhisperDB years had resurfaced. The team then decided to build M3DB, a custom timeseries database with an embedded inverted index, from scratch which became the foundation of the platform. Over the course of about a year and a half, M3DB was developed and rolled out across Uber.
+ + + + + Learn more + + + + +
+ 
+ OOMs and slow queries became a regular occurrence, which prevented teams from effectively monitoring their services. The new query engine, dubbed M3Query, takes advantage of the fact that M3DB stores data in highly compressed blocks, and decompresses the data client side one datapoint at a time while functions are lazily applied. These optimizations minimize network, compute and memory resources. The new query engine, which supports PromQL and the legacy Graphite language, gave teams an extremely fast and powerful way to gain insights into their metrics.
+ + + + + Learn more + + + + +
+ 
+ As the number of metrics being ingested skyrocketed, it became clear that there wasn’t a one-size-fits-all methodology when it came to data storage policies. Different teams wanted their metrics stored at different resolutions and retention periods. For example, infrastructure teams needed metrics stored at very high resolutions, but not for too long. Conversely, business teams didn’t need fine granularities, but wanted to retain data for much longer. The solution was a combination of two components, M3Coordinator and M3Aggregator, which work in tandem to ingest metrics and aggregate them in a highly customizable way all while providing high availability and efficiency.
+ + + + + Learn more + + + + +
+ 
+ Today, the M3 project is maintained by open source contributors around the globe, and is currently used by some of the largest enterprises in the world.
+ + + +The M3 Technical Steering Committee
+
+ 
+ Uber
+
+ 
+ Chronosphere
+
+ 
+ Uber
+
+ 
+ Uber
+
+ 
+ Chronosphere
+
+ If you are interested in proposing a feature for M3, click the link and follow the steps provided.
+ + Propose Feature + +
+
+
+ 
+ Welcome to the M3 community! There are a variety of resources to share the latest news, answer questions, and troubleshoot any issues. See the below list for more information on these resources.
+
+ 
+ 
+ 
+ 
+
+ 
+ + Learn how and why M3 was created and used by Uber +
+
+ + CNCF Member Webinar - August 2020 +
+
+ + Keynote from KubeCon Seattle 2018 +
+
+ + Lightning talk on M3 from PromCon 2018 +
+- M3, a metrics platform, and M3DB, a distributed time series database, were developed at Uber out - of necessity. After using what was available as open source and finding we were unable to use them - at our scale due to issues with their reliability, cost and operationally intensive nature - we built our own metrics platform piece by piece. We used our experience to help us build a native - distributed time series database, a highly dynamic and performant aggregation service, query engine - and other supporting infrastructure. -
-Out of the box support for handling multiple clusters, built on top of etcd
-Built-in replication with configurable durability and read consistency for your needs
-Reads and writes can be set to quorum or lower or higher depending on durability preferences
-M3DB can act as the long term storage for Prometheus using remote read/write endpoints
-Efficient compression algorithm inspired by Gorilla TSZ, configurable as lossless or lossy
-Configurable precision from seconds to nanoseconds, with the ability to switch precision with any write
-Handles out of order writes seamlessly, only limited by the size of the configured database block size
-All features are fully available and open source. Any contributions are welcome!
-
+
+
+
+ + Putting billions of timeseries to work at Uber with autonomous monitoring +
+ + ++ "At a scale of 1.5 million data points ingested per second, it started getting very expensive to monitor our metrics and we had to turn down our replication factor (RF) to 2 on Cassandra. With M3DB, we were able to bring RF back to 3 while also cutting down significantly on hardware / storage costs." +
+
+ + Uber +
++ June 2020 +
+ + ++ "We have 5000 domestic Walmart stores that need monitoring, so we wanted a centralized solution that was cheaper than the obvious alternatives and scalable...With M3, the advantages are very good performance on writes and queries, it’s cost effective versus alternatives, and it scales linearly." +
+
+ + System Telemetry Architect, Walmart Labs +
++ June 2020 +
+ + ++ "We needed a metrics stack to visualize, detect, and alert on the growing machine learning infrastructure that LinkedIn relies on to provide a best in class experience for its users." +
+
+ + SRE, LinkedIn +
++ Querying millions to billions of metrics with M3DB's inverted index +
+ + ++ "When querying millions or billions of metrics, you want something flexible and sublinear in speed as the query’s become longer and longer the more distinct values you have. This led us to the creation of M3DB’s inverted index." +
+
+ + CTO and Co-Founder of Chraonosphere, Former Tech Lead at Uber +
++ Putting billions of timeseries to work at Uber with autonomous monitoring +
+ + ++ "At a scale of 1.5 million data points ingested per second, it started getting very expensive to monitor our metrics and we had to turn down our replication factor (RF) to 2 on Cassandra. With M3DB, we were able to bring RF back to 3 while also cutting down significantly on hardware / storage costs." +
+
+ + Uber +
++ June 2020 +
+ + ++ "We have 5000 domestic Walmart stores that need monitoring, so we wanted a centralized solution that was cheaper than the obvious alternatives and scalable...With M3, the advantages are very good performance on writes and queries, it’s cost effective versus alternatives, and it scales linearly." +
+
+ + System Telemetry Architect, Walmart Labs +
++ June 2020 +
+ + ++ "We needed a metrics stack to visualize, detect, and alert on the growing machine learning infrastructure that LinkedIn relies on to provide a best in class experience for its users." +
+
+ + SRE, LinkedIn +
++ Querying millions to billions of metrics with M3DB's inverted index +
+ + ++ "When querying millions or billions of metrics, you want something flexible and sublinear in speed as the query’s become longer and longer the more distinct values you have. This led us to the creation of M3DB’s inverted index." +
+
+ + CTO and Co-Founder of Chraonosphere, Former Tech Lead at Uber +
+
+ 
+ 
+ 
+ 
+ 
+ 
+
+ + M3 is the obvious choice for Cloud Native companies looking to scale up their Prometheus based monitoring systems. M3 can be used as Prometheus Remote Storage and has 100% PromQL compatibility. +
++ M3 was originally developed at Uber in order to provide visibility into Uber’s business operations, microservices and infrastructure. With its ability to horizontally scale with ease, M3 provides a single centralized storage solution for all monitoring use cases. +
+
+
+ 
+ 
+
+ 
+ Proven at the largest scales in the world by storing 10s of billions of active metric time series.
+
+ Three replicas of data with quorum writes and reads for consistency.
+
+ Optimized compression algorithm resulting in 11X compression ratio.
+
+ Proven in production to ingest more than one billion datapoints per second while serving more than two billion datapoint reads per second.
+
+ Compatible with Prometheus, StatsD, and Carbon ingestion formats as well as PromQL, Graphite query languages.
+
+ Open sourced under the Apache 2 license with a highly active community. Contributions welcomed.
+
+ 
+ 
+ 
+