by Jemshad O K (@jemshad) on Wednesday, January 29, 2014

+18
Vote on this proposal
Status: Submitted
Section
Full talk

Technical level
Intermediate

Objective

Explains how to scale graphite/carbon from it's default few thousand updates/min. to handle around 500K updates/min.

Description

Graphite/Carbon as a metrics collection and graphing solution

Collecting metrics and being able to represent them graphically in an efficient, scalable way is a key part in making your application clusters more robust in terms of both scale and performance. Graphite with it's carbon backend is an excellent tool for such a purpose given the rich set of mathematical transformations that it supports on the underlying data. Though graphite as a frontend worked well, the carbon backend had it's own limitations when coming to the data storing and retrieval part.

Topics covered

The session would cover little bit on graphite compared to other metrics collection and graphing solutions out there and then we move on to the specifics of using graphite at scale. We would go through various issues we faced, especially with it's datastore, and how we solved them at InMobi.

Requirements

  • Basic Understanding of Graphite
  • Some idea about Round Robin Databases and Time Series Databases

Speaker bio

  • Tech Lead, InMobi

http://www.linkedin.com/in/jemshad

Comments

  • 1
    [-] Pradeep Chhetri (@pradeepchhetri) 2 years ago

    You mentioned 500K updates/min. Is it on a single machine or across a cluster ? If cluster, then how many machines ? Do they run on SSDs ? Will you be talking about removing any single points of failure in your complete architecture?

    • 1
      [-] Jemshad O K (@jemshad) 2 years ago

      Hi Pradeep,

      500K updates is on a single node with normal Hard Disks. Removing SPOF is not really in the scope of this particular talk as this is more about performance tuning of graphite.

  • 1
    [-] Pradeep Chhetri (@pradeepchhetri) 2 years ago

    Nice. May I know what kind of node is this: Physical/Virtual ?

  • 1
    [-] Ranjib Dey (@ranjibd) 2 years ago

    grpahite is already popular with extensive documentation and integrations. is there any thing specifically you want to cover which can not be found in internet easily (like scaling graphite installation, relay server etc). For introductory content, it might be better to have this as workshop against a particular tech stack (like how to use graphite with LAMP deployments)

Login with Twitter or Google to leave a comment