Rootconf 2015

DevOps and scaling infrastructure

How LinkedIn is leveraging kafka for log aggregation and monitoring

Submitted by Muzammil Rehman (@muzammilr) via akbar (@akbar) on Wednesday, 15 April 2015

videocam_off

Technical level

Intermediate

Section

Full talk

Status

Confirmed & Scheduled

View proposal in schedule

Vote on this proposal

Login to vote

Total votes:  +9

Objective

Understanding log aggregation and monitoring for a large scale web infrastructure.

Description

Apache Kafka is publish-subscribe messaging rethought as a distributed commit log. It is used within LinkedIn for metrics and intra-application queuing.
Will give an overview on how we leverage kafka for the log aggregation (ELK) and service monitoring.

Speaker bio

Muzammil Rehman Mohammed
I am SRE Manager at LinkedIn with 8 years of industry experience. ELK and kafka are integral part of my day to day activity, apart from my regular responsibilities I build and manager ELK cluster for various team.
I am a technology enthusiasts and passionate on developing large scale infrastructure, I believe complex and large operations don’t necessarily have to be complex, simplicity is the best way.

Links

Slides

http://www.slideshare.net/akbarabi/rootconf-47285373

Comments

Login with Twitter or Google to leave a comment