Enterprises concentrate on leveraging big data for valuable insights, but they don’t often do the same thing when it comes to analyzing their own systems. To diagnose an issue, operators are left to look in cumbersome log files which are extremely difficult to work with and don’t include all the right data. This session will describe a metrics platform that continuously collects operational data over time (near real-time data stored for days, weeks and months) to quickly provide visibility into the big data system and drill down for Spark problem diagnosis, tuning and chargeback reporting.
Matt McCline has deep expertise with database and transaction management systems. An Apache Hive PMC and Apache ORC committer, Matt has amassed decades of engineering and systems architecture experience with leading organizations including Hortonworks, Greenplum, Microsoft and Tandem Computers.