Loading…
DevConf.US '18 has ended
DevConf.us 2018 is the 1st annual, free, Red Hat sponsored technology conference for community project and professional contributors to Free and Open Source technologies held at the Boston University in the historic city of Boston, USA.

When: Friday, August 17 to Sunday, August 19, 2018

Venue: Boston University, George Sherman Union Building
Back To Schedule
Saturday, August 18 • 10:20am - 10:55am
Scalable Monitoring Using Prometheus with Spark

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Feedback form is now closed.
As spark applications move to a containerized environment, there are many questions about how to best configure server systems in the container world. In this talk we will demonstrate a set of tools to better monitor performance and identify optimal configuration settings. We will demonstrate how Prometheus, a project that is now part of the Cloud Native Computing Foundation (CNCF), can be applied to monitor and archive system performance data in a containerized spark environment. In our examples, we will gather spark metric output through Prometheus and present the data with Grafana dashboards. We will use our examples to demonstrate how performance can be enhanced through different tuned configuration settings.
Speakers
avatar for Diane Feddema

Diane Feddema

Principal Software Engineer, AI/ML Performance on RHEL and OpenShift Operator Development, Red Hat
Diane Feddema is a principal software engineer at Red Hat Inc, in the Performance and Scale team. Diane is currently focused on developing and applying machine learning techniques for performance analysis using hardware accelerators, automating these analyses and displaying data in... Read More →
avatar for Zak Hassan

Zak Hassan

Senior Software Engineer - AI/ML CoE, CTO Office, Red Hat Inc.
Currently focused on developing analytics platform on OpenShift and leveraging Open Source ML Frameworks: Apache Spark, Tensorflow and more. Designing high performance and scalable ML platform that exposes metrics through cloud-native technology: Prometheus and Kubernetes.

Saturday August 18, 2018 10:20am - 10:55am EDT
Conference Auditorium Boston University, George Sherman Union Building
  Containers and orchestration, Presentation

Attendees (47)