Spark Master

Introduction

Here we'll go over the Spark Master.

Requirements 

Spark Master Data Sources
 
Open up the configuration modal for Spark, and click on the Configure Data Sources tab.  In the Add a data source drop down menu [1], select Spark Master.  Click on Edit [2] to the right to expose the fields that need to be configured [3]:
 

SparkMasterConfig.png

 
The plugin requires the following information:
 
Field Name Default Value Description
Host _HOST:8080_  
Port 8080  
UrlPath /metrics/master/json  
Run as Unix User nobody  

 

Spark Master Metrics

 

spark_master jvm.gc.copy.count Count null
spark_master jvm.gc.copy.time Count null
spark_master jvm.gc.mscompact.count Count null
spark_master jvm.gc.mscompact.time Milliseconds null
spark_master jvm.heap.committed Bytes null
spark_master jvm.heap.init Bytes null
spark_master jvm.heap.max Bytes null
spark_master jvm.heap.usage Bytes null
spark_master jvm.heap.used Bytes null
spark_master jvm.nonheap.committed Bytes null
spark_master jvm.nonheap.init Bytes null
spark_master jvm.nonheap.max Bytes null
spark_master jvm.nonheap.usage Bytes null
spark_master jvm.nonheap.used Bytes null
spark_master jvm.pools.codecache.committed Bytes null
spark_master jvm.pools.codecache.init Bytes null
spark_master jvm.pools.codecache.max Bytes null
spark_master jvm.pools.codecache.usage Bytes null
spark_master jvm.pools.codecache.used Bytes null
spark_master jvm.pools.edenspace.committed Bytes null
spark_master jvm.pools.edenspace.init Bytes null
spark_master jvm.pools.edenspace.max Bytes null
spark_master jvm.pools.edenspace.usage Bytes null
spark_master jvm.pools.edenspace.used Bytes null
spark_master jvm.pools.permgen.committed Bytes null
spark_master jvm.pools.permgen.init Bytes null
spark_master jvm.pools.permgen.max Bytes null
spark_master jvm.pools.permgen.usage Bytes null
spark_master jvm.pools.permgen.used Bytes null
spark_master pools.survivorspace.committed Bytes null
spark_master jvm.pools.survivorspace.init Bytes null
spark_master jvm.pools.survivorspace.max Bytes null
spark_master jvm.pools.survivorspace.usage Bytes null
spark_master jvm.pools.survivorspace.used Bytes null
spark_master jvm.pools.tenuredgen.committed Bytes null
spark_master jvm.pools.tenuredgen.init Bytes null
spark_master jvm.pools.tenuredgen.max Bytes null
spark_master jvm.pools.tenuredgen.usage Bytes null
spark_master jvm.pools.tenuredgen.used Bytes null
spark_master jvm.total.committed Bytes null
spark_master jvm.total.init Bytes null
spark_master jvm.total.max Bytes null
spark_master jvm.total.used Bytes null
spark_master workers.alive.count Count Number of workers that are alive in the cluster
spark_master total.app.count Count Number of apps running including waiting ones
spark_master waiting.app.count Count Number of apps waiting to be run
spark_master total.workers.count Count Total number of workers that are supposed to be in this cluster. See workers_alive_count to see current number of workers alive

 If you have any questions or comments about this article, feel free to contact us at support@opsclarity.com.

Have more questions? Submit a request

0 Comments

Article is closed for comments.