Monitoring
==========

General documentation on monitoring Spark jobs can be found at: 
https://spark.apache.org/docs/latest/monitoring.html

Command Line
------------

A listing of all Spark applications can be found along with their Application ID by typing:

:code:`yarn application -list`

To cancel a job, users can issue the following command with the Application ID obtained above.

:code:`yarn application -kill <Application ID>`


Web UI
------

.. note:: Accessing the web UI requires you to be connected to the Booth VPN vpn.chicagobooth.edu

Spark history server
!!!!!!!!!!!!!!!!!!!!

https://vulcan-spark-history.chicagobooth.edu/

YARN
!!!!

Apache Hadoop YARN is the agent responsible for managing resources on Vulcan.
The following URL provides a listing of YARN applications where users can find their application logs.

https://vulcan-jobs.chicagobooth.edu

.. Details on the active YARN configuration can be found in XML format at the following URL:
.. http://hdpmn01.chicagobooth.edu:8088/conf