.. include:: Accessing Files on Pythia ========================== +----------------------+------------+---------------+---------------------------+------------------------------------------------------------------+ | Pythia Mount Point | Type | Managed Quota | Auto-deletion | Features | +======================+============+===============+===========================+==================================================================+ | ``/home`` | NFS | Yes | No | user home directories shared with Mercury | +----------------------+------------+---------------+---------------------------+------------------------------------------------------------------+ | ``/project`` | NFS | Yes | No | share project directories shared with Mercury | +----------------------+------------+---------------+---------------------------+------------------------------------------------------------------+ | ``/project_gpfs`` | GPFS | Yes | No | shared, performant, parallel-access, scalable, persistent | +----------------------+------------+---------------+---------------------------+------------------------------------------------------------------+ | ``/hpc_temp`` | Local NVMe | No | Yes, after job completion | high-speed, short-lived; suitable for demanding GPU applications | +----------------------+------------+---------------+---------------------------+------------------------------------------------------------------+ Long Term Storage ----------------- Home Directories !!!!!!!!!!!!!!!! Booth home directories are auto-mounted on Pythia upon login. User home directories on Pythia are the same home directories present on Mercury. These directories are private and cannot be shared. * :code:`/home/`: user's private home directory Project Shares !!!!!!!!!!!!!! Users may request that administrators mount their Booth shared project folders on Pythia. These directories are much larger than home directories (Terabytes) and are meant for storing large files associated with research projects. Faculty members can add/remove collaborators to these project shares as necessary. Shared project folders are the primary way that researchers can access large amounts of disk space. * :code:`/project/`: network group share directory Project GPFS Space !!!!!!!!!!!!!!!!!! Pythia has 120 TB of shared GPFS space in :file:`/project_gpfs`. There is a $1000/TB cost charged under a PI's account for using GPFS storage, see our storage policy on this `section`_ .. _section: https://pythia-docs.chicagobooth.edu/about.html#storage-management-policy When running GPU-accelerated workloads on an HPC cluster, data locality and I/O performance can significantly affect job runtime. To maximize GPU performance, it is recommended to copy input data to GPFS storage before running GPU-intensive jobs and to store output data there during computation. You can review our section on how to efficiently prefetch data to GPFS /project_gpfs/ `here`_ .. _here: https://pythia-docs.chicagobooth.edu/additional_resources.html#prefetch-data-to-gpfs .. note:: The GPFS project directory is only available on the compute nodes, not on the front end nodes. Short Term Storage ------------------ Local NVME !!!!!!!!!! The gpu nodes pgpu006-022, pgpu016-021 have 56TB of local NVME storage under :file:`/hpc_temp` Local NVME storage availability under :file:`/hpc_temp` varies across nodes, with pgpu006-011 and pgpu016-021 offering 56TB, while pgpu012-15 provide 1.5TB. This is the recommended place to store temporary job files such as transformed data, temporary logs, parallel job metadata or intermediate output. It is not to be used for file/data storage not related to running jobs. We recommend placing your scratch files within a personal folder organized by job number: :file:`/hpc_temp/${SLURM_JOB_USER}/${SLURM_JOB_ID}/`. Here, ${SLURM_JOB_USER} and ${SLURM_JOB_ID} are environment variables which you can query in your job script. Your code should automatically delete the job-specific files and folder upon successful completion. Files placed there will automatically be purged after 14 days. I/O Tools --------- Command line utilities !!!!!!!!!!!!!!!!!!!!!! A common way of transferring files to and from a remote cluster is to use the :code:`scp` command. .. code-block:: bash # copy a file to a local dir; use the same name for the file $ scp @pythia.chicagobooth.edu:/myfile.txt /. # copy a file to a local dir; use a different name for the file $ scp @pythia.chicagobooth.edu:/myfile.txt /renamed.txt # copy a directory using the recursive flag (-r) $ scp -r @pythia.chicagobooth.edu: If you need to check the space available in your home directory or project share, you can use the :code:`df -h` command. .. code-block:: bash # Check home directory space $ df -h /home// Filesystem Size Used Avail Use% Mounted on sisyphus-n.chicagobooth.edu:/ifs/home// 14G 7G 7G 50% /home/ # Check project share space $ df -h /project/ sftp !!!! Another common way of transferring files is via sftp. One software package that provides cross-platform support for sftp transfers is `Filezilla`_. Filezilla is an easy to use sftp client with a graphical user interface. The split window shows your local machine's directory tree on one side, and the remote files in the other. Filezilla's GUI allows simple drag-and-drop for transferring files. .. figure:: _static/filezilla.png :scale: 80 % Filezilla A connection to Pythia can be established by simply entering the following fields and clicking the Quickconnect button. :Host: :code:`sftp://pythia.chicagobooth.edu` :Username: :code:`` :Password: :code:`` .. _`Filezilla`: https://filezilla-project.org/ Mounting home directory on a Desktop !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Users who are connected to the Chicago Booth network (either through a wired ethernet connection or through the Booth VPN) can access files on Pythia by mapping to their home directory. To map your home directory: Windows ####### .. note:: If you are off campus or on a wireless connection, you must use Booth VPN to access files on Pythia. 1. Right-click on the My Computer icon. 2. Click on the Map Network Drive… link. 3. Choose a free drive letter from the Drive: dropdown. 4. Enter one of the following for the Folder field :PhD students: :code:`\\\\home.chicagobooth.edu\\home\\phd\\` :Faculty: :code:`\\\\home.chicagobooth.edu\\home\\fac\\` :Staff: :code:`\\\\home.chicagobooth.edu\\home\\staff\\` :Collaborator: :code:`\\\\home.chicagobooth.edu\\home\\collab\\` 5. Uncheck the Reconnect at logon checkbox. 6. Click on the Finish button. 7. At the prompt, enter your Chicago Booth credentials as follows, :code:`GSB\` and :code:``. Your Booth home directory will now appear in My Computer under Network Drives. .. note:: Project shares can similarly be mounted by using :code:`\\\\project.chicagobooth.edu\\` Mac ### .. note:: If you are off campus or on a wireless connection, you must use Booth VPN to access files on Pythia. 1. From the Finder menu, click on Go |rarr| Connect to Server... 2. Enter one of the following for Server Address :PhD students: :code:`smb://home.chicagobooth.edu/home/phd/` :Faculty: :code:`smb://home.chicagobooth.edu/home/fac/` :Staff: :code:`smb://home.chicagobooth.edu/home/staff/` :Collaborator: :code:`smb://home.chicagobooth.edu/home/collab/` 3. Click on the Connect button. 4. Click on the Registered User radio button. 5. Enter your Chicago BoothID and password 6. Uncheck the Remember this password in my keychain checkbox. 7. Click on the Connect button. Your Booth home directory will now appear on your desktop. .. note:: Project shares can similarly be mounted by using :code:`smb://project.chicagobooth.edu/` Snapshots and File Recovery --------------------------- Snapshots on Pythia are performed nightly for user home directories, project shares and GPFS. Nightly snapshots are available for 30 days for project shares and 15 days for GPFS storage and provides users with a short-term file versioning system. Files older than 30 days are not recoverable. Note that ".snapshot" is a hidden folder not visible by directory listing commands. Here are some example commands for using snapshots. .. code-block:: bash # Navigate to the hidden ".snapshot" folder $ cd /project/IT/.snapshots # View available snapshots taken in last 30 days $ ls project_30day_2026-04-25_23-45 project_30day_2026-04-24_23-45 project_30day_2026-04-23_23-45 project_30day_2026-04-22_23-45 # Restore a single file cp ~/.snapshot/project_30day_2026-04-22_23-45/file ~/target/folder/. # Navigate to the hidden GPFS ".snapshots" folder $ cd /project_gpfs/.snapshots/ # View available snapshots taken in last 15 days $ ls @GMT-2026.04.26-04.00.14 @GMT-2026.04.25-04.00.14 @GMT-2026.04.24-04.00.14