Accessing Windows-file-shares

If you want to use data that is stored on file-shares (netshare or zidshare or similar) on our HPC clusters there are at least two methods available. Both are meant to be used on the login-nodes of our systems and are not available/recommended to be used on worker nodes.

smbclient

The smbclient tool offers an FTP-like interface to file-shares in order to interactively copy files from there to your scratch-folder on any of our clusters (or back).

The recommended way of working with smbclient is to first change to the directory where you want to copy the files and directories to (or from). Then run the tool like this:

smbclient -U UIBK/$USER //zidshare/<share-name>

You have to replace "<share-name>" with the name of your share and potentially "zidshare" with a different name in case your share is on another "server".
If you want to connect with a different user, please replace "$USER" with the appropriate username (c-number).

When you are connected you will see a smb: \> prompt.
You can use the following commands (file completion with <TAB> is working remotely and locally):

ls or dir to see the content of remote directories.
cd <remote-directory> to change directories remotely.
lcd <local-directory> to change the current local directory.
get <remote-file> to copy a remote file to the current local directory.
mget <remote-pattern> to copy all remote files and directories matching the pattern (recursively) to the current local directory.
put <local-file> and mput <local-pattern> to upload a single or many local files (and directories recursively).
exit to close the connection.

smbnetfs

You can not use smbnetfs on the worker-nodes (i.e. in a job)! If you need to work with files on a Windows-file-share in a job you first have to copy them into a sub-directory of your scratch-folder on the login-node!

Important Caveat

Using smbnetfs requires you to put your password in clear into a file, which is not good practice. The file is secured by standard Linux permissions, so the problem is limited to admin-users (aka "root"). But they will potentially be able to read it! The preferred way for accessing file-shares is using smbclient as mentioned above.

Preparations

If you want to use smbnetfs you have to make some preparations (only once per login-node) in order to create the necessary folders and the configuration file:

 mkdir -p ~/.smb ~/netfs; chmod 700 ~/.smb
 conf="$HOME/.smb/smbnetfs.conf"
 touch $conf
 chmod 600 $conf
 echo "auth \"zidshare.uibk.ac.at\" \"UIBK/$USER\" \"<password>\"" >> $conf
 echo 'host zidshare.uibk.ac.at visible=true' >> $conf

Please replace <password> with the actual password you use for our general services!

If you want do change or extend the configuration (which is a simple text-file) you can of course also use an editor: e.g.

nano ~/.smb/smbnetfs.conf

You can e.g. add these two lines to get access to your Windows-Nethome (drive letter I:):

 auth "nethome.uibk.ac.at" "UIBK/<yourusername>" "<password>"
 host nethome.uibk.ac.at visible=true

Starting smbnetfs

For getting access to the files you have to run

 smbnetfs ~/netfs

After this you will see a directory structure in ~/netfs that looks similar to this (which is the output of tree -L 2 ~/netfs):

netfs
├── nethome.uibk.ac.at
│   ├── ~
│   ├── <yourusername>
│   └── home
└── zidshare.uibk.ac.at
    ├── share
    └── user

As you see there is a virtual sub-directory for every server you are using and every share within a server. In this particular example the "share" sub-folder of "zidshare.uibk.ac.at" contains the data that in Windows normally is visible in drive J: and "user" contains the data of K:. The Windows-drive I: is accessible in the subfolder "~" or "<yourusername>" of "nethome.uibk.ac.at".

To copy files to and from the network share you can simply use "local" copy commands. E.g.

cp -r ~/netfs/zidshare.uibk.ac.at/user/tmp/some/folder $SCRATCH/some/other/folder/

Stopping smbnetfs

The files and folders stay accessible until the login-node is rebooted or until you stop smbnetfs by running

 fusermount -u ~/netfs