Skip to content

Hyak: Using Synology Storage

Synology network shares, and other SMB (windows file sharing) drives can be mounted on Hyak using the ‘rclone’ command.

Caveats:

  • Filesystems mounted this way will be limited by the network speeds between Hyak and the file server.
  • Additionally, executable files cannot be run from a filesystem mounted this way.
  • Each mount is available only to the current node. If you need to access data on multiple jobs you will need to mount the share each time a new node is launched, e.g. from your batch script

One-Time rclone Setup

The following steps must be completed once per user for ever file share you wish to access. Steps must be completed on an interactive compute node, as modules are not available on the Login nodes. All commands assume a user group of ‘coenv.’ Lines starting with ‘$’ are commands to be entered on the command line.

First, launch a new interactive node:

# Allocate a compute node with 4 CPUs and 8Gb of RAM for 2 hours
$ salloc -A coenv -p compute -N 1 -c 4 --mem=8G --time=2:00:00

Load the rclone module within the newly launched interactive node:

$ module load coenv/rclone

NOTE: The version of rclone that comes installed with Hyak does not support mounting of SMB file shares. The following step will fail if you don’t load this module.

Test that the correct module is loaded. The version should be higher than 1.61:

$ rclone --version
rclone v1.63.1
- os/version: rocky 8.5 (64 bit)
...

Configure your Synology/SMB file share as an rclone remote:

$ rclone config

Type ‘n’ and press enter:

Current remotes:

Name                 Type
====                 ====

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>

Choose a name for your file share (e.g. ‘Synology’) and press enter:

Enter name for new remote.
name>

You will be presented with a long list of options. Enter ’39’ for SMB or just type ‘smb’ and press enter:

Option Storage.
Type of storage to configure.Choose a number from below, or type in your own value.
...
39 / SMB / CIFS
   \ (smb)
...
Storage> 

Type in the server name or ip address of your file share/Synology box:

Option host.
SMB server hostname to connect to.
E.g. "example.com".
Enter a value.
host>

Enter your Synology Username and press enter:

Option user.
SMB username.
Enter a string value. Press Enter for the default (shrike).
user>

Leave this blank and press enter:

Option port.
SMB port number.
Enter a signed integer. Press Enter for the default (445).
port>

Type ‘y’ to save a password for this server. You will be prompted for the password twice, the password will not be printed to the screen:

Option pass.
SMB password.
Choose an alternative below. Press Enter for the default (n).
y) Yes, type in my own password
g) Generate random password
n) No, leave this optional password blank (default)
y/g/n>

Leave the ‘Option domain’ and ‘Option spn’ fields blank and press enter:

Option domain.
Domain name for NTLM authentication.
Enter a string value. Press Enter for the default (WORKGROUP).
domain>

Option spn.
Service principal name.
Rclone presents this name to the server. Some servers use this as further
authentication, and it often needs to be set for clusters. For example:
    cifs/remotehost:1020
Leave blank if not sure.
Enter a value. Press Enter to leave empty.
spn>

Type ‘n’ and press enter:

Edit advanced config?
y) Yes
n) No (default)
y/n>

Verify the information you entered is correct and then type ‘y’ and press enter:

Configuration complete.
Options:
- type: smb
- host: synology.environment.uw.edu
- pass: *** ENCRYPTED ***
Keep this "Synology" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>

Type ‘q’ and press enter to exit the configuration. You will be returned to a command prompt:

Current remotes:

Name                 Type
====                 ====
Synology             smb


e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>

Verify the configuration with the following command. You should see a directory listing that reflects the files and folders on your synology share. Note that the colon at the end of the share name is required:

$ rclone ls Synology:

Finally, Create a new directory to mount your data in. This should be in your home directory and will only be accessibly by your user:

$ mkdir $HOME/mnt

Mounting a Share with rclone

Mounting a Synology share with rclone required a newer version of rclone than the one that comes loaded on Hyak by default. The College of the Environment provides a newer version of rclone as a module. On a compute node, load this module with the following command:

$ module load coenv/rclone

NOTE: Modules are only available on compute nodes. This command will not work on the login node, and therefore you will not be able to access your SMB shares from the login node. If you need to work with files on a Synology or SMB share, first allocate an interactive compute node using the command above.

You will need a dedicated directory in your home directory to mount rclone shares. If you have not already created one, use the command:

$ mkdir $HOME/mnt

Verify your rclone remotes are set up with the command. If you do not see your server listed, complete the configuration steps above.

$ rclone listremotes
Synology:

To mount the ‘Synology’ remote on the local node use this command:

$ rclone mount Synology: $HOME/mnt --daemon

This command will mount the ‘Synology’ remote under the directory ‘mnt’ in your home directory. The ‘–daemon’ flag at the end is required in order to return you to your command prompt. Without that flag rclone will appear to hang. Also note the colon ‘:’ at the end of the share name. This is required for rclone.

After mounting, the files in your Synology share will appear in the directory: /mmfs1/home/<username>/mnt/Synology.