Skip to content
My.Environment

Kopah: Bulk Object Storage for Hyak

UW IT provides an object storage service for bulk storage that is compatible with S3. Object storage is not analogous to ‘file storage’ in that objects are lumped into ‘buckets’ that are non-hierarchical. Objects can be stored and retrieved, but cannot be modified in-place. This model is efficient for storing large datasets that are static, but presents some limitations compared to file system storage.

Kopah Access

The College of the Environment currently does not have a Kopah storage subscription. Kopah pricing and Account information is available from the Research Computing Club’s Kopah Documentation site. If you are interested in purchasing Kopah storage, please Submit an ESITS Help Request.

Kopah GUI Access: Cyberduck

Cyberduck is an open-source client for managing and transferring files to various cloud storage services and protocols, including FTP, SFTP, and S3. It offers a simple, intuitive interface that allows users to easily upload, download, and organize their files. Cyberduck is available for both Windows and macOS, making it a versatile tool for seamless file management across different platforms.

To get started with Cyberduck, install the software from the developer’s website on your local computer.

Steps for connecting to Kopah:

  1. For ease of use, a pre-configured connection profile for Kopah is available. Click this link to Download the profile. After the file is finished downloading, change the filename to kopah.cyberduckprofile. You may decide to move this file from your Downloads directory and store it in a more peranent location. Once the file is in its final location, double-click it within a file explorer it will open in Cyberduck.
  2. Open a new Cyberduck window and locate the Open Connection Icon. Image shows that the needed "Open Connection" button is located in the top taskbar in Cyberduck
  3. You should see your profile to connect to Kopah S3 showing in the drop-down box. Enter you Access Key and Secret Key, which can be found in your Home directory on klone. When you requested your account the Kopah team deposited your Kopah keys, in your Home directory.
Image shows to click on drop menu of page and to select the option SFTP(SSH File Transfer Protocol) which is shown as the third option
  1. Once that is complete, Kopah S3 will appear on your Cyberduck bookmarks or server list. Connect by double clicking on the Kopah S3 bookmark.
  1. You can now interact with Kopah using the GUI! Buckets can be created by right clicking and selecting New Folder…. Objects can be added to bucket by dragging and dropping files and directories into the buckets.

In the example above there are two buckets, bucket1 and bucket2, each with one file.

Kopah CLI Access: s3cmd

s3cmd is a free command line tool and client for uploading, retrieving and managing data in Kopah. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and if automated backup to Kopah is desired.

s3cmd is available for uploading data to Kopah from your local computer and with usage on klone.

Local s3cmd usage

To get started with s3cmd, install the software on your local computer. Click here to Download s3cmd from the developer’s website.

Create an s3cmd configuration file in your home directory. Call it .s3cfg. There are many ways to create this file.

For example, Mac and Linux users you can use the text editor nano in a Terminal window.

cd ~
nano .s3cfg
## Use Ctrl + X to exit nano

Windows users could use Wordpad or another text editor application.

.s3cfg should contain the following details:

.s3cfg

[default]
host_base = s3.kopah.uw.edu
host_bucket = s3.kopah.uw.edu/%(bucket)
use_https = True
public_url_use_https = True
# Login credentials
access_key = <ACCESS_KEY>
secret_key = <SECRET_KEY>

Where the word <ACCESS_KEY> is replaced with your Kopah Access Key and the word <SECRET_KEY> is replaced with your Kopah Secret Key.

After that is complete. s3cmd can be used to access your Kopah storage data with a large suite of commands. The s3cmd help includes example commands for a variety of tasks.

s3cmd --help

The following are a small collection of the many commands available with s3cmd.

command action
s3cmd mb s3://BUCKET make bucket
s3cmd ls [s3://BUCKET[/PREFIX]] List objects or buckets
s3cmd la List all object in all buckets
s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX] Put a file into the bucket
s3cmd put --acl-public FILE [FILE...] s3://BUCKET[/PREFIX] Put a file into a bucket and make it public
s3cmd get s3://BUCKET/OBJECT LOCAL_FILE Get a file from the bucket
s3cmd setacl --acl-private s3://BUCKET/OBJECT Make an object in the bucket private.