⚠️ 2021-11-01: The TIPCC cluster is retired. If you're looking for a modern cluster, please see our new C4 cluster, which is in production since January 11, 2021.

Access UCSF Box

Although there is no hard quota in place on UCSF Box, the storage is limited and not really meant to store large data sets. There are reports that the UCSF Box maintainers have started to reach out to users who misuse it to store huge amounts of data. Please use it responsibly. /2020-01-27

It is possible to access UCSF Box using FTP over a secure SSL connection (FTPS).

In order to do this, you need to setup a UCSF Box-specific password as explained in https://ucsf.app.box.com/services/box_ftp_server. Afterward, you can use, for instance, the lftp client to verify access to your account;

$ lftp --user alice.aliceson@ucsf.edu ftps://ftp.box.com
Password: XXXXXXXX
lftp alice.aliceson@ucsf.edu@ftp.box.com:~> ls
drwx------  1 owner group     0 Jun 12  2014 Grant_R01.pdf
drwx------  1 owner group     0 Sep 30  2016 Secure-alice.aliceson@ucsf.edu
lftp alice.aliceson@ucsf.edu@ftp.box.com:~> exit
$ 

Never specify your password via a command-line argument! If you do, it will be visible to all other users via commands such as ps and htop.

Automatic authentication

When starting lftp as above, you need to manually enter your password, which can be tedious or even prevent automatic file transfers in batch scripts. A solution to this is to set up the FTP credentials in ~/.netrc. Here is what it could look like:

$ cat ~/.netrc
machine ftp.box.com
	login alice.aliceson@ucsf.edu
	password AliceSecretPwd2017

The ~/.netrc file must be kept private, otherwise its content could be readable to other users.

Since the password is fully visible in plain text, make sure to keep this file private at all times, otherwise users on the system can see all your credentials, i.e.

$ chmod 600 ~/.netrc
$ ls -l ~/.netrc 
-rw------- 1 alice alice 72 Jul  3 15:10 /home/alice/.netrc

To verify that the automatic authentication works, try to log in again. You should no longer be prompted for your password - instead lftp gets it automatically from ~/.netrc. For example:

$ lftp --user alice.aliceson@ucsf.edu ftps://ftp.box.com
lftp alice.aliceson@ucsf.edu@ftp.box.com:~> ls
drwx------  1 owner group     0 Jun 12  2014 Grant_R01.pdf
drwx------  1 owner group     0 Sep 30  2016 Secure-alice.aliceson@ucsf.edu
lftp alice.aliceson@ucsf.edu@ftp.box.com:~> exit
$ 

Note that curl also recognizes ~/.netrc credentials, e.g.

$ curl --netrc -O ftps://ftp.box.com/Grant_R01.pdf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15.6M  100 15.6M    0     0  1561k      0  0:00:10  0:00:10 --:--:-- 3918k
$ ls -la Grant_R01.pdf
-rw-r--r-- 1 alice cluster 16453180 Jul 10 21:13 Grant_R01.pdf