Auto mount NFS share#
All Linux servers and desktops in GEO domain have transparent access to NFS users profiles and research storage volumes with help of NIS, YP and Automount.
You can get list of mount points available at your computer:
ypcat -k auto.master
/data yp:auto.data
/software yp:auto.software
/Volumes yp:auto.volumes
/Users/home yp:auto.home
It means that information about /data
mount point is available via auto.data
binding, /software
- via auto.software
and so forth.
ypcat -k auto.data
gis -fstype=nfs resstoconfs0001.res.vuw.ac.nz:/GeoColridge_01/gis
sabine -fstype=nfs resstoconfs0001.res.vuw.ac.nz:/GeoSabine_01
hawea -fstype=nfs resstoconfs0001.res.vuw.ac.nz:/GeoHawea_01
First column is share alias, second - mount options, third - path to network share
To make the share appear you just need to get there using any command e.g. ls, cd, df, :
ls /data/gis
To save network resources all volumes will automatically unmount after 5 minutes timeout.
Transfer data#
You might need to work with large data sets on research storage. Uploading or downloading them can take a long time. It’s important to choose the proper tools. Please avoid of using any GUI file managers: their indexing and progress services may result to massive performance degradation on data transfer.
For data location.
Home profiles are mapped to /Users/home/
, research storage - to /Volumes/
and /data/
.
Please use ypcat
on the server to find out where the particular volume is located.
SSH#
To transfer some small amount of data (few files) you might simply run
scp my_local_file USERNAME@sgees001.geo.vuw.ac.nz:/path/to/destination/
Otherwise please use a more reliable instruments, like rsync. In very short, syntax is rsync [FLAGS] <FROM> <TO>
. For more details please refer to documentation.
A typical rsync command might look as:
rsync -av --progress --log-file=rsync.log \
/media/portable_hdd/field_trip_data/ \
USERNAME@sgees001.geo.vuw.ac.nz:/Volumes/GeoPhysics_42/users-data/USERNAME/field_trip_data/
NFS#
Once mapped, NFS volume appears as a regular directory, then you can use any tools for copying or moving your data across.
Globus#
Gigabytes, terabytes, petabytes—research data is large and distributed. Globus lets you efficiently, securely, and reliably transfer data directly between systems separated by an office wall or an ocean. Focus on your research and offload your data transfer headaches to Globus.
Get unified access to your research data, across all systems, using any existing identity. Laptop? HPC cluster? Cloud storage? Tape archive? Access them all using just a web browser. Data stored at a different institution? At a supercomputing facility? All you need is your campus login.
Globus client.#
You need to install and configure a copy of client software on each end.
Linux GUI (and CLI, see the docs): https://downloads.globus.org/globus-connect-personal/linux/stable/globusconnectpersonal-latest.tgz
Windows GUI: https://downloads.globus.org/globus-connect-personal/windows/stable/globusconnectpersonal-latest.exe
MacOS GUI: https://downloads.globus.org/globus-connect-personal/mac/stable/globusconnectpersonal-latest.dmg
CLI (Python, any O/S): https://github.com/globus/globus-cli
Linux version doesn’t require root privileges:
wget https://downloads.globus.org/globus-connect-personal/linux/stable/globusconnectpersonal-latest.tgz
tar xf globusconnectpersonal-latest.tgz
cd globusconnectpersonal-3.2.2
./globusconnect
Web interface (dashboard):#
- Navigate to https://app.globus.org/
- Use your existing organizational login
- Pick up ‘Victoria University of Wellington’
- Continue with your staff credentials
Here you can start/finish transfer and monitoring the process, configure connections e.t.c…
Documentation: https://docs.globus.org/ #
Good start point is here: https://docs.globus.org/how-to/get-started/
…then read the following:
https://www.globus.org/globus-connect-personal
https://docs.globus.org/how-to/globus-connect-personal-linux
RSYNC daemon#
We’ve got rsyncd
running on several servers - this is really helpful when you need to transfer a bunch of files.
Please contact computing system administrator for more details.
VPN#
If you’re outside the campus, you still may connect to any of the servers with VPN
Remote Session with X2Go#
X2Go is a Remote Desktop solution, which some vendors vaguely call Remote Control. This is not to be confused with Microsoft Remote Desktop Connection, which is a competing Remote Desktop solution and protocol.
With X2Go, you mainly access Linux computers. The Linux computer you are accessing is the X2Go Server. (There are ways to connect to a Windows computer as well; some of them are only available with the Linux X2Go Client.)
You can connect from a computer running Linux, Windows or Mac OS X. This computer is the X2Go Client.
SGEES Linux machines already have X2Go client installed. For Windows or MacOS download and install client here
All details about x2Go connection setup are here