Use Globus to upload, download, or move files on ACCESS
- ACCESS
Once you are logged in to Globus, the File Manager interface is self-explanatory and you will probably understand how to use it without further help. If you need some hints or further details, the Globus website provides an excellent Getting Started guide. The next few sections explain how to find ACCESS's systems within Globus so you can move data to and from them, how to upload and download files, and how to set up your local system (personal or campus computer) for larger transfers that run in the background.
Find ACCESS resources
Globus lets you move files to and from collections. Almost all of the ACCESS resources have Globus collections for the storage available to authorized researchers. To find the collection for an ACCESS resource, just click the Collection Search box at the top of the File Manager, type the resource’s name and the word "ACCESS," and the resource’s ACCESS collections will appear, as shown in Figure 1. (Make sure the collection you pick has the green "Greek columns" icon that signifies the collection is owned by a Globus-supporting organization.) If the resource you are looking for is especially new or is no longer online and you can't find it in the list, use the ACCESS Help Desk to inquire about its availability. Select the collection you want to move files to or from, and the File Manager will allow you to browse the files and folders there.
Find your local computers
You can easily create a Globus collection on your own computer so you can transfer lots of files or very large files to or from your computer in the background while you do other work. To do this, download and install the Globus Connect Personal software. (Globus Connect Personal is available for Windows, Mac, and Linux systems.) Globus Connect Personal runs as a background task, and you can start and stop it like any other application. While Globus Connect Personal is running, your system will be available in Globus's File Manager interface (but only to you!) and you will be able to start transfers between it and any other Globus collection, including ACCESS resources.
You can start and stop Globus Connect Personal on your system whenever you need to, even shutting your system down, hibernating, and moving from one network to another. Globus will automatically find your system and continue any active transfers whenever you are connected to a network and have the tool running. Globus Connect Personal is designed to work with most firewalls and NAT devices. It does not require administrative privileges to run on your system.
Find other research systems
Many colleges, universities, research institutions, and laboratories offer Globus collections for their systems. To find a collection provided by an organization, try typing the name of the organization in the Collection Search box at the top of the File Manager page. You may be surprised how many organizations are already listed and available. To confirm the organization that provides a collection, look for the green "Greek columns" icon that signifies the collection is owned by a Globus-supporting organization. Also check the "Owner" field in the listing. You'll need to be authorized by the organization to access its collections.
Any Linux-based server system can host Globus collections by installing Globus Connect Server. On most servers, installing Globus Connect Server requires just a few commands. Once the software is installed, anyone with a local account on the server can move and share files between it and other Globus collections, including ACCESS resource collections. Documentation for Globus Connect Server is available on the Globus website.
Upload and download files
Nearly all ACCESS resources allow you to upload or download files with your web browser. When viewing a folder on an ACCESS resource in the File Manager, the Upload icon will be active. (See Figure 2.)
Click the Upload button and your browser will open a file selection dialog to choose the file(s) you want to upload. You may select multiple files, but you may not select folders. Once you've selected the files to upload, Globus will upload the files to the folder you are viewing in the File Manager. See Figure 3.
You can also download files to your computer. When viewing a folder in the File Manager, select a single file by clicking its name. (Download is currently only available for single files.) The Download button will activate if this feature is available on this ACCESS server. See Figure 4.
When you click the Download button, you might need to re-authenticate. Follow the prompts, and your file will download to your computer as shown in Figure 5. You may then open it in any application.
If the Upload and Download buttons do not activate as described above, the ACCESS resource you are using hasn't been updated to the latest Globus software. You can use Globus Connect Personal to transfer files to/from this resource (see above "Find Your Local Computers") or request an upgrade by submitting a support ticket.
Data integrity
All Globus transfers between two servers (including a Globus Connect Personal server on your own computer) automatically use advanced checksums to verify file integrity during and after each transfer. If an integrity check fails, Globus will automatically re-transfer the file. You can disable this feature using the "Transfer & Sync Options" interface (see Figure 6), but it is enabled by default. IMPORTANT: Browser uploads and downloads do not include integrity checks.
Data transfer performance
The key principle to understand is that any given data transfer can only perform as well as the slowest component involved in the transfer. It may be a single slow network link, the disk drive on your laptop, or an overloaded file system on a supercomputer, but regardless, the overall transfer can perform no faster than the slowest component in the chain. For example, for most transfers into an ACCESS system coming from a desktop or laptop system with a fast network connection, the slowest component will be the disk drive on the desktop/laptop.
It is also important to remember that ACCESS resources are shared resources, so it will rarely be the case that your transfer has all of the capacity available to it. Your transfer will be sharing bandwidth with other users, other applications, and other systems. The file systems on all ACCESS resources are also shared, and the total storage and network capacity will be divided among both data transfers and running applications at all times.
The Energy Sciences Network (ESnet) has put together a nice summary of what to expect for cross-country data transfer performance. Notably, under ideal conditions (if you aren’t sharing resources with anyone else), a transfer of 1 terabyte (1 million megabytes) will take three hours over a 1 Gbps network connection, and 20 minutes over a 10 Gbps connection. The connections between most ACCESS resources are currently 10 Gbps, but as noted above, these network connections and the storage systems at both ends are shared with other researchers using those resources. University campus connections are typically 1 Gbps or 10 Gbps, but sometimes less. Home network connections are measured in Mbps (1000 Mbps = 1 Gbps).
If your data transfer performance is significantly less than what you expect, you may wish to check the performance of the network to which your laptop or desktop is connected. There are many freely available bandwith-checking utilities on the web that can give you practical readings, and your local IT administrator can give you details of the maximum performance achievable on your local network. To discuss data transfer performance with experts, open a support ticket.