Science Gateways for Developers and Operators

This page documents required and recommended steps for developers. For additional assistance, ACCESS provides MATCH support services to assist gateway developers and administrators.

Science Gateways can democratize access to the cyberinfrastructure that enables cutting-edge science

What is an ACCESS Science Gateway?

An ACCESS Science Gateway is a web or application portal that provides a graphical interface for executing applications and managing data on ACCESS and other resources. ACCESS science gateways are community services offered by ACCESS users to their communities; each gateway is associated with at least one active ACCESS allocation. For an overview of the steps a gateway provider must take to start an ACCESS Science Gateway, see the Gateways for PIs page.

See the Science Gateways Listing for a complete list of current operational gateways.

Science gateway developers and administrators may include PIs as well as their collaborators, staff, and students. The PI should add these team members to the ACCESS allocation; see Manage Users for more details. It is recommended that the allocation have at least one user with the Allocation Manager role, in addition to the PI.

Operations Checklist

The PI obtains an ACCESS allocation.
The PI adds developer and administrator team members to the allocation.
Register the gateway.
Request for a community account to be added to the allocation. Details for requesting community accounts will be added shortly after ACCESS begins operations.
Add the ACCESS logo to the gateway.
Integrate the user counting scripts with the gateway's submission mechanism.
Join the ACCESS gateway community mailing list (optional).

Building and Operating

Science gateways can be developed using many different frameworks and approaches. General issues include managing users, remotely executing and managing jobs on diverse ACCESS resources, tracking jobs, and moving data between ACCESS and the user environment. ACCESS specific issues include tracking users, monitoring resources, and tracking use of the gateway allocation. For a general overview of best practices for building and operating a science gateway, please see the material developed by the Science Gateways Community Institute, an independently funded ACCESS service provider. The Institute provides support for different frameworks that can be used to build science gateways.

ACCESS supports a wide range of gateways and does not require specific middleware; gateways can use team-developed middleware or third party provided middleware. Gateways that run jobs and access data on ACCESS resources may be hosted on the PI's local servers or directly on ACCESS resources that support persistent Web services, middleware, and databases; these include Bridges, Comet, and Jetstream.

Managing User Accounts

ACCESS science gateways are community provided applications. Gateway users are not required to have ACCESS accounts or allocations. ACCESS allows all users jobs to run on the gateway's community account instead. Gateways thus map their local user accounts to the gateway's single community account. ACCESS does require quarterly reporting of the number of unique users who executed jobs on ACCESS resources, as described below.

ACCESS Community Accounts

ACCESS allows science gateways that run applications on behalf of users to direct all submission requests to a gateway community user account. Designated gateway operators have direct shell access to their community account, but normal users do not. The community account simplifies administration of the gateway, since the gateway administrators have access to input and output files, logs, etc, for all their users, and users don't need to request individual gateway accounts.

A community account has the following characteristics:

Only a single community user account (i.e., a ACCESS username/password) is created.
The Science Gateway uses the single ACCESS community user account to launch jobs on ACCESS.
The gateway user running under the community account has privileges to run only a limited set of applications.

Requesting a Community Account: Details for requesting community accounts will be added shortly after ACCESS begins operations.

Accessing Community Accounts: Administrators access community accounts through SSH and SCP using the community account username and password that is provided with the account. Details will be added shortly after ACCESS begins operations.

Community Accounts on Sites with Two-Factor Authentication: Some ACCESS resources, including Stampede and Wrangler, require two-factor authentication. Gateways can request exceptions to this policy for their community accounts by contacting ACCESS Help Desk. The gateway will need to provide the static IP addresses of the server or servers it uses to connect to the resource.

Unique Science Gateway User Accounts

It is the gateway developer's responsibility, as described below, to implement gateway logins or otherwise uniquely identify users in order to track usage. These accounts can be local to the gateway and do not need to correspond to user accounts on ACCESS. The gateway maps these accounts to the gateway's common community account.

Gateways may optionally choose to use ACCESS's OAuth2-based authentication process for authentication. This is a service provided by Globus Auth.

Connecting to ACCESS Resources

The most common type of ACCESS science gateway allows users to run scientific applications on ACCESS computing resources through a browser interface. This section describes ACCESS policies and requirements for doing this.

Community Allocations

Gateways typically provide their users with a community-wide allocation acquired by the PI on behalf of the community. The gateway may implement internal restrictions on how much of this allocation a user can use.

If a user is consuming an excessive amount of resources, the gateway may require these "power users" to acquire their own allocations, either through the Startup or XRAC allocation process. After obtaining the allocation, the user adds the gateway community account to her/his allocation. The user's jobs still run under the community account, but the community account uses the user's, rather than the gateway PI's, allocation. This is implemented by adding the allocation string to the batch script. This is the standard -A option for the SLURM schedulers used by many ACCESS resources.

Interacting with HPC Resources

Science gateways that run jobs on behalf of their users submit them just like regular users. For ACCESS's HPC resources, this means using the local batch scheduler to submit jobs and monitor them. Gateways execute scheduler commands remotely through SSH and use SCP for basic file transfer. Gateways may choose to work with third party middleware and gateway framework providers to do this efficiently. For more information on third party software providers, consult the Science Gateways Community Institute service provider web site.

ACCESS Resources for Gateway Hosting

ACCESS includes resources that have special Virtual Machine (VM) and related capabilities for gateways and similar persistent services. These resources are allocated through the standard ACCESS allocation mechanisms.

Bridges is designed for jobs that need large amounts of shared memory. It also has allocatable VMs that have access to Bridges' large shared file system. VM users can directly access scheduler command line tools to Bridge's computing resources inside their VMs.
Comet, like Bridges, is a computing cluster with co-located Virtual Machines. Users can also request entire, self-contained Virtual Clusters that can run both the gateway services and computing jobs.
Jetstream is an ACCESS cloud computing resource. Gateway users can get persistent VMs for use in gateway service hosting. They can also get multiple VMs configured as a Virtual Cluster with a private scheduler for running computing jobs.

Science Gateway Usage Metrics: Unique Users per Quarter

ACCESS requires all gateways to report the number of unique users per quarter who have executed jobs on ACCESS resources. This is a key metric that ACCESS in turn reports to the NSF. Compliance with this requirement justifies ACCESS's investment in the science gateway community. ACCESS collects this information through a simple API that is integrated into the job submission process.

View instructions, and materials from informational webinar on Gateway Attributes Reporting on Oct 1, 2019:

Security and Accounting

ACCESS has specific security and accounting requirements and recommendations for connecting to its resources to optimize your gateway for prevention and triage of security incidents or inadvertent misuse.

Security and Accounting Requirements and Recommendations

The following security and accounting steps are required.

Required: Notify the ACCESS Help Desk immediately if you suspect the gateway or its community account may be compromised.
Required: Keep Science Gateway contact info up to date on the Science Gateways Listing in case ACCESS staff should need to contact you. ACCESS reserves the right to disable a community account in the event of a security incident.
Required: Use the gateway_submit_attributes tool to submit gateway username with job.

Additional recommendations are as follows:

Collect Accounting Statistics
Maintain an audit trail (keep a gateway log)
Provide the ability to restrict job submissions on a per user basis
Safeguard and validate programs, scripts, and input
Protect user passwords on the gateway server and over the network
Do not use passwordless SSH keys.
Perform Risk and Vulnerability Assessment
Backup your gateway routinely
Develop an an incident response plan for your gateway; review and update it regularly
Put a contingency plan in place to prepare for a disaster or security event that could cause the total loss or lock down of the server
Monitor changes to critical system files such as SSH with tripwire or samhain (open source)
Make sure the OS and applications of your gateway service are properly patched - Run a vulnerability scanner against them such as nessus
Make use of community accounts rather than individual accounts

These are described in more detail below in separate sections. The Science Gateways Community Institute service provider also provides information on best practices.

What To Do In Case of a Security Incident

Whether a threat is confirmed or suspected, quick action and immediate communication with ACCESS Security Working Group is essential. Please contact the ACCESS Help Desk immediately.