We review the Globus Connect Server v5 (GCSv5) architecture and deployment model, and describe the process for creating a Globus endpoint on your HPC cluster, lab server, or other multi-user storage system. You will experiment with installing Globus Connect Server, and configuring basic options on the endpoint.

1. Introduction to Globus for System
Administrators
Brigitte Raumann
braumann@uchicago.edu
Case Western Reserve University
October 23, 2023

2. Our focus in this session
• Step by step tutorial on how to make your multi-user
storage accessible by your users via Globus
• Overview of configuration options and best practices

3. Globus Connect
Server v5 Overview

4. Globus Connect Server v4
will be discontinued
December 18, 2023
Migration tools are available, please
update!

8. Globus Connect Server v5 Architecture

9. Globus Connect Server v5 Architecture

10. Globus Connect Server v5 Architecture
Goal of
today’s
tutorial

11. Globus Connect Server
install walkthrough
docs.globus.org/globus-connect-server
docs.globus.org/globus-connect-server/v5.4/quickstart

12. Globus Connect Server
install walkthrough
Key Prerequisite: Network Accessibility
docs.globus.org/globus-connect-server/v5.4/#open-tcp-ports_section
Port 443
must be publicly routable
Ports 50000-51000
used only during transfers as needed
can be on private net

13. Globus Connect Server
install walkthrough
Preliminaries
Satisfy technical prerequisites
Install GCS packages on your server
Deploy Globus Connect Server
1. Set up the endpoint
2. Add data transfer node(s) to the endpoint
3. Create a POSIX storage gateway
4. Create a mapped collection

14. Configure subscription features
5. Associate endpoint with a subscription
6. Create a guest collection to enable data sharing
7. Enable browser down/upload (HTTPS access)
8. Add non-POSIX storage systems to the endpoint
Globus Connect Server
install walkthrough

15. GCS v5 install walkthrough
Preliminaries
Satisfy technical prerequisites
Install GCS packages on your server
Already done on your servers.

16. GCS v5 install walkthrough
Install GCS packages on your server
$ curl -LOs http://downloads.globus.org/globus-connect-
server/stable/installers/repo/deb/globus-repo_latest_all.deb
$ dpkg -i globus-repo_latest_all.deb
$ apt-key add /usr/share/globus-repo/RPM-GPG-KEY-Globus
$ apt-get update
$ apt-get --assume-yes install globus-connect-server54
Already done on your servers.
docs.globus.org/globus-connect-server/v5.4/#install_section

17. Claim your server
1. Log into Globus at
app.globus.org
2. Select “claim your
virtual machine” at
bit.ly/gw-tut
3. Enter your name,
email address, and
identity used to log
into Globus then note
the number in your
server DNS name
bit.ly/gw-tut

18. SSH into your server
1. Open a terminal window on your laptop
2. $ ssh admin<#>@tut<#>.globusdemo.org
$ Passwd:
$ ssh admin3@tut3.globusdemo.org
Example
# of your DNS server
from the spreadsheet
Commands at
bit.ly/gw-tut

19. 19

20. 1. Create the Endpoint
$ globus-connect-server endpoint setup
> "My Endpoint"
> --organization "My Organization"
> --contact-email me@uchicago.edu
> --owner me@globusid.org
Commands at
bit.ly/gw-tut
docs.globus.org/globus-connect-server/v5.4/#create_the_endpoint

21. What does endpoint setup do?
• Creates your endpoint in the Globus services
• Creates the endpoint in Globus Transfer service
• Registers a Globus Auth client
• Registers a domain name of endpoint
• Obtain host certificate for interaction with endpoint
• Writes deployment-key.json

22. The Deployment Key
• Generated by the endpoint setup command
• Located in deployment-key.json
• Contains
• Client ID and secret
• Encryption key to endpoint configuration stored in the Globus
service
• Used to add data transfer nodes to the endpoint
• Can be used to recover your deployment
• Cannot be recovered by Globus
• Treat it like a password – know where it is and secure it

23. Where we
are so far

24. 24

25. Data Transfer Node set up
$ sudo globus-connect-server node setup
Note: deployment-key.json should be in same directory or specified location
What does node setup do?
• It adds your machine to your endpoint
• Starts services on your machine
Commands at
bit.ly/gw-tut
docs.globus.org/globus-connect-server/v5.4/#gcsv5-node-setup

26. Display endpoint details
$ globus-connect-server login localhost
$ globus-connect-server endpoint show
Commands at
bit.ly/gw-tut
docs.globus.org/globus-connect-server/v5.4/reference/

27. Where we
are so far

28. 28

29. Storage Gateways policies define…
• Who may use Globus to access your storage?
• Which parts of the file system are accessible via
Globus?
• What are the authentication requirements?
• What type of storage?
–POSIX or AWS S3* or Google Cloud* or Box* or …
*Subscription feature

30. Who may use Globus to access your storage
• Which Globus users?
–Select one or more Globus identity domains
• Which local users?
– Deny or allow local users or groups
• How do Globus users relate (map) to local users?
–Configure the method to map Globus user to local account
Storage Gateways policies define…
docs.globus.org/globus-connect-server/v5.4/data-access-guide/#creating_a_storage_gateway

31. Which Globus users?
• User must have an identity from one of the configured
domains
– On access attempts, linked identities will be scanned for a match
– If no identity from the required domain(s), user will be asked to link one
– Note: Domain restriction for data sharing are configured on mapped
collection, not storage gateway
• Identity domains may include…
– any organization in the Globus federated IdP list
– your institution’s identity provider trusted by Globus
– a local OpenID Connect (OIDC) server using your PAM stack
docs.globus.org/globus-connect-server/v5.4/data-access-guide/#authentication_policies

32. Which local users?
• You can further narrow the access universe using…
--user-allow
--user-deny
--posix-group-allow (POSIX storage gateways only)
--posix-group-deny (POSIX storage gateways only)
docs.globus.org/globus-connect-server/v5.4/data-access-guide/#user_access_overview

33. How do Globus users and local users relate (map)?
• Default: Strip identity domain (everything after “@”)
– e.g. userX@globusdemo.org maps to local account userX
– Best for campus identities w/synchronized local accounts
• Use --identity-mapping option on storage gateway
– Specify expression in a JSON document
– Execute a custom script which Globus Connect Server calls
when it needs to map an identity.
docs.globus.org/globus-connect-server/v5.4/identity-mapping-guide/
docs.globus.org/globus-connect-server/v5.4/data-access-guide/#identity_mapping_overview

34. gigi@globus.org
tennislover@gmail.com
jbarker@case.edu
rmcnally@case.edu
cstanley@case.edu
jbarker@state.edu
hsiebel@state.edu
Mapping identities to local accounts
Globus identities Local users
dan@iit.edu
Allow Globus identity domain case.edu
jbarker
rmcnally
ascott
Use default mapping
Deny user rmcnally

35. What parts of the file system are accessible via Globus
• Can restrict user access via Globus to subtrees
–restrict access via Globus to user’s home directory, for example
• Use --restrict_paths to specify narrower read,
read/write, or deny access for specific paths
– You provide a JSON doc that lists paths for each permission type
Storage Gateways policies define…
docs.globus.org/globus-connect-server/v5.4/data-access-guide/#data_access_policies

36. What are the authentication requirements
• How often should users reauthenticate?
–Default 11 days
• Should extra authentication assurances be required?*
–Session isolation?
–MFA?
*Subscription feature
Storage Gateways policies define…
docs.globus.org/globus-connect-server/v5.4/data-access-guide/#authentication_policies

37. 3. Create a storage gateway
$ globus-connect-server storage-gateway create
posix
> "My POSIX Storage Gateway"
> --domain globusid.org
> --user-deny root
> --authentication-timeout-mins 180
$ globus-connect-server storage-gateway list
Commands at
bit.ly/gw-tut

38. Where we
are so far

39. 39

40. Mapped Collection
A collection is the data access interface that Globus
presents to your user.
A mapped collection is only accessible to Globus users
that “map” to a local account.
docs.globus.org/globus-connect-server/v5.4/data-access-guide/#mapped_collection

41. Mapped collection policies to consider
• What is the base path or root of your mapped collection?
• Topmost directory available to user. Recommend narrowest base path
possible
• Must all transfers be encrypted?
• Can HTTPS be used to move data?
• Are users allowed to share* data? If so, what are sharing
policies?
• More policies…
*Subscription feature
docs.globus.org/globus-connect-server/v5.4/data-access-guide/#data_access_collection_create

42. 4. Create a mapped collection
$ globus-connect-server collection create
> f77ff456-1f18-41d3-94a7-f3fd8858ea4d
> /
> "State University HPC Center"
> --organization "State University"
> --contact-email support@example.org
> --description "Gamma storage at State U"
Specifying "/" as the base path sets the collection root to the local user’s home directory
Adding metadata to your collection will help your users.
Collection base path Storage gateway UUID
Commands at
bit.ly/gw-tut
Collection name

43. Globus Connect Server v5 Architecture
Done!

44. Access our mapped
collection

45. We are using the default identity
mapping, so…
Create a local user account that is the same are your user name
in the storage gateway’s allowed domain.
– e.g., for jbaer@case.edu create local account “jbaer”
$ sudo adduser --disabled-password --gecos 'jbaer' jbaer
Access your mapped collection via the web app…
…and move some files, if you like
Commands at
bit.ly/gw-tut

46. Subscription
Features

47. 5. Associating your endpoint with a subscription
Must be subscription manager.
$ globus-connect-server endpoint set-subscription-id
or go to app.globus.org/console/endpoints
Confirm: $ globus-connect-server endpoint show
Commands at
bit.ly/gw-tut

48. Globus Data Sharing
• Primary access (via a mapped collection) requires an
account on the host system
• Subscribers may allow mapped collection users to
share with others who don’t have accounts on the
host system (via guest collections)

49. 6. Enabling sharing (guest collections)
• Configure sharing policies on mapped collection
• You can restrict the authorized accounts…
o --sharing-user-allow
--sharing-user-deny
o --posix-sharing-group-allow
o --posix-sharing-group-deny
• …and sharing paths…
o --sharing-restrict-paths (specify JSON PathRestrictions)
• You can also set policies for specific user/path
combinations
o $ globus-connect-server sharing-policy create ...

50. Data sharing configuration considerations
Any restrictions on the sharing permissions levels? Read-only sharing? Read-
only for some paths and read/write for other paths?
Any restrictions on which users may share? Only users with training may
share?
Any restrictions on paths that may be shared with guests? Only share home
directory?
Any user specific sharing policies? Can Alice share any folder she can access,
but Bob can only share folders in his home directory?
Any restrictions on the identity domain of the guest? No sharing with
gmail.com?

51. Data sharing monitoring and management
Admins of mapped collections may…
View access control lists.
Delete access control lists.
Delete guest collections.
Delete guest collections according to last time they were accessed or created.

52. 7. Enable web browser upload and
download
• Authorized users can
upload, download files via a
browser
• Must have permissions to
the collection
– Collection configuration governs
access
– Web server is a different
application (separate
authentication)

53. Using the
management console

54. The Management Console
Monitor and manage transfers*
• Real time overview of transfers in and out of your system
• Pause or cancel transfers by endpoint or by user
Set a pause rule for current and future transfers or
users*
• Ideal for maintenance mode
Manage your endpoint
• Edit metadata*
• Associate it with a subscription
Manage your clients
*Subscription feature

55. Resources
• GCSv5 Guides: docs.globus.org/globus-connect-
server/
• Migration: docs.globus.org/globus-connect-
server/migrating-to-v5.4/
• Globus support: support@globus.org