The past few months saw some of the most important outputs of the project both for the supplier and user community of OCRE. In this general update webinar, the project will provide an update on some of the key developments in the project including the conclusion of the OCRE IaaS+ Tender and the signing of more than 400 framework agreements, the presentation of the OCRE Cloud Catalogue of commercial cloud services for the European research community, the validation test suite for suppliers, research projects selected to receive adoption funding support. as well as some of the work in progress for Earth Observation digital services procurement.

1. Cloud Validation Suite
João Fernandes (joao.fernandes@cern.ch)
CERN, Senior Member of Staff – IT Department
Ignacio Peluaga Lozada (ignacio.peluaga.lozada@cern.ch)
CERN, Cloud Architect – IT Department
04/03/2021

3. 3/23/21 3

4. • Automated
• End to End automated deployment using open technologies
• Terraform, Ansible, Docker, Kubernetes
• Abstracted
• Allow heterogenous, containerised tests to be deployed containing all
dependencies, providing working examples in how to deploy applications
• Heterogeneous
• Generic tests, applicable in any scientific field benchmarking dynamic systems
• Central repository for results
• Collect test and validation results in a structured manner (CERN S3 service)
Approach
3/23/21 joao.fernandes@cern.ch | Cloud Test Suite 4

5. Cloud Test Suite
3/23/21 joao.fernandes@cern.ch | Cloud Test Suite 5

6. • Current tests
• Traditional CPU benchmarking (CERN)
• End to end network measurements using perfSONAR (U. Michigan)
• Basic S3 endpoint validation (CERN)
• Generation of public cloud clusters on-demand for batch workload execution (INFN)
• Machine Learning (ML) distributed training of Generative Adversarial Networks (GANs) (CERN)
• Satellite image analysis and generation using ML Progressive Growing GANs (UNOSAT)
• Under development
• HPCaaS: validation of scientific packages using massive parallel MPI (Photon-Neutron science community)
• Reproducibility of scientific analysis on cloud: Higgs Boson demo, generic burst-out from on-prem (CERN)
• COSBench: open source benchmarking tool for S3 (CEPH community)
• Benchmarking of vendor specific offers (Google TPUs, AWS Graviton, Graphcore IPUs…)
• QCaaS: benchmarking quantum reinforcement learning algorithms for industry 4.0 (HPC community)
• Moving up the stack (SaaS): FAIR evaluators, AAI tests, etc.
Evolving Catalog
3/23/21 joao.fernandes@cern.ch | Cloud Test Suite 6

7. • Initiate runs in the original platforms with the current integrated tests
• Vendors are welcome to run tests themselves before CERN makes ‘OCRE’ runs
• Test and validation to be part of the selection criteria for adoption funding
• Results to be stored as JSON files on CERN CEPH service
• OCRE consortium members can access all the results
• Vendors access only their specific results, via pre-signed S3 URLs
• Dashboard to become available during the year for user friendly data parsing and result display
• No results will be made public without agreement of the respective vendor
• OCRE aims to create a certification process for successfully validated platforms
• Process to be communicated and agreed with vendors
• Requirement: modest vendor sponsored subscription of 5k eur valid for 6 months
• Timeline: April to December 2021
OCRE context
3/23/21 joao.fernandes@cern.ch | Cloud Test Suite 7

8. • Communication Channel: Cloud-Test-Suite@cern.ch
• A CERN representative will handle all communication between the
vendors technical representatives and the CERN developers
• A CERN technical representative will be responsible for deploying
and performing validation tests
• Permissions to create additional local user accounts in the award subscription are
needed for cases where members need access to run or complete those tests.
Communication with the Technical Team
3/23/21 joao.fernandes@cern.ch | Cloud Test Suite 8

9. • Even commodity cloud services need validation against research workloads
”Is this cloud offering designed to the use of my research team?”, “How can I run multiple
configurations for my software?”, “Is the cloud setup too time consuming?”, “Do they have GPUs?”
etc.
• Approach is open, aiming to validate scientific requirements as wide as
possible, providing scientists working examples of deployment
• In the context of OCRE, the test suite will be used as part of the selection
criteria for the adoption funds available in the project.
• Roadmap for the European Open Science Cloud: expansion of the validation
process in view of the activities with commercial providers
• Covering aspects such as data processing, governance, compliance (i.e., data locations, data
access levels, etc.), cost optimisation and execution of exit strategies
Summary
3/23/21 joao.fernandes@cern.ch | Cloud Test Suite 9

10. home.cern
Test Suite Repository:
https://github.com/cern-it-efp/EOSC-Testsuite
Test Suite Documentation:
https://eosc-testsuite.readthedocs.io/en/latest/
Demo (EGI conference 2020):
https://www.youtube.com/watch?v=ZznFp9IlGR0
Contact:
Cloud-Test-Suite@cern.ch
Thank you. Questions?