This is one of two presentations to students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many other cloud services? In this comprehensive yet still high-level overview of Google Cloud tools & APIs with the purpose of inspiring you as to what's possible. The session introduces Google's serverless platforms and machine learning & other APIs, tools that have an immediate impact on projects, alleviating the need to think about computing infrastructure as well as dispensing with the need to have machine learning expertise. We'll wrap up w/online resources like videos & hands-on tutorials to get you started! The main takeaways are where to run your code, store your data, and analyze your data, all in the cloud!
This talk is 1-hr in length.
The other version of this talk ("A") is an 45-mins long and focuses more on APIs platforms.
Powerful Google developer tools for immediate impact! (2023-24 B)
1. Powerful Google Cloud tools
for immediate impact!
Vancouver :: Fall 2023
Wesley Chun
Principal, CyberWeb
@wescpy@
Principal Consultant, CyberWeb
● Mission: produce accelerated Python
developers, enable developers to be
successful using Google Cloud and
other Google developer tools & APIs
● Focus: Python, Google Cloud (GCP) &
Google Workspace (GWS) APIs; GAE
migrations; Google X-product sol'ns
● Services: technical consulting,
training, engineering, speaking, code
samples, hands-on tutorials, public
technical content (blogs, social, etc.)
About the speaker
Previous experience / background
● Software Engineer & Developer Advocate
○ Google, Sun, HP, Cisco, EMC, Xilinx &
○ Original Yahoo!Mail engineer/SWE
● Technical trainer, teacher, instructor
○ Teaching Math, Linux, Python since '83
○ Adjunct CS Faculty at local SV colleges
● Python community member
○ Popular Core Python series author
○ Python Software Foundation Fellow
● AB (Math/CS) & CMP (Music/Piano), UC
Berkeley and MSCS, UC Santa Barbara
● Adjunct Computer Science Faculty, Foothill
College (Silicon Valley)
GWS Dev Show
goo.gl/JpBQ40
GAE migration
bit.ly/3xk2Swi
2. Impactful developer tools: why & agenda
● People know Google for search, YouTube, Chrome, Android, Gmail
● Google also has an extensive set of developer tools, APIs & platforms
● Many of these developer services part of Google Cloud
● Cloud has taken industry by storm & these skills are in-demand
● Look at most impactful tools for work/school projects you can use today!
Cloud computing
& Google Cloud
1
Where to run
your code
2
Google APIs
primer
3
Where to
analyze your
data
4 5
Where to store
your data
7
Wrap-up
6
Inspiration
01
Cloud computing
& Google Cloud
Intro and overview
3. Getting things done using someone else’s computers, especially
where someone else worries about maintenance, provisioning, system
administration, security, networking, failure recovery, upgrades, etc.
Google Compute Engine, Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
Cloud service levels/"pillars"
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google BigQuery, Cloud SQL, Vertex AI,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Google Apps Script
Salesforce1/force.com
Google Workspace (was G Suite/Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
4. Google Compute Engine, Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
Outsourcing of apps (SaaS)
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google BigQuery, Cloud SQL, Vertex AI,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Google Apps Script
Salesforce1/force.com
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
Google Workspace (was G Suite/Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google Compute Engine, Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
Outsourcing of hardware (IaaS)
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google BigQuery, Cloud SQL, Vertex AI,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Google Apps Script
Salesforce1/force.com
Google Workspace (was G Suite/Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
5. Google Compute Engine, Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
Outsourcing of logic-hosting (PaaS)
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google BigQuery, Cloud SQL, Vertex AI,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Google Apps Script
Salesforce1/force.com
Google Workspace (was G Suite/Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
Google BigQuery, Cloud SQL, Vertex AI,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Google Compute Engine, Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
IaaS/PaaS gray area (DataB/S/P-aaS?)
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google Apps Script
Salesforce1/force.com
Google Workspace (was G Suite/Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
6. Google Compute Engine, Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
SaaS/PaaS gray area
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google BigQuery, Cloud SQL, Vertex AI,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Google Workspace (was G Suite/Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
Google Apps Script
Salesforce1/force.com
Summary of responsibility
SaaS
Software as a Service
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
IaaS
Infrastructure as a Service
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
PaaS
Platform as a Service
Managed by YOU Managed by cloud vendor
Applications
Data
Runtime
Middleware
OS
Virtualization
Servers
Storage
Networking
on-prem
all you, no cloud
7. Theme - Spec/Reqs
Logistics - Design app
Space - Provision HW
Food - Build & serve app
Manage - Manage app
on-prem
(DIY)
IaaS
(Compute Engine)
PaaS
(App Engine)
SaaS
(Google Workspace
(formerly G Suite))
Pick theme
Plan party
Find space
Cook
On-call
Pick theme
Plan party
Rent hall
Cook
On-call
Pick theme
Plan party
Rent hall
Hire Caterer
Hire manager
Pick theme
Hire planner
Rent hall
Hire caterer
Hire manager
Imagine you’re hosting a party...
formerly
( )
8. What is Google Cloud Platform?
some Google Cloud Platform products
Compute Big Data
BigQuery
Cloud
Dataflow
Cloud
Dataproc
Cloud
Datalab
Cloud
Pub/Sub
Cloud Life
Sciences
Storage & Databases
Cloud
Storage
Cloud
Bigtable
Cloud
Datastore
Cloud SQL
Cloud
Spanner
Persistent
Disk
Machine Learning
Vertex AI/GenAI
Cloud
Vision
Cloud
Speech APIs
Cloud Natural
Language
Cloud
Translation
Cloud
Jobs
Data
Studio
Cloud
Dataprep
Cloud Video
Intelligence
AutoML suite
Compute
Engine
App
Engine
Kubernetes
Engine/GKE
GPU
Cloud
Functions
Cloud Run
Identity & Security
Cloud IAM
Cloud Resource
Manager
Cloud Security
Scanner
Key
Management
Service
BeyondCorp
Data Loss
Prevention API
Identity-Aware
Proxy
Security Key
Enforcement
Networking
Transfer
Appliance
Cloud
Firestore
Cloud VPC
Cloud Load
Balancer
Cloud
Armor
What is Google Cloud Platform?
some Google Cloud Platform products (tl;dr)
Compute Big Data
BigQuery
Cloud
Dataflow
Cloud
Dataproc
Cloud
Datalab
Cloud
Pub/Sub
Cloud Life
Sciences
Storage & Databases
Cloud
Storage
Cloud
Bigtable
Cloud
Datastore
Cloud SQL
Cloud
Spanner
Persistent
Disk
Machine Learning
Vertex AI/GenAI
Cloud
Vision
Cloud
Speech APIs
Cloud Natural
Language
Cloud
Translation
Cloud
Jobs
Data
Studio
Cloud
Dataprep
Cloud Video
Intelligence
AutoML auite
Compute
Engine
App
Engine
GPU
Cloud
Functions
Identity & Security
Cloud IAM
Cloud Resource
Manager
Cloud Security
Scanner
Key
Management
Service
BeyondCorp
Data Loss
Prevention API
Identity-Aware
Proxy
Security Key
Enforcement
Transfer
Appliance
Cloud
Firestore
Cloud Run
Kubernetes
Engine/GKE
Networking
Cloud Load
Balancer
Cloud VPC
Cloud
Armor
9. 02
Where to run
your code
Serverless simplifies it all!
> Google Compute Engine configurable
VMs of all shapes & sizes, from
"micro" to 416 vCPUs, 11.776 TB
RAM, 256 TB HDD/SSD plus Google
Cloud Storage for data lake "blobs"
(Debian, CentOS, CoreOS, SUSE, Red Hat Enterprise Linux,
Ubuntu, FreeBSD; Windows Server 2008 R2, 2012 R2, 2016, 1803,
1809, 1903/2019, 1909)
cloud.google.com/compute
cloud.google.com/storage
Yeah, there are VMs & big disk… but why*?
10. Serverless: what & why
● What is serverless?
○ Misnomer (a "PMM") :-)
○ "No worries"
○ Developers focus on writing code & solving business problems*
○ Servers (physical & virtual) completely abstracted away from the user*
● Why serverless?
○ Fastest growing segment of cloud... per analyst research:
■ $1.9B (2016) and $4.25B (2018) ⇒ $7.7B (2021), $14.93B (2023), and $21.1B (2025)^
■ $4.18B (2018) and $6.05B (2020) ⇒ $31.53B (2026) and $53.08B (2028)†
○ What if you go viral? Autoscaling: your new best friend
○ What if you don't? Code not running? You're not paying.
* Forbes (May 2018)
^ (in USD) CB Insights (Sep 2018), MarketsandMarkets™ (Jan 2019)
† (in USD) Reports and Data (Jul 2019 , Jan 2020, and Oct 2021)
Google Compute Engine, Google Cloud Storage
AWS EC2 & S3; Rackspace; Joyent
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
Google Workspace (was G Suite/Google Apps)
Yahoo!Mail, Hotmail, Salesforce, Netsuite, Office 365
Google App Engine, Cloud Functions
Heroku, Cloud Foundry, Engine Yard, AWS Lambda
Google BigQuery, Cloud SQL,
Cloud Datastore, NL, Vision, Pub/Sub
AWS Kinesis, RDS; Windows Azure SQL, Docker
Serverless: PaaS-y compute/processing
Google Apps Script
Salesforce1/force.com
11. Google App Engine
App-hosting in the cloud ("Serverless 1.0")
cloud.google.com/appengine
Why does App Engine exist?
● Focus on code not DevOps
○ Web app or mobile backend
● Enhance productivity
● Deploy globally
● Fully-managed
● Auto-scaling
● Pay-per-use
● Familiar languages
● Test w/local dev server
● "Bundled" services like DB,
caching, tasks, storage, etc.
12. Hello World (Python "MVP")
app.yaml
runtime: python39
main.py
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello():
return 'Hello World!'
requirements.txt
flask
Deploy:
$ gcloud app deploy
Access globally:
PROJECT_ID.appspot.com
cloud.google.com/appengine/docs/standard/python3/quickstart
Google Cloud Functions
Function-hosting in the cloud ("Serverless 2.0")
cloud.google.com/functions
13. Why does Cloud Functions exist?
● Don't have entire app?
○ No framework "overhead" (LAMP, MEAN...)
○ Deploy short utilities (alerts, ETL...), monoliths →
loosely-coupled microservices, event-driven tasks
● Event-driven
○ Triggered via HTTP or background events
■ Pub/Sub, Cloud Storage, Firebase, etc.
○ Auto-scaling & highly-available; pay per use
● Flexible development environment
○ Cmd-line or developer console (in-browser)
○ Develop/test locally with Functions Framework
● Cloud Functions for Firebase
○ Mobile app use-cases
main.py
def hello_world(request):
return 'Hello World!'
Deploy:
$ gcloud functions deploy hello --runtime python39 --trigger-http
Access globally (curl):
$ curl REGION-PROJECT_ID.cloudfunctions.net/hello
Access globally (browser):
https://REGION-PROJECT_ID.cloudfunctions.net/hello
Hello World (Python "MVP")
cloud.google.com/functions/docs/quickstart-python
14. Google Cloud Run
Container-hosting in the cloud ("Serverless 2.0")
cloud.run
The rise of containers... ● Any language
● Any library
● Any binary
● Ecosystem of base images
● Industry standard
FLEXIBILITY
15. “We can’t be locked in.”
“How can we use
existing binaries?”
“Why do I have to choose between
containers and serverless?”
“Can you support language _______ ?”
Serverless inaccessible for some...
CONVENIENCE
Cloud Run: code, build, deploy
.js .rb .go
.sh
.py ...
● Any language, library, binary
○ HTTP port, stateless
● Bundle into container
○ Build with Docker OR Cloud Build
○ Image ⇒ Container/Artifact Registry
● Deploy to Cloud Run (managed or GKE)
● GitOps: CD push-to-deploy from Git
○ See documentation & announcement
○ CI/CD with GitHub Actions tutorial
● Cloud Buildpacks: Docker, Dockerfiles,
and knowledge of containers optional
State
HTTP
16. Hello World (Python "MVP")
main.py
import os
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0', port=int(os.environ.get('PORT', 8080)))
cloud.google.com/run/docs/quickstarts/build-and-deploy
requirements.txt
flask
Hello World (Python "MVP")
Dockerfile
FROM python:3-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "main.py"]
.dockerignore
Dockerfile
README.md
*.pyc
*.pyo
.git/
__pycache__
Build (think docker build and docker push) then deploy (think docker run):
$ gcloud builds submit --tag gcr.io/PROJ_ID/IMG_NAME
$ gcloud run deploy SVC_NAME --image gcr.io/PROJ_ID/IMG_NAME
OR… Build and Deploy (1-line combination of above commands):
$ gcloud run deploy SVC_NAME --source .
Access globally:
SVC_NAME-HASH-REG_ABBR.a.run.app
Docker &
Dockerfile
OPTIONAL!!
17. ● Build containers easily & securely without creating/managing Dockerfiles
● Open source, open standard; based on CNCF Buildpacks spec v3
● Used by GCF Functions Framework to deploy locally-developed functions
● Supports most common development tools
○ Go 1.10+
○ Node.js 10+
○ Python 3.7+
○ Java 8 & 11
○ .NET Core 3.1+
● Blog posts
○ See cloud.google.com/blog/products/containers-kubernetes/google-cloud-now-supports-buildpacks and
cloud.google.com/blog/products/serverless/build-and-deploy-an-app-to-cloud-run-with-a-single-command
Deploy to Cloud Run with Buildpacks
github.com/GoogleCloudPlatform/buildpacks
$ ls
index.js package.json
$ gcloud run deploy myapp --source .
$ ls
main.py requirements.txt
$ gcloud run deploy myapp --source .
Node.js, Go,
Python, etc.?
It's the same
command!
Use case category
"Serverless 1.0" "Serverless 2.0"
App Engine
(Standard)
Cloud Run
Cloud
Functions
Web service logic-hosting
Web app hosting ✓ ✓
HTTP services ✓ ✓ ✓
Container hosting ✓
"Backoffice" (Apps and APIs)
Web & mobile backends ✓ ✓
Internal/intranet apps ✓ ✓
Automation
Large-scale data processing ✓ ✓
Workflow & orchestration ✓ ✓
Event-driven automation ✓ ✓
GitOps: Git push-to-deploy (CD) ✓
Common use cases
18. Flexibility in options
Cloud
Functions
App
Engine
Cloud
Run
local
server
● "Nebulous" sample web app
○ Flask/Python 2 or 3
○ Express/Node.js 10+
○ Uses Cloud Translation API
● Deployable to on-prem server
● Also GCP serverless compute
○ App Engine
○ Cloud Functions
○ Cloud Run
● With only config changes
● No changes to app code
Cloud
Translation
My "Google Translate" MVP
goo.gle/2Y0ph5q
youtu.be/eTotLOVR7MQ
Google Apps Script
Customized serverless JS runtime for automation, and extension
and integration with Google Workspace (formerly G Suite) and
other Google & external services
20. What can you do with this?
Accessing maps from
spreadsheets?!?
goo.gl/oAzBN9
This… with help from Google Maps & Gmail
function sendMap() {
var sheet = SpreadsheetApp.getActiveSheet();
var address = sheet.getRange("A2").getValue();
var map = Maps.newStaticMap().addMarker(address);
GmailApp.sendEmail('friend@example.com', 'Map',
'See below.', {attachments:[map]});
}
JS
g.co/codelabs/apps-script-intro
23. General steps
1. Go to Cloud Console
2. Login to Google/Gmail account
(Workspace domain may require admin approval)
3. Create project (per application)
4. Enable APIs to use
5. Enable billing (CC, Free Trial, etc.)
6. Download client library(ies)
7. Create & download credentials
8. Write code*
9. Run code (may need to authorize)
Google APIs: how to use
*In your code
1. Import API client library
2. Create API client object
3. Use client to make API Calls
Costs & pricing
● GCP & GMP: pay-per-use (CC req'd)
● GWS: "subscription" (incl. $0USD/mo.)
● GMP: $200/mo. free usage
● GCP Free Trial: $300/1Q
● GCP "Always Free" tier
○ Some products have free tier
○ Daily or monthly quota
○ Must exceed to incur billing
● More on both programs at
cloud.google.com/free
Cloud/GCP console
console.cloud.google.com
● Hub of all developer activity
● Applications == projects
○ New project for new apps
○ Projects have a billing acct
● Manage billing accounts
○ Financial instrument required
○ Personal or corporate credit cards,
Free Trial, and education grants
● Access GCP product settings
● Manage users & security
● Manage APIs in devconsole
24. Client libraries and credentials types
● Two different client library types
○ Platform-level client libraries (lower-level)
■ Multiple product groups as a "lowest-common denominator"
■ Install: developers.google.com/api-client-library
○ Product-level client libraries (higher-level)
■ Custom client libraries made specifically for 1 product or product group
■ Found on product or product group page(s)
● Three different credentials types
○ Simple: API keys (access public data)
■ Simplest form of authorization: an API key; tied to a project
○ Authorized: OAuth client IDs (access data owned by [human] user)
■ Provides additional layer of security via OAuth2 (RFC 6749)
○ Authorized: service accounts (access data owned by an app/robot user)
■ Provides additional layer of security via OAuth2 or JWT (RFC 7519)
Blog series:
dev.to/wescpy
&
Google APIs client
libraries for many
languages; demos in
developers.google.com/api-
client-library
cloud.google.com/apis/docs
/cloud-client-libraries
25. OAuth2 or
API key
HTTP-based REST APIs 1
HTTP
2
Google APIs request-response workflow
● Application makes request
● Request received by service
● Process data, return response
● Results sent to application
(typical client-server model)
04
How to analyze
your data
AI/ML services via API
26. AI & machine learning
Puppy or muffin?
Source: twistedsifter.com/2016/03/
puppy-or-bagel-meme-gallery
27. Machine learning is learning
from rules plus experience.
Inference Phase
Training Phase
answers
data
model ("rules")
Machine
Learning
Model
predictions
(new) data
28. Machine Learning: Cloud Vision
Google Cloud Vision API lets developers
extract metadata and understand the
content of an image, identify & detect
objects/labels, text/OCR, landmarks,
logos, facial features, products, XC, etc.
cloud.google.com/vision
from google.cloud import vision
image_uri = 'gs://cloud-samples-data/vision/using_curl/shanghai.jpeg'
client = vision.ImageAnnotatorClient()
image = vision.types.Image()
image.source.image_uri = image_uri
response = client.label_detection(image=image)
print('Labels (and confidence score):')
print('=' * 30)
for label in response.label_annotations:
print(label.description, '(%.2f%%)' % (label.score*100.))
Vision: label annotation/object detection
29. $ python3 label-detect.py
Labels (and confidence score):
==============================
People (95.05%)
Street (89.12%)
Mode of transport (89.09%)
Transport (85.13%)
Vehicle (84.69%)
Snapshot (84.11%)
Urban area (80.29%)
Infrastructure (73.14%)
Road (72.74%)
Pedestrian (68.90%)
Vision: label annotation/object detection
g.co/codelabs/vision-python
Google Photos
The app has a
search bar?!?
blog.google/products/photos/smarter-photo-albums-without-work
30. Global view
Problem
● 1B ppl depend on seafood
● 85% at/over-fishing or recovering
● 20% caught illegal, undoc'd, unreg'd
● Analysts monitoring unscalable
One solution
● globalfishingwatch.org/map
● Machine-learning classifiers:
○ Ship type: cargo, tug, sail, fishing
○ Ship size
○ Gear: longline, purse seine, trawl
○ Movement tracking: when and
where vessels are fishing
quickdraw.withgoogle.com
Cloud Vision demo: Quick Draw game
experiments.withgoogle.com/quick-draw
31. ML "building block" APIs
● Gain insights from data using GCP's
pre-trained machine learning models
● Leverage the same technology as Google
Translate, Photos, and Assistant
● Requires ZERO prior knowledge of ML
● If you can call an API, you can use AI/ML!
● cloud.google.com/products/ai/building-blocks
Vision Video
Intelligence
Speech
(S2T & T2S)
Natural
Language
Translation
Machine Learning: AutoML
AutoML: a suite of cloud APIs for
developers with limited machine
learning expertise; auto-selects best
models & allows for further training
of those models for your data
(Translation, Vision, Natural Language,
Video Intelligence, Tables)
cloud.google.com/automl
cloud.google.com/automl-tables
32. Machine Learning: Cloud Generative AI
Google Cloud Generative AI is a set
of tools & APIs (like Vertex AI &
Duet AI) that make it easier for
developers to build generative
AI-powered services & applications;
includes Gemini & PaLM 2 models
cloud.google.com/ai/generative-ai
Full Spectrum of AI & ML Offerings
App developer Data scientist,
developer
Data scientist, Researcher
(w/infrastructure access &
DevOps/SysAdmin skills)
AI Platform
(now Vertex AI)
Auto ML
Build custom models,
use OSS SDK on fully-
managed infrastructure
ML APIs
App developer,
data scientist
Use/customize pre-built
models
Use pre-built/pre-
trained models
Build custom models, use/
extend OSS SDK, self-manage
training infrastructure
33. Google and Jupyter Notebooks
Users have many ways to access an indispensable data science tool
● Google Cloud Vertex AI Workbench
○ cloud.google.com/notebooks
● Kaggle
○ kaggle.com
● Google Research CoLaboratory
○ colab.research.google.com
● Google Cloud Dataproc Hub
○ cloud.google.com/dataproc/docs/tutorials/dataproc-hub-overview
● Google Cloud Datalab
○ cloud.google.com/datalab/docs/how-to/working-with-notebooks
05
Where to store
your data
And more ways to analyze it
34. Storing Data: Cloud SQL & AlloyDB
Relational DB servers in the cloud;
High-performance, fully-managed
600MB to 416GB RAM; up to 64 vCPUs
Up to 10 TB storage; 40,000 IOPS
Types:
MySQL
Postgres; AlloyDB for high perf
SQLServer
cloud.google.com/databases
cloud.google.com/{sql, alloydb}
Storing Data: Cloud Firestore
The best of both worlds: the next
generation of Cloud Datastore
(w/product rebrand) plus features
from the Firebase realtime database
(For choosing between Firebase & Cloud Firestore: see
firebase.google.com/docs/firestore/rtdb-vs-firestore;
for choosing between Firestore Datastore & Firestore Native modes:
see cloud.google.com/datastore/docs/firestore-or-datastore)
cloud.google.com/firestore
35. Storing and Analyzing Data: BigQuery
Google BigQuery: fully-managed data
warehouse for large-scale data
analytics with built-in machine
learning (BQML); issue SQL queries
across multi-terabytes of data. BQ
Sandbox lets anyone query up to
1TB/mo for free with no obligation
cloud.google.com/bigquery
BigQuery: querying Shakespeare words
TITLE = "The most common words in all of Shakespeare's works"
QUERY = '''
SELECT LOWER(word) AS word, sum(word_count) AS count
FROM [bigquery-public-data:samples.shakespeare]
GROUP BY word ORDER BY count DESC LIMIT 10
'''
rsp = BQ.jobs().query(body={'query': QUERY}, projectId=PROJ_ID).execute()
print('n*** Results for %r:n' % TITLE)
print('t'.join(col['name'].upper() # HEADERS
for col in rsp['schema']['fields']))
print('n'.join('t'.join(str(col['v']) # DATA
for col in row['f']) for row in rsp['rows']))
36. Top 10 most common Shakespeare words
$ python bq_shake.py
*** Results for "The most common words in all of Shakespeare's works":
WORD COUNT
the 29801
and 27529
i 21029
to 20957
of 18514
a 15370
you 14010
my 12936
in 11722
that 11519
● BigQuery public data sets: cloud.google.com/bigquery/public-data
● BQ sandbox (1TB/mo free): cloud.google.com/bigquery/docs/sandbox (see blog post)
● Other public data sets: cloud.google.com/public-datasets (Google Cloud),
research.google/tools/datasets (Google Research), and Kaggle (kaggle.com)
● COVID-19 BigQuery data sets
○ How to use our data sets (see blog post)
○ JHU Coronavirus COVID-19 Global Cases data set
○ List of all COVID-19 data sets
● Cloud Life Sciences API: cloud.google.com/life-sciences (see blog post)
● Cloud Healthcare API: cloud.google.com/healthcare (see blog post)
BigQuery and public data sets
37. Google Workspace: Google Sheets
Sheets API gives you programmatic
access to spreadsheets; perform
(w/code) almost any action you can
do from the web interface as a user
developers.google.com/sheets
06
Inspiration
Use multiple Google APIs
to create unique solutions
39. Gmail message processing with GCP
Gmail
Cloud
Pub/Sub
Cloud
Functions
Cloud
Vision
Workspace
(formerly G Suite)
GCP
Star
message
Message
notification
Trigger
function
Extract
images
Categorize
images
Inbox augmented with Cloud Function
40. ● Gmail API: sets up notification forwarding to Cloud Pub/Sub
● developers.google.com/gmail/api/guides/push
● Pub/Sub: triggers logic hosted by Cloud Functions
● cloud.google.com/functions/docs/calling/pubsub
● Cloud Functions: "orchestrator" accessing GCP (and Google Workspace/G Suite) APIs
● Combine all of the above to add custom intelligence to Gmail
● Deep dive code blog post
● cloud.google.com/blog/products/application-development/
adding-custom-intelligence-to-gmail-with-serverless-on-gcp
● Application source code
● github.com/GoogleCloudPlatform/cloud-functions-gmail-nodejs
App summary
Hangouts Chat Productivity Tracker
Chat bot that's GCP & Google Workspace (formerly G Suite) aware
41. Productivity tracker Hangouts Chat bot
Google Workspace
(formerly G Suite)
GCP
Sheets Natural Language
START
or LOG
END
Hangouts
Chat
App
Engine
Cloud
SQL
42. App summary
● Motivation
● Do coding contract jobs during school year
● Needed to track time spent on work (and non-work)
● Who doesn't want to be more productive?
● Hangouts Chat bot framework & API... build bots to:
● Automate workflows
● Query for information
● Other heavy-lifting
● Google Workspace (formerly G Suite) app that leverages app-hosting, NL processing, and storage
tools from GCP
● Application source code
● github.com/gsuitedevs/hangouts-chat-samples/tree/master/python/productivity_tracker
● Presented at GDG Silicon Valley (Aug 2018)
● meetup.com/gdg-silicon-valley/events/252858660
07
Wrap-up
Summary & resources
43. Summary: Google developer tools & APIs
● Google provides more than just apps
○ More than search, YouTube, Android, Chrome, Gmail, and Docs
○ Google technology & integration available to developers via APIs
● Tour of Google (Cloud) APIs & developer tools
○ GWS: not just a set of productivity apps… you can code them too!
○ GCP: compute, storage, networking, security, data & machine learning tools
■ Serverless: frees developers from infrastructure
■ So you can focus on building solutions
● Interesting possibilities when using >1 Google product families/APIs
● With Google: run your apps, analyze & store your data
Other Google APIs & platforms
● Google Workspace (G Suite) (code Gmail, Drive, Docs, Sheets, Slides!)
○ developers.google.com/gsuite
● Firebase (mobile development platform and RT DB plus ML-Kit)
○ firebase.google.com and firebase.google.com/docs/ml-kit
● Google Data Studio (data visualization, dashboards, etc.)
○ datastudio.google.com/overview
○ goo.gle/datastudio-course
● Actions on Google/Assistant/DialogFlow (voice apps)
○ developers.google.com/actions
● YouTube (Data, Analytics, and Livestreaming APIs)
○ developers.google.com/youtube
● Google Maps (Maps, Routes, and Places APIs)
○ developers.google.com/maps
● Flutter (native apps [Android, iOS, web] w/1 code base[!])
○ flutter.dev
44. ● Documentation (most APIs have "Quickstarts")
○ GCP: cloud.google.com/{docs,appengine,functions,run,vision,automl,translate,language,
speech,texttospeech,video-intelligence,firestore,bigquery,compute,storage,gpu,tpu}
○ GWS: developers.google.com/{gsuite,drive,calendar,gmail,docs,sheets,slides,apps-script}
● Introductory "codelabs" ([free] self-paced, hands-on tutorials)
○ GWS APIs: g.co/codelabs/gsuite-apis-intro (featuring Drive API)
○ Apps Script: g.co/codelabs/apps-script-intro
○ App Engine: codelabs.developers.google.com/codelabs/cloud-app-engine-python
○ Cloud Functions: codelabs.developers.google.com/codelabs/cloud-starting-cloudfunctions
○ Cloud Run: codelabs.developers.google.com/codelabs/cloud-run-hello-python3
○ Others: g.co/codelabs (all Google codelabs) and g.co/codelabs/cloud (GCP-only)
● Videos: youtube.com/GoogleCloudPlatform (GCP) and goo.gl/JpBQ40 (GWS)
● Code samples: github.com/GoogleCloudPlatform (GCP) and github.com/googleworkspace (GWS)
● Cloud Free Trial (new users) and Always Free (daily/monthly tier) programs: cloud.google.com/free
● Know AWS/Azure? Compare with GCP products at cloud.google.com/docs/compare/aws
● Language support: cloud.google.com/{python,java,nodejs,go,php,ruby,dotnet}
Resources (industry)
● Google Cloud (GCP)
○ Code samples for students: goo.gle/hackathon-toolkit
○ Videos: youtube.com/GoogleCloudPlatform
○ Codelabs (free, self-paced, hands-on tutorials): g.co/codelabs/cloud
○ Cloud Skills Boost/QwikLabs (free only for students): go.qwiklabs.com/hey-student
● GCP docs/quickstarts - cloud.google.com/{docs,appengine,functions,run,vision,automl,translate,
language, speech,texttospeech,video-intelligence,firestore,bigquery,compute,storage,gpu,tpu}
● Like GCP? Wanna use it in class or your research lab? Send your profs to cloud.google.com/edu to
apply for GCP teaching/research or Cloud Skills Boost/Qwiklabs/training credits!
● Know AWS/Azure? Compare w/GCP at cloud.google.com/docs/compare/aws
● GCP Free Trial (ignore) and Always Free (tier) - cloud.google.com/free
● Mobile apps: Firebase - firebase.google.com and Flutter - flutter.dev
● Workspace - developers.google.com/{workspace,drive,docs,sheets,slides} and goo.gl/JpBQ40
● Other Google (non-GCP) codelabs: g.co/codelabs
● Developer Student Clubs - g.co/dev/dsc - Global Google student developer groups
● Internships & other career stuff - google.com/students
Resources (students)
45. Prepare for careers in a cloud-first
world. Ask your faculty to apply now.
g.co/edu/cloudready
Career Readiness
track: certification
On-demand
Get free access to Architecting
with Google Compute Engine
specialization on-demand on
Coursera.
(100% discount)
Self-paced labs
Get free access to relevant
Google Cloud quests & other
Cloud Skills Boost self-paced
labs powered by Qwiklabs.
(100% discount: 10+ labs)
Google Cloud Certified
Prepare to take the exam and
earn your Cloud certification.
(100% discount for faculty, 50% discount for
students per adherence to timelines and
other criteria)
Careers at Google
Software Engineer (SWE): not the only technical role @Google
● Application Engineer
● Business Analyst
● Developer Advocate (DA)
● Developer Programs Engineer (DPE)
● Hardware Engineer
● Mechanical Engineer
● Network Engineer
● Product Manager (PM)
● Program Manager (PgM or TPM)
● Quantitative Analyst (Quant)
● Release Engineer
● Research Scientist
● Security Engineer
● Site Reliability Engineer (SRE)
● Software Engineer in Test (SET)
● System Administrator
● Technical Writer (TW)
● UX Content Strategist
● UX Designer
● UX Engineer
● Student careers: google.com/students
● Technical development guide: g.co/techdevguide
● Sample technical interview
46. Bring me to your organization
... it is my job to help you!
● "Transfer of Info" tech talks
● Half- or full-day seminars
● Hands-on "codelab" workshops
● Multi-day training courses
● Engineering consulting
● Migration strategy & planning
● cyberwebconsulting.com
● appenginemigration.com
Slides
you're looking
at them now
Work
cyberwebconsulting.com
Books
corepython.com
Blog
dev.to/wescpy
Img proc post
goo.gle/3nPxmlc
GCP+GWS 101 (2019)
youtu.be/ri8Bfptgo9Q
...& 102 talks (2023)
youtu.be/3IQ4Yv80lJg
Progress bars
goo.gl/69EJVw
Thank you! Questions?
Wesley Chun
Principal Consultant, CyberWeb
Python, GCP & GWS specialist
@wescpy (Tw/X, SO, GH, IG, LI)