Run a mock round

Cost in on-demand BigQuery is driven by bytes scanned, so reduce what each query reads.

• Partition tables, usually by date, and add partition filters so only relevant partitions are scanned.
• Cluster on columns used in filters and joins; clustering prunes blocks within partitions.
• Select only the columns you need. BigQuery is columnar, so SELECT * is expensive.
• Use materialised views or scheduled queries to precompute expensive aggregations.
• Avoid correlated subqueries and cartesian joins, and use approximate functions when exact counts are unnecessary.
• Check bytes processed with a dry run before running.

SELECT DATE(ts) AS d, COUNT(*) AS c
FROM `proj.dataset.events`
WHERE ts >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
GROUP BY d;

For steady heavy usage, consider flat-rate slot reservations and monitor slot utilisation in INFORMATION_SCHEMA.JOBS.

An organization is the root node of the Google Cloud resource hierarchy. A project is the fundamental resource container where you enable APIs, create resources, and attach billing.

Hierarchy: organization, then folders, then projects, then resources. IAM policies and organization policies inherit downward.

• Organization: created with Google Workspace or Cloud Identity, it lets you centrally apply policy and manage billing accounts.
• Project: required for almost every resource. It has a project ID, number, and name. Billing attaches here, and resources cannot span projects.
• Folders: group projects by department or environment and apply policy at that level.

gcloud projects create my-app-prod --organization=123456789
gcloud projects list --format='table(projectId,name)'

Best practice is one project per environment or service boundary, Shared VPC or VPC peering for connectivity, and least-privilege IAM at folders.

A GCP VPC is a global, software-defined network. Subnets are regional, and instances in different regions can communicate over the internal network.

• VPC networks come in auto mode, with one subnet per region, or custom mode. Custom is recommended for control.
• Subnets have a primary CIDR and optional secondary ranges for GKE pods and services.
• Firewall rules are stateful, applied at the VPC level, and use priorities. They target instances by network tags or service accounts.
• Rules define direction, protocol, ports, source and destination ranges, and action. Default rules allow internal traffic and deny ingress from the internet.

gcloud compute firewall-rules create allow-https \
  --network=my-vpc --allow=tcp:443 \
  --source-ranges=0.0.0.0/0 --target-tags=web

Use hierarchical firewall policies for organisation-wide rules and Shared VPC to centralise networking across projects.

A service account key is a downloadable JSON file containing a private key that lets any holder authenticate as that service account. It is long-lived and does not expire unless you delete it.

Why to avoid keys:

• They are bearer credentials; if leaked, an attacker gets the account's permissions until the key is revoked.
• They are hard to rotate and audit, and often end up in source control or CI logs.
• Google recommends workload identity instead.

Better options:

• Attach a service account to Compute Engine, GKE, or Cloud Run so the workload gets short-lived tokens automatically.
• Use Workload Identity Federation to let external workloads, such as GitHub Actions, exchange their identity for GCP tokens.
• If a key is unavoidable, store it in Secret Manager, restrict access, and rotate regularly.

gcloud iam service-accounts keys list --iam-account=sa@proj.iam.gserviceaccount.com

Treat keys as a last resort.

Cloud Storage classes trade storage cost for retrieval cost and minimum storage duration.

• Standard: frequent access, no minimum duration. Best for hot data and serving websites.
• Nearline: accessed less than once a month, 30-day minimum.
• Coldline: accessed less than once a quarter, 90-day minimum.
• Archive: accessed less than once a year, 365-day minimum, and the lowest storage price.

All classes offer millisecond access and the same durability; only cost and minimum duration differ. Set the class at upload or let Autoclass move objects automatically based on access.

gcloud storage cp logs/*.gz gs://my-bucket/logs/ \
  --storage-class=NEARLINE
gcloud storage buckets update gs://my-bucket --lifecycle-file=lifecycle.json

Lifecycle rules can transition or delete objects. Beware early-deletion charges if you move data before the minimum duration elapses.

All run workloads, but at different levels of abstraction.

• Compute Engine: raw VMs with full control of the OS, networking, and disks. Best for legacy apps, custom kernels, or lift-and-shift. You manage patching and scaling, although managed instance groups help.
• Google Kubernetes Engine: managed Kubernetes. Runs containerised workloads with orchestration, autoscaling, and a rich ecosystem. Autopilot mode manages nodes for you. Choose it when you need portability, complex scheduling, or many services.
• Cloud Run: serverless containers. Scales to zero, handles HTTP and events, and bills per request. Best for stateless APIs and jobs with unpredictable traffic.

A rule of thumb: start with Cloud Run for simple stateless services, GKE for a Kubernetes-native platform, and Compute Engine only when you need machine-level control.

GCP IAM binds principals to roles on resources. A service account is a special identity for workloads rather than humans.

• Principals: users, groups, service accounts, domains.
• Roles: basic (owner, editor, viewer), predefined, or custom. Roles are collections of permissions.
• Policy: a binding of principal, role, and resource, inherited down the hierarchy.

Service accounts:

• Identified by an email such as sa@project.iam.gserviceaccount.com.
• Prefer attaching them to resources such as Compute Engine, GKE Workload Identity, or Cloud Run so workloads get short-lived tokens.
• Avoid downloading long-lived JSON keys; if you must, rotate them and store them in Secret Manager.

gcloud iam service-accounts create reader --display-name="Reader"
gcloud projects add-iam-policy-binding my-proj \
  --member="serviceAccount:reader@my-proj.iam.gserviceaccount.com" \
  --role="roles/storage.objectViewer"

Follow least privilege and prefer Workload Identity Federation over keys.

Plan for node, zone, and control-plane failures.

• Regional cluster: the control plane is replicated across three zones and nodes spread across zones, so a single zone outage does not take the cluster down. Zonal clusters are cheaper but not zone-resilient.
• Node pools: use multiple node pools for different workloads and enable cluster autoscaling plus node auto-repair and auto-upgrade.
• Workloads: set resource requests and limits, PodDisruptionBudgets, and topology spread constraints so replicas spread across zones.
• Networking: use regional load balancing and multi-cluster ingress or Gateway for cross-region failover.
• Data: use regional persistent disks or a replicated database, and back up with Backup for GKE.

gcloud container clusters create-auto prod \
  --region=us-central1 --release-channel=stable

Test with node drains and zone failures, and watch quota, IP exhaustion, and upgrade windows. Autopilot reduces operational burden but limits node customisation.