Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions deploy-scripts/lib/gcp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,12 @@
# gcp.sh - Google Cloud Platform resource management functions
#
# This module handles discovery and cleanup of GCP resources (Pub/Sub topics and subscriptions)
# created during deployment
# created during deployment.
#
# NAMESPACE requirements
# - Must be unique to prevent Pub/Sub topic/subscription collisions across deployments
# - Must be DNS-1123 compliant (lowercase alphanumeric, hyphens, start/end with alphanumeric)
# - Default: hyperfleet-e2e-$USER (when using .env configuration)

# ============================================================================
# Constants
Expand Down Expand Up @@ -48,10 +53,11 @@ discover_pubsub_topics() {
fi

# List topics that match the namespace pattern
# NAMESPACE must be unique and DNS-1123 compliant (default: hyperfleet-e2e-$USER when using .env)
# Topics are named:
# - ${NAMESPACE}-${resource_type} (e.g., hyperfleet-e2e-clusters, hyperfleet-e2e-nodepools)
# - ${NAMESPACE}-${resource_type}-dlq (e.g., hyperfleet-e2e-clusters-dlq)
# - ${NAMESPACE}-${resource_type}-${adapter_name}-dlq (e.g., testb-clusters-cl-namespace-dlq) - ACTUAL (Temporary))
# - ${NAMESPACE}-${resource_type} (e.g., hyperfleet-e2e-jdoe-clusters, hyperfleet-e2e-jdoe-nodepools)
# - ${NAMESPACE}-${resource_type}-dlq (e.g., hyperfleet-e2e-jdoe-clusters-dlq)
# - ${NAMESPACE}-${resource_type}-${adapter_name}-dlq (e.g., hyperfleet-e2e-jdoe-clusters-adapter1-dlq)
local topics=()
local all_topics

Expand Down Expand Up @@ -114,8 +120,9 @@ discover_pubsub_subscriptions() {
fi

# List subscriptions that match the namespace pattern
# NAMESPACE must be unique and DNS-1123 compliant (default: hyperfleet-e2e-$USER when using .env)
# Subscriptions are named: ${NAMESPACE}-${resource_type}-${adapter_name}
# Example: hyperfleet-e2e-clusters-example1-namespace, hyperfleet-e2e-nodepools-validation
# Example: hyperfleet-e2e-jdoe-clusters-adapter1, <unique-namespace>-clusters-adapter1
local subscriptions=()
local all_subscriptions

Expand Down
22 changes: 15 additions & 7 deletions docs/runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,14 @@ export NODEPOOL_TIER0_ADAPTERS_DEPLOYMENT="${NODEPOOL_TIER0_ADAPTERS_DEPLOYMENT:
# Adapters for API cluster/nodepool configuration
export API_ADAPTERS_CLUSTER="${API_ADAPTERS_CLUSTER:-cl-namespace,cl-job,cl-deployment,cl-maestro}"
export API_ADAPTERS_NODEPOOL="${API_ADAPTERS_NODEPOOL:-np-configmap}"


# NAMESPACE must be unique to prevent GCP Pub/Sub topic/subscription collisions.
# Set in the .env.example file as:
export NAMESPACE="${NAMESPACE:-hyperfleet-e2e-$(echo ${USER:-default} | tr '[:upper:]' '[:lower:]')}"
# Or can manually set it with as the namespace is DNS-1123 compliant
export NAMESPACE=<unique_namespace>
Comment on lines +158 to +162
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

DNS-1123 guidance is incomplete; current example can still generate invalid namespaces.

$(echo ${USER:-default} | tr '[:upper:]' '[:lower:]') only lowercases; it does not remove/normalize invalid DNS-1123 chars (e.g., _, .). That can still produce invalid namespace/topic/subscription names and break deploy flows.

Proposed doc fix
 # NAMESPACE must be unique to prevent GCP Pub/Sub topic/subscription collisions.
 # Set in the .env.example file as:
-export NAMESPACE="${NAMESPACE:-hyperfleet-e2e-$(echo ${USER:-default} | tr '[:upper:]' '[:lower:]')}"
-# Or can manually set it with as the namespace is DNS-1123 compliant
-export NAMESPACE=<unique_namespace>
+export NAMESPACE="${NAMESPACE:-hyperfleet-e2e-$(echo "${USER:-default}" | tr '[:upper:]' '[:lower:]' | sed -E 's/[^a-z0-9-]+/-/g; s/^-+//; s/-+$//')}"
+# Or set it manually to a unique DNS-1123 value (example):
+export NAMESPACE="hyperfleet-e2e-jdoe"

As per coding guidelines, “Focus on major issues impacting performance, readability, maintainability and security… Validate changes against HyperFleet architecture standards,” and the referenced runbook content requires NAMESPACE to be unique and DNS-1123 compliant.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# NAMESPACE must be unique to prevent GCP Pub/Sub topic/subscription collisions.
# Set in the .env.example file as:
export NAMESPACE="${NAMESPACE:-hyperfleet-e2e-$(echo ${USER:-default} | tr '[:upper:]' '[:lower:]')}"
# Or can manually set it with as the namespace is DNS-1123 compliant
export NAMESPACE=<unique_namespace>
# NAMESPACE must be unique to prevent GCP Pub/Sub topic/subscription collisions.
# Set in the .env.example file as:
export NAMESPACE="${NAMESPACE:-hyperfleet-e2e-$(echo "${USER:-default}" | tr '[:upper:]' '[:lower:]' | sed -E 's/[^a-z0-9-]+/-/g; s/^-+//; s/-+$//')}"
# Or set it manually to a unique DNS-1123 value (example):
export NAMESPACE="hyperfleet-e2e-jdoe"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/runbook.md` around lines 158 - 162, Update the NAMESPACE example to
produce DNS-1123 compliant names by normalizing the USER-derived portion:
convert to lowercase, replace any non-alphanumeric characters with hyphens,
collapse repeated hyphens, trim leading/trailing non-alphanumeric/hyphen
characters and optionally limit the length; update the .env.example export of
NAMESPACE to describe this sanitization and provide an example command that uses
USER-derived input and these normalization steps so generated namespaces are
valid for GCP Pub/Sub; reference the NAMESPACE variable and the .env.example
export line in the change.


```

#### Verify Deployment
Expand Down Expand Up @@ -261,8 +269,8 @@ The following tools are available to help debug and interact with HyperFleet com
**Important:** Set the `NAMESPACE` environment variable to match the namespace used during deployment. Some test cases deploy adapters dynamically and need to target the same namespace where your HyperFleet components are running.

```bash
# Set NAMESPACE if you deployed to a custom namespace
export NAMESPACE=my-custom-namespace
# Set NAMESPACE if you deployed to a unique namespace
export NAMESPACE=<unique_namespace>
./bin/hyperfleet-e2e test --label-filter=tier0
```

Expand All @@ -281,7 +289,7 @@ Unexpected error:

1. **Check if all pods are running:**
```bash
kubectl get pods -n hyperfleet-e2e
kubectl get pods -n ${NAMESPACE}
```

Expected output - all pods should show `Running` with `READY 1/1`:
Expand All @@ -296,13 +304,13 @@ Unexpected error:
2. **Check pod logs for errors:**
```bash
# Check API logs
kubectl logs -n hyperfleet-e2e deployment/hyperfleet-api --tail=50
kubectl logs -n ${NAMESPACE} deployment/hyperfleet-api --tail=50

# Check Sentinel logs
kubectl logs -n hyperfleet-e2e deployment/hyperfleet-sentinel --tail=50
kubectl logs -n ${NAMESPACE} deployment/hyperfleet-sentinel --tail=50

# Check adapter logs
kubectl logs -n hyperfleet-e2e deployment/cl-namespace-adapter --tail=50
kubectl logs -n ${NAMESPACE} deployment/cl-namespace-adapter --tail=50
```

3. **Verify API connectivity:**
Expand All @@ -314,7 +322,7 @@ Unexpected error:
4. **Check service endpoints:**
```bash
# Verify LoadBalancer has external IP
kubectl get svc -n hyperfleet-e2e hyperfleet-api
kubectl get svc -n ${NAMESPACE} hyperfleet-api
```


Expand Down