Skip to content

docs: Document --model-reference flag for serverless create #631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
promptless wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

docs: Document --model-reference flag for serverless create #631

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2a4bc9d
Document --model-reference flag for serverless create
promptless[bot] May 7, 2026
b21f97a
Merge branch 'main' into promptless/serverless-model-reference
promptless[bot] May 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions runpodctl/reference/runpodctl-serverless.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ Create a new Serverless endpoint from a template or from a Hub repo:
# Create from a template
runpodctl serverless create --name "my-endpoint" --template-id "tpl_abc123"

# Create from a template with model references
runpodctl serverless create --template-id "tpl_abc123" --gpu-id ADA_24 \
--model-reference https://example.com/models/llama:v1

# Create from a Hub repo
runpodctl hub search vllm # Find the hub ID
runpodctl serverless create --hub-id cm8h09d9n000008jvh2rqdsmb --name "my-vllm"
Expand Down Expand Up @@ -159,6 +163,10 @@ Execution timeout in seconds. Jobs that exceed this duration are terminated. The
Environment variable in `KEY=VALUE` format. Use multiple `--env` flags to set multiple variables. When deploying from `--hub-id`, these values override the Hub release defaults.
Copy link
Copy Markdown
Contributor Author

@promptless promptless Bot May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added --model-reference flag documentation based on PR #276 in runpodctl. The PR adds the flag to cmd/serverless/create.go with constraints: only works with --template-id (not --hub-id) and requires GPU compute type.

Source: runpod/runpodctl#276

</ResponseField>

<ResponseField name="--model-reference" type="string">
Model reference URL to attach to the endpoint. Use multiple `--model-reference` flags to attach multiple models. Only supported with `--template-id` (not `--hub-id`) and requires GPU compute type.
</ResponseField>

### Update an endpoint

Update endpoint configuration:
Expand Down
Loading