VMware AIops

Author: Wei Zhou, VMware by Broadcom — wei-wz.zhou@broadcom.com This is a community-driven project by a VMware engineer, not an official VMware product. For official VMware developer tools see developer.broadcom.com.

English | 中文

AI-powered VMware vCenter/ESXi VM lifecycle and deployment tool — 60 tools.

Companion skills handle everything else:

Skill	Scope	Install
vmware-monitor	Read-only: inventory, health, alarms, events, metrics	uv tool install vmware-monitor
vmware-storage	Datastores, iSCSI, vSAN management	uv tool install vmware-storage
vmware-vks	Tanzu Namespaces, TKC cluster lifecycle	uv tool install vmware-vks

Need read-only monitoring only? Use VMware-Monitor — zero destructive code in the codebase.

⚡ Quick Investigation Reports (read-only)

Triage → investigate → act, all in one conversation. Five opinionated read-only reports aggregate and correlate server-side and hand back a high-signal result (never raw inventory), so you can decide where to look before changing anything. Each renders a self-contained offline HTML snapshot with --html (no external assets; drill-down detail collapses in native <details>, zero JavaScript). All delegate to the vmware-monitor library using AIops's own vCenter connection.

Question	Command	What it correlates
"What needs attention now?" across all vCenters	vmware-aiops attention	Every vCenter merged into one globally-ranked issue list; unreachable targets degrade gracefully
"Is anything on fire?" across all clusters	vmware-aiops summary	Every cluster's hosts + VM power + live CPU/mem + alarms → ranked top-N issues + per-cluster status
"What's happening around this VM?"	vmware-aiops investigate vm <name>	VM state + host + cluster + backing datastores + snapshots + alarms + performance + a merged event timeline
"What's happening around this host?"	vmware-aiops investigate host <name>	Host state + cluster + the VMs it runs + mounted datastores + alarms + performance + correlated timeline
"What's happening around this datastore?"	vmware-aiops investigate datastore <name>	Capacity/free + mounting hosts + VMs it backs + alarms + correlated timeline

vmware-aiops attention                            # what needs attention now, all vCenters
vmware-aiops investigate vm web-01 --hours 72     # everything around a VM, then act on it
vmware-aiops investigate vm web-01 --html         # → offline snapshot in ~/vmware-health/

Via MCP these are the tools cluster_health_summary, cross_vcenter_attention, vm_investigation_bundle, host_investigation_bundle, datastore_investigation_bundle. (Requires vmware-monitor installed.)

Quick Install (Recommended)

Works with Claude Code, Cursor, Codex, Gemini CLI, Trae, and 30+ AI agents:

# Via Skills.sh
npx skills add vmware-skills/VMware-AIops

# Via ClawHub
clawhub install @zw008/vmware-aiops

PyPI Install (No GitHub Access Required)

# Install via uv (recommended)
uv tool install vmware-aiops

# Or via pip
pip install vmware-aiops

# China mainland mirror (faster)
pip install vmware-aiops -i https://pypi.tuna.tsinghua.edu.cn/simple

Offline / Air-Gapped Install (from source)

This project uses the modern PEP 517 build system (hatchling), so there is no setup.py by design — that is expected, not a missing file. If you cloned the source and hit ERROR: File "setup.py" or "setup.cfg" not found ... editable mode currently requires a setuptools-based build, your pip is older than 21.3 and cannot do an editable (-e) install with a non-setuptools backend. Editable mode is a developer convenience, not needed to run the tool — do one of:

# From the source tree — a normal (non-editable) install builds a wheel:
pip install .              # NOT  pip install -e .

# ...or upgrade pip first, and editable works too:
pip install --upgrade pip && pip install -e .

For a truly air-gapped host, build the wheels on a connected machine and copy them over — the target then needs no network:

# On a connected machine, collect this package + its dependencies as wheels:
pip wheel . -w dist        # → dist/*.whl   (or: uv build, for just this package)

# Copy dist/ to the air-gapped host, then install offline:
pip install --no-index --find-links dist vmware-aiops

---

Why this over other VMware MCP servers

Most open-source VMware MCP servers (e.g. bright8192/esxi-mcp-server, giuliolibrando/vmware-vsphere-mcp-server) are single-vCenter VM wrappers: list/power/snapshot a VM, basic monitoring, a confirm=True flag. They explicitly do not cover networking, storage, Kubernetes, ops analytics, load balancing, or compliance — and "logging is documented" is not an audit trail.

This is one skill in an 11-package family that covers the whole estate and runs every tool through a governed harness:

	Other VMware MCP servers	This family
VM lifecycle + monitoring	✅	✅
NSX networking (segments/gateways/NAT/routing/IPAM)	❌	✅ vmware-nsx
NSX security (DFW/groups/IDS-IPS/traceflow)	❌	✅ vmware-nsx-security
Storage (datastore/iSCSI/vSAN)	❌	✅ vmware-storage
Tanzu Kubernetes (Supervisor/Namespace/TKC)	❌	✅ vmware-vks
Aria Operations (metrics/alerts/capacity)	❌	✅ vmware-aria
AVI / NSX ALB load balancing + AKO	❌	✅ vmware-avi
Compliance baselines + drift (CIS/SCG/等保/PCI)	❌	✅ vmware-harden
Governed harness (unified audit, policy engine, token budget + runaway breaker, graduated risk tiers, undo-token, prompt-injection sanitize)	❌	✅ vmware-policy on every tool

If you only ever power-cycle VMs in one vCenter, a single-file server is fine. If you run a real (regulated, NSX-segmented, multi-domain) VMware estate and need an AI operator an auditor can sign off on, that's what this family is for — see docs/compliance-ready.md.

Capabilities Overview

What This Skill Does

Category	Tools	Count
VM Lifecycle	power on/off, TTL auto-delete, clean slate	6
Deployment	OVA, template, linked clone, batch clone/deploy	8
Guest Ops	exec commands, upload/download files, provision	5
Plan/Apply	multi-step planning with rollback	4
Cluster	create, delete, HA/DRS config, add/remove hosts	6
Datastore	browse files, scan for images	2
Network	dvSwitch portgroup list/create, host VMkernel list/add/remove, DF-bit MTU-path ping	6

CLI vs MCP: Which Mode to Use

Scenario	Recommended	Why
Local/small models (Ollama, Qwen <32B)	CLI	~2K tokens context vs ~10K for MCP; small models struggle with many tool schemas
Token-sensitive workflows	CLI	SKILL.md + Bash tool = minimal overhead
Cloud models (Claude, GPT-4o)	Either	Both work; MCP gives structured JSON I/O
Automated pipelines / Agent chaining	MCP	Type-safe parameters, structured output, no shell parsing
Monitoring / storage / K8s	Companion skills	See vmware-monitor, vmware-storage, vmware-vks

Rule of thumb: Use CLI for cost efficiency and small models. Use MCP for structured automation with large models.

Architecture

User (Natural Language)
  ↓
AI CLI Tool (Claude Code / Gemini / Codex / Aider / Continue / Trae / Kimi)
  ↓ reads SKILL.md / AGENTS.md / rules
  ↓
vmware-aiops CLI
  ↓ pyVmomi (vSphere SOAP API)
  ↓
vCenter Server ──→ ESXi Cluster ──→ VM
    or
ESXi Standalone Host ──→ VM

Version Compatibility

vSphere / VCF Version	Support	Notes
VCF 9.1 / vSphere 9.1	✅ Full	Released 2026-05-12. pyVmomi <10.0 resolves and connects via SOAP; new REST-only features (PATCH /deployment/size, IPv6-only GOSC) not yet wrapped — see VCF Python SDK for those.
VCF 9.0 / vSphere 9.0	✅ Full	pyVmomi 8.0.3+ connects against vSphere 9 SOAP API. From VCF 9, pyVmomi is also bundled inside the unified VCF Python SDK.
8.0 / 8.0U1-U3	✅ Full	CreateSnapshot_Task deprecated → use CreateSnapshotEx_Task
7.0 / 7.0U1-U3	✅ Full	All APIs supported
6.7	✅ Compatible	Backward-compatible, tested
6.5	✅ Compatible	Backward-compatible, tested

pyVmomi auto-negotiates the API version during SOAP handshake — no manual configuration needed. The same codebase manages 7.0 / 8.0 / 9.0 / 9.1 environments seamlessly.

Official Broadcom References

• SDKs: https://developer.broadcom.com/sdks — VCF Python SDK (recommended for VCF 9+, bundles pyVmomi + vSAN SDK), vSphere Automation SDK for Python (REST APIs)
• REST APIs: https://developer.broadcom.com/xapis — vSphere Automation API, VCF API, SDDC Manager API
• CLI Tools: https://developer.broadcom.com/tools — PowerCLI 9.1, ESXCLI, OVF Tool

---

Common Workflows

Deploy a Lab Environment

1. Browse datastore for OVA images → vmware-aiops datastore browse <ds> --pattern "*.ova"
2. Deploy VM from OVA → vmware-aiops deploy ova ./image.ova --name lab-vm --datastore ds1
3. Install software inside VM → vmware-aiops vm guest-exec lab-vm --cmd /bin/bash --args "-c 'apt-get install -y nginx'" --user root
4. Create baseline snapshot → vmware-aiops vm snapshot-create lab-vm --name baseline
5. Set TTL for auto-cleanup → vmware-aiops vm set-ttl lab-vm --minutes 480

Batch Clone for Testing

1. Create plan: vm_create_plan with multiple clone + reconfigure steps
2. Review plan with user (shows affected VMs, irreversible warnings)
3. Apply: vm_apply_plan executes sequentially, stops on failure
4. If failed: vm_rollback_plan reverses executed steps
5. Set TTL on all clones for auto-cleanup

Migrate VM to Another Host

1. Check VM info via vmware-monitor → verify power state and current host
2. Migrate: vmware-aiops vm migrate my-vm --to-host esxi-02
3. Verify migration completed

---

VM Lifecycle

Operation	Command	Confirmation	vCenter	ESXi
Power On	vm power-on <name>	—	✅	✅
Graceful Shutdown	vm power-off <name>	Double	✅	✅
Force Power Off	vm power-off <name> --force	Double	✅	✅
Reset	vm reset <name>	—	✅	✅
Suspend	vm suspend <name>	—	✅	✅
Create VM	vm create <name> --cpu --memory --disk	—	✅	✅
Delete VM	vm delete <name>	Double	✅	✅
Reconfigure	vm reconfigure <name> --cpu --memory	Double	✅	✅
Create Snapshot	vm snapshot-create <name> --name <snap>	—	✅	✅
List Snapshots	vm snapshot-list <name>	—	✅	✅
Revert Snapshot	vm snapshot-revert <name> --name <snap>	Double	✅	✅
Delete Snapshot	vm snapshot-delete <name> --name <snap> [--no-wait]	Double	✅	✅
Task Status	vm task-status <task-id>	—	✅	✅
Clone VM	vm clone <name> --new-name <new>	Double	✅	✅
vMotion	vm migrate <name> --to-host <host>	Double	✅	❌
Set TTL	vm set-ttl <name> --minutes <n>	Double	✅	✅
Cancel TTL	vm cancel-ttl <name>	—	✅	✅
List TTLs	vm list-ttl	—	✅	✅
Clean Slate	vm clean-slate <name> [--snapshot baseline]	Double	✅	✅
Guest Exec	vm guest-exec <name> --cmd /bin/bash --args "..."	Double	✅	✅
Guest Exec (with output)	vm guest-exec-output <name> --cmd "df -h"	—	✅	✅
Guest Upload	vm guest-upload <name> --local f.sh --guest /tmp/f.sh	Double	✅	✅
Guest Download	vm guest-download <name> --guest /var/log/syslog --local ./syslog	—	✅	✅

Guest Operations require VMware Tools running inside the guest OS. guest-exec-output auto-detects Linux/Windows shell and captures stdout/stderr.

Plan → Apply (Multi-step Operations)

For complex operations involving 2+ steps or 2+ VMs, use the plan/apply workflow instead of executing individually:

Step	What Happens
1. Create Plan	AI calls vm_create_plan — validates actions, checks targets in vSphere, generates plan with rollback info
2. Review	AI shows plan to user: steps, affected VMs, irreversible warnings
3. Apply	vm_apply_plan executes sequentially; stops on failure
4. Rollback (if failed)	Asks user whether to rollback, then vm_rollback_plan reverses executed steps (irreversible steps skipped)

Plans stored in ~/.vmware-aiops/plans/, auto-deleted on success, auto-cleaned after 24h.

VM Deployment & Provisioning

Operation	Command	Speed	vCenter	ESXi
Deploy from OVA	deploy ova <path> --name <vm>	Minutes	✅	✅
Deploy from Template	deploy template <tmpl> --name <vm>	Minutes	✅	✅
Linked Clone	deploy linked-clone --source <vm> --snapshot <snap> --name <new>	Seconds	✅	✅
Attach ISO	deploy iso <vm> --iso "[ds] path/to.iso"	Instant	✅	✅
Convert to Template	deploy mark-template <vm>	Instant	✅	✅
Batch Clone	deploy batch-clone --source <vm> --count <n>	Minutes	✅	✅
Batch Deploy (YAML)	deploy batch spec.yaml	Auto	✅	✅

Cluster Management

Operation	Command	Confirmation	vCenter	ESXi
Cluster Info	cluster info <name>	—	✅	❌
Create Cluster	cluster create <name> [--ha] [--drs]	—	✅	❌
Delete Cluster	cluster delete <name>	Double	✅	❌
Add Host	cluster add-host <cluster> --host <host>	Double	✅	❌
Remove Host	cluster remove-host <cluster> --host <host>	Double	✅	❌
Configure HA/DRS	cluster configure <name> [--ha/--no-ha] [--drs/--no-drs]	Double	✅	❌

remove-host requires the host to be in maintenance mode first; the host is moved out of the cluster into the datacenter's host folder as a standalone host.

Alarm Management

Operation	Command	Confirmation	vCenter	ESXi
List Triggered Alarms	alarm list [--target <t>]	—	✅	❌
Acknowledge Alarm	alarm acknowledge <entity> <alarm>	—	✅	❌
Clear (Reset) Alarms	alarm reset <entity> <alarm>	Double	✅	❌

Blast radius: vSphere has no per-alarm clear API. alarm reset uses AlarmManager.ClearTriggeredAlarms, which clears all triggered alarms matching the named alarm's entity type (host/VM/all) and current status (red/yellow) — not just the named one. The named alarm is looked up first (typos fail fast), and the output's scope field reports exactly what was cleared. Cleared alarms re-trigger automatically if their underlying condition persists.

Datastore Browser

Feature	vCenter	ESXi	Details
Browse Files	✅	✅	List files/folders in any datastore path
Scan Images	✅	✅	Discover ISO, OVA, OVF, VMDK across all datastores

Scheduled Scanning & Notifications

Feature	Details
Daemon	APScheduler-based, configurable interval (default 15 min)
Multi-target Scan	Sequentially scan all configured vCenter/ESXi targets
Scan Content	Alarms + Events + Host logs (hostd, vmkernel, vpxd)
Log Analysis	Regex pattern matching: error, fail, critical, panic, timeout, corrupt
Structured Log	JSONL output to ~/.vmware-aiops/scan.log
Webhook	Slack, Discord, or any HTTP endpoint
Daemon Management	daemon start/stop/status, PID file, graceful shutdown

Safety Features

Feature	Details
Dry-Run Mode (CLI only)	--dry-run on any destructive CLI command prints exact API calls without executing
Plan → Confirm → Execute → Log	CLI workflow: show current state, confirm changes, execute, audit log
Double Confirmation (CLI only)	Destructive CLI commands (power-off, delete, reconfigure, snapshot-revert/delete, clean-slate, guest-exec, guest-upload, cluster delete/remove-host, alarm clear) require 2 sequential prompts and take no bypass flag
No confirmation on the MCP path	The 43 write tools an agent sees over MCP act on the first call — no confirmed= handshake, no approval tier, no read-only switch. What decides whether a write lands is the privilege of the vCenter account, and what records it is the audit trail. See What protects you
Rejection Logging	Declined CLI confirmations are recorded in the audit trail
Audit Trail	All operations logged to ~/.vmware-aiops/audit.log (JSONL) with before/after state
Input Validation	VM name, CPU (1-128), memory (128-1048576 MB), disk (1-65536 GB) validated
Password Protection	.env file loading with permission check; never in shell history
SSL Self-signed Support	verify_ssl: false — only for ESXi with self-signed certs in isolated labs; production should use CA-signed certificates
Prompt Injection Protection	vSphere event messages and host logs are truncated, stripped of control characters, and wrapped in boundary markers before output
Webhook Data Scope	Sends notifications to user-configured URLs only — no third-party services by default
Task Waiting	All async operations wait for completion and report result
State Validation	Pre-operation checks (VM exists, power state correct)

vCenter vs ESXi Comparison

Capability	vCenter	ESXi Standalone
vMotion migration	✅	❌
Cross-host clone	✅	❌
Cluster management	✅	❌
All VM lifecycle ops	✅	✅
OVA/Template/Linked Clone deploy	✅	✅
Datastore browsing & image scan	✅	✅
Snapshots	✅	✅
Guest operations	✅	✅

Inventory, alarms, events, sensors, host services, and scanning are now in vmware-monitor.

What protects you

The table above lists two different surfaces and it is worth being blunt about which protections apply to which, because getting this wrong is worse than having no protection at all — a guardrail you believe in is one you stop compensating for.

On the CLI, a destructive command asks twice and takes no bypass flag, and --dry-run previews any write. That defends a mistyped command typed by a human. It does not defend against an agent, which satisfies both prompts with yes |.

Over MCP, there is no confirmation step at all. All 43 write tools — vm_delete, cluster_delete, vm_guest_exec among them — act on the first call. Seven host-networking and DRS tools take a confirm argument that defaults to a no-write preview, but that is a preview switch, not an approval gate: one more call is all it takes. This is deliberate. A confirmed= handshake was considered in July 2026 and cut, along with the earlier VMWARE_READ_ONLY switch, because neither was a real boundary — the switch was enforced on the MCP path only and any agent with a shell walked around it via the CLI, and a handshake is a speed-bump a model that intends to act steps over.

What actually decides whether a write lands is the vCenter/ESXi service account. Give the skill an account with the privileges the work needs and no more; vCenter refuses the rest itself, on every surface, with no way around it from inside this skill. To run an agent read-only, give it a read-only vCenter role — one decision, enforced where it is made. Every call is then recorded in ~/.vmware/audit.db before the caller sees a result, which is how you find out what happened.

vm_guest_exec is the one to think hardest about. It runs a caller-supplied command inside the guest OS with the credentials handed to it, which the documentation's own example makes root; nothing bounds what the command may be. The guest account is a separate authorization boundary from the vCenter one — a read-only vCenter role does not constrain what this tool does inside a VM. If you do not need guest operations, do not configure guest credentials.

The full inventory of which tools are gated and which are not is in references/capabilities.md, where the numbers are checked against the live tool registry by the test suite rather than maintained by hand.

---

Troubleshooting

"VM not found" error

VM names are case-sensitive in vSphere. Use exact name from vmware-monitor inventory vms.

Guest exec returns empty output

Use vm_guest_exec_output instead of vm_guest_exec — it auto-captures stdout/stderr. Basic vm_guest_exec only returns exit code.

Deploy OVA times out

Large OVA files (>10GB) may exceed the default 120s timeout. The upload happens via HTTP NFC lease — ensure network between the machine running vmware-aiops and ESXi is stable.

Plan apply fails mid-way

Run vmware-aiops plan list to see failed plan status. Ask user if they want to rollback with vm_rollback_plan. Irreversible steps (delete_vm) are skipped during rollback.

Connection refused / SSL error

1. Verify target is reachable: vmware-aiops doctor
2. For self-signed certs: set verify_ssl: false in config.yaml (lab environments only)

---

Supported AI Platforms

Platform	Status	Config File	AI Model
Claude Code	✅ Native Skill	skills/vmware-aiops/SKILL.md	Anthropic Claude
Gemini CLI	✅ Context file + MCP	skills/vmware-aiops/SKILL.md	Google Gemini
OpenAI Codex CLI	✅ Skill + AGENTS.md	skills/vmware-aiops/SKILL.md	OpenAI GPT
Aider	✅ Conventions	skills/vmware-aiops/SKILL.md	Any (cloud + local)
Continue CLI	✅ Rules	skills/vmware-aiops/SKILL.md	Any (cloud + local)
Trae IDE	✅ Rules	skills/vmware-aiops/SKILL.md	Claude/DeepSeek/GPT-4o/Doubao
Kimi Code CLI	✅ Skill	skills/vmware-aiops/SKILL.md	Moonshot Kimi
MCP Server	✅ MCP Protocol	vmware_aiops/mcp_server/	Any MCP client
Python CLI	✅ Standalone	N/A	N/A

Platform Comparison

Feature	Claude Code	Gemini CLI	Codex CLI	Aider	Continue	Trae IDE	Kimi CLI
Cloud AI	Anthropic	Google	OpenAI	Any	Any	Multi	Moonshot
Local models	—	—	—	Ollama	Ollama	—	—
Skill system	SKILL.md	Context file	SKILL.md	—	Rules	Rules	SKILL.md
MCP support	Native	Native	Via Skills	Third-p