backup: veeam kvm integration by shwstppr · Pull Request #12991 · apache/cloudstack

shwstppr · 2026-04-09T17:09:43Z

Description

Design spec: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=421954133

This PR introduces the initial implementation of Veeam integration support for KVM in CloudStack by adding a UHAPI-compatible server and image server components.

Veeam Backup & Replication interacts with virtualization platforms using its Universal Hypervisor API (UHAPI). To enable backup and restore workflows for CloudStack-managed KVM environments, this change introduces a UHAPI server that exposes CloudStack resources through a UHAPI-compatible interface.

In addition to the control plane APIs, an image server component is introduced to handle the data transfer operations required during backup and restore workflows.

Architecture

The integration consists of two main components:

UHAPI Server (Control Plane) named CloudStack Veeam Control Service

A lightweight UHAPI server runs inside the CloudStack management server and exposes endpoints under:

/ovirt-engine
    - /api - For APIs
    - /sso - For authentication
    - /services/pki-resource - For certificates

This server provides inventory discovery APIs required by Veeam and translates CloudStack resources into the structures expected by UHAPI.

The server:

exposes infrastructure inventory
handles authentication and session tokens
maps CloudStack resources to UHAPI-compatible representations

Image Server (Data Plane) named CloudStack Image Service

A separate image server component is introduced to handle backup and restore data transfer operations.

This component:

serves disk image data during backup
receives image data during restore operations
exposes endpoints used by Veeam worker components
integrates with CloudStack storage to read and write VM disk data

The separation between both these components server ensures that:

metadata APIs and control operations remain lightweight
bulk image transfer operations are handled independently

Documentation PR: apache/cloudstack-documentation#642

Co-authored by @abh1sar @weizhouapache

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Signed-off-by: Abhishek Kumar <[email protected]>

todo: patch (needed?)

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2026-04-29T05:33:21Z

@blueorangutan package

blueorangutan · 2026-04-29T05:34:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-29T07:01:41Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17647

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr · 2026-04-29T16:08:22Z

@blueorangutan test

blueorangutan · 2026-04-29T16:10:04Z

@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

github-actions · 2026-04-30T03:29:36Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

Signed-off-by: Abhishek Kumar <[email protected]>

blueorangutan · 2026-04-30T07:07:29Z

[SF] Trillian test result (tid-15987)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 50128 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12991-t15987-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

shwstppr · 2026-04-30T07:13:20Z

@blueorangutan package

blueorangutan · 2026-04-30T07:14:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-30T08:07:04Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17665

weizhouapache · 2026-04-30T07:33:28Z

+# Enable TLS for image server transfers. The keys are read from:
+# cert file = /etc/cloudstack/agent/cloud.crt
+# key file = /etc/cloudstack/agent/cloud.key
+image.server.tls.enabled=true


@shwstppr
is /etc/cloudstack/agent/cloud.ca.crt used ?

@abh1sar can tell better but I think yes we use cloud certificates for image server.
To the Veeam worker VM, we just pass the Root CA from the MS

that's right. Is it ok to show the file names?

got it, the control service and image service use the same Root CA (these key/cert are generated from it too), right ?

No, control service uses the certificate configured for the management server by the operator.
Image service uses the internal CA.
control service CA can be different for example in case when multiple management servers are behind LB and that LB has its own SSL config.

weizhouapache · 2026-04-30T07:42:11Z

+import com.cloud.utils.EnumUtils;
+
+@APICommand(name = "createImageTransfer",
+        description = "Create image transfer for a disk in backup. This API is intended for testing only and is disabled by default.",


This API is intended for testing only and is disabled by default.

this sentence exists in all APIs in this folder. is this correct ? @abh1sar

yes Wei, I don't want to expose these APIs to end users. Only Veeam control service uses them.
But they are useful in testing. For example, we can run integration tests using these even without veeam.
Any better way to handle this?

maybe add a global setting and expose them only when the setting is set to true ?

yes, the global setting is already there expose.kvm.backup.export.service.apis

weizhouapache · 2026-04-30T08:25:03Z

@@ -10075,4 +10113,33 @@ private void setVncPasswordForKvmIfAvailable(Map<String, String> customParameter
            vm.setVncPassword(customParameters.get(VmDetailConstants.KVM_VNC_PASSWORD));
        }
    }
+
+    protected boolean isBlankInstanceDefaultTemplate(VirtualMachineTemplate template) {
+        return KVM_VM_DUMMY_TEMPLATE_NAME.equals(template.getUniqueName());


maybe rename to KVM_BLANK_VM_TEMPLATE_NAME or so

weizhouapache · 2026-04-30T08:30:13Z

+    'listVmCheckpoints' : 'Backup and Recovery',
+    'deleteVmCheckpoint' : 'Backup and Recovery',
+    'ImageTransfer' : 'Backup and Recovery',
+    'VmCheckpoint' : 'Backup and Recovery',
    'UnmanagedInstance': 'Virtual Machine',


maybe only the last two are needed

'ImageTransfer' : 'Backup and Recovery', 'VmCheckpoint' : 'Backup and Recovery',

weizhouapache · 2026-04-30T08:30:30Z

@@ -87,6 +87,7 @@ export default {
    }
  },
  created () {
+    console.log('---------------', this.$route.meta.name)


remove it ?

weizhouapache

overall LGTM
left some minor comments and questions.

great job @shwstppr @abh1sar !

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr and others added 30 commits April 9, 2026 09:38

[WIP] plugins: veeam control service

7c23b26

Signed-off-by: Abhishek Kumar <[email protected]>

wip

065ec85

Signed-off-by: Abhishek Kumar <[email protected]>

changes for discovery

a30eb28

Signed-off-by: Abhishek Kumar <[email protected]>

changes

f52b114

Signed-off-by: Abhishek Kumar <[email protected]>

changes

2784468

Signed-off-by: Abhishek Kumar <[email protected]>

changes

81c3b5b

Signed-off-by: Abhishek Kumar <[email protected]>

Basic working version-1

f396c5c

Create volume on the given storage pool

73df3cb

Image server basic working version in SSVM.

23ecb1f

Image server with disk upload

5389fe6

upload fix-1

aae158b

image upload working

10f65b6

wip: changes for imagetransfer handling

7b45d2e

Signed-off-by: Abhishek Kumar <[email protected]>

Added progress to upload Image Transfers

2350661

get Options to return capabilities for upload

9ee9748

add license to image_server.py

f83fd00

fix precommit, license

2bc3114

Signed-off-by: Abhishek Kumar <[email protected]>

veeam control changes

3460a5d

Signed-off-by: Abhishek Kumar <[email protected]>

server changes

b926c74

Signed-off-by: Abhishek Kumar <[email protected]>

Support multiple disks and checkpoints

da62e9a

extents(zero/dirty) and capabilities - working

4173947

todo: patch (needed?)

Patch (zero, data) + Flush support in image_server.py

91a081b

Backup of stopped VMs

c36cd2c

api/server: create dummy KVM VM without volume and network is optional

ca4112e

worker vm deployment wip

a366929

Signed-off-by: Abhishek Kumar <[email protected]>

Support file backend for cow format: api and server

586134d

Image server support for file backend (qcow2 upload)

6ca1c9b

ut failure in UserVmManagerImplTest

fba7c63

wip

a89f872

Signed-off-by: Abhishek Kumar <[email protected]>

fixes to allow worker vm deployment

106fbdb

Signed-off-by: Abhishek Kumar <[email protected]>

address comments

574f0ea

Signed-off-by: Abhishek Kumar <[email protected]>

apache deleted a comment from blueorangutan Apr 29, 2026

minor cleanup and tests

568c1aa

Signed-off-by: Abhishek Kumar <[email protected]>

github-actions Bot added the status:has-conflicts label Apr 30, 2026

shwstppr added 3 commits April 30, 2026 11:29

more tests, fix pre-commit

7bdd703

Signed-off-by: Abhishek Kumar <[email protected]>

Merge remote-tracking branch 'apache/main' into integration-veeam-kvm

eaab07d

fix datacenter href bug

905be92

Signed-off-by: Abhishek Kumar <[email protected]>

github-actions Bot removed the status:has-conflicts label Apr 30, 2026

another pre-commit fix

1f306b7

Signed-off-by: Abhishek Kumar <[email protected]>

shwstppr changed the title ~~[WIP] backup: veeam kvm integration~~ backup: veeam kvm integration Apr 30, 2026

weizhouapache reviewed Apr 30, 2026

View reviewed changes

weizhouapache approved these changes Apr 30, 2026

View reviewed changes

abh1sar and others added 7 commits May 4, 2026 14:56

address review comments

1f9cbd4

fix ut and remove sudo from LibvirtDeleteVmCheckpointCommand

c65dfa1

address review comments

1a37925

Signed-off-by: Abhishek Kumar <[email protected]>

filter storagedomains for hypervisor

100a5c5

Signed-off-by: Abhishek Kumar <[email protected]>

conditional logging for request data

b452d20

Signed-off-by: Abhishek Kumar <[email protected]>

fix parsing ovf memory

83490a9

Signed-off-by: Abhishek Kumar <[email protected]>

fix license

8038dd6

Signed-off-by: Abhishek Kumar <[email protected]>

Conversation

shwstppr commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

shwstppr commented Apr 29, 2026

Uh oh!

blueorangutan commented Apr 29, 2026

Uh oh!

blueorangutan commented Apr 29, 2026

Uh oh!

shwstppr commented Apr 29, 2026

Uh oh!

blueorangutan commented Apr 29, 2026

Uh oh!

github-actions Bot commented Apr 30, 2026

Uh oh!

blueorangutan commented Apr 30, 2026

Uh oh!

shwstppr commented Apr 30, 2026

Uh oh!

blueorangutan commented Apr 30, 2026

Uh oh!

blueorangutan commented Apr 30, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abh1sar Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

shwstppr commented Apr 9, 2026 •

edited

Loading

abh1sar Apr 30, 2026 •

edited

Loading