GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — Nas backup: Fix mount/unmount error handling and timeout in LibvirtRestoreBackupCommandWrapper by abh1sar · Pull Request #14006 · apache/cloudstack Conversation
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@ ## 4.22 #14006 +/- ## ========================================= Coverage 17.86% 17.86% - Complexity 16037 16039 +2 ========================================= Files 5928 5928 Lines 534479 534496 +17 Branches 65410 65412 +2 ========================================= + Hits 95468 95479 +11 - Misses 428173 428179 +6 Partials 10838 10838
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan package |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19015 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This PR fixes restore-from-backup behavior for KVM/libvirt by correctly treating failed mount/umount operations as failures (via exit codes instead of relying on exceptions) and by restoring the intended command timeouts so restores don’t hang for Script’s 1-hour default.
Changes:
Script.executeCommandForExitValue(timeout, ...) and fail fast on non-zero exit codes.executeCommandForExitValue(...) and fail on non-zero exit codes (instead of silently ignoring failure).Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtRestoreBackupCommandWrapper.java | Validates mount/umount exit codes and restores rsync timeout usage during restore operations. |
| plugins/hypervisors/kvm/src/test/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtRestoreBackupCommandWrapperTest.java | Updates mocks for timeout-aware executeCommandForExitValue and adds a test asserting the configured mount timeout is used. |
plugins/hypervisors/kvm/src/test/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtRestoreBackupCommandWrapperTest.java:605
if (Arrays.stream(invocation.getArguments()).map(String::valueOf).anyMatch("mount"::equals)) {
mountTimeout[0] = invocation.getArgument(0);
return 1; // stop the restore right after the mount
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
…repository The directory created for the mount is removed by the caller in a finally block, but that block is only reached once the mount has succeeded, so a repository that cannot be mounted left an empty directory behind on every attempt. It is now removed before the failure is reported, best effort and logged if it cannot be. The unmount ran without a timeout and so fell back to the one hour default in Script. Unmounting a repository that has become unreachable blocks as easily as mounting one, and this runs in the cleanup path of a restore, so it is now bounded by the configured mount timeout like the mount itself.
|
Hi @shwstppr can you please take a look |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan package |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
code lgtm
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19173 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Tested LGTM,
Environment: Advanced zone 2x KVM hosts.
Success path — PASS
Failure path — PASS
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
code lgtm
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan package |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19239 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan test |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
[SF] Trillian test result (tid-16986)
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
merging based on approvals and Abhishek's manual verification |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Conflicts: - LibvirtRestoreBackupCommandWrapperTest.java: apache#14006 appended three mount/unmount timeout tests at the end of the class, where the volume mapping tests end too; kept both.
…eouts apache#14006 runs the mount, the umount and the rsync of a restore through Script.executeCommandForExitValue(long, String...) so that they honour the configured timeouts. The volume mapping tests still stubbed and verified the overloads without a timeout. The two tests asserting where a backup is written therefore failed, and the checks that a backup is never written into the wrong volume passed without verifying anything, as nothing called that overload any more. Stub and verify the timeout variants instead, as apache#14006 does for the existing tests.
Resolves conflicts with the incremental NAS backup backport (apache#13877) and the restore mount/unmount timeout fix (apache#14006). Both sets of zone settings and command fields are kept; the agent wrappers carry both the mount timeout and the LUKS key file, and the enhancement flags plus temporary passphrase file move into upstream's new runBackupScript. Compression and LUKS encryption rewrite each backup file with qemu-img convert, which would flatten an incremental through its backing chain (or leave an encrypted child whose parent needs its own secret). So a zone with either enabled now takes full backups (decideChain returns legacy-full), and nasbackup.sh refuses incremental mode combined with -c/-e as a backstop. Covered by decideChainReturnsLegacyFullWhenCompressionOrEncryptionEnabled. Claude-Session: https://claude.ai/code/session_011MtPDN3Yg3DR8wu4EN8nkp
Since apache#14006 mount/umount also go through Script.executeCommandForExitValue, so the LUKS restore tests' call capture picked them up (4 calls, not 2). Filter the capture to qemu-img command lines; the no-passphrase test now asserts no qemu-img command ran instead of no command at all. Claude-Session: https://claude.ai/code/session_011MtPDN3Yg3DR8wu4EN8nkp
…storeBackupCommandWrapper (apache#14006) * kvm: detect mount failures and honour the configured timeouts on backup restore Script.executeCommand returns null when the command fails, it does not throw, so the try/catch around the mount and umount of the backup repository could never fire and the return value was discarded. A repository that fails to mount was therefore treated as mounted, and the restore carried on against an empty directory until it failed later with a misleading "backup file not found". A failed umount was ignored the same way, leaking the mount. Both now go through executeCommandForExitValue and check the exit value. The same refactor also dropped the timeouts. mountTimeout was still passed into mountBackupDirectory but never used, and the rsync of the volume lost the command timeout, so both fell back to the one hour default in Script instead of the configured values. An unresponsive repository could hold a restore up for an hour rather than failing after nas.backup.restore.mount.timeout seconds. * kvm: clean up after a failed mount and bound the unmount of a backup repository The directory created for the mount is removed by the caller in a finally block, but that block is only reached once the mount has succeeded, so a repository that cannot be mounted left an empty directory behind on every attempt. It is now removed before the failure is reported, best effort and logged if it cannot be. The unmount ran without a timeout and so fell back to the one hour default in Script. Unmounting a repository that has become unreachable blocks as easily as mounting one, and this runs in the cleanup path of a restore, so it is now bounded by the configured mount timeout like the mount itself.
* origin/main: (156 commits) Upgrade Netris SDK version to 1.1.0 since 24.0 (apache#14365) feature: instance boot group and readiness rules (apache#13907) dont show unathorized notification for prelogin (apache#14054) .github: Upgrade docker/login-action to fix ASF Allowlist failure (apache#14276) virtual router: Add route-maps to BGP peers for Routed Mode (apache#9964) Add pre-commit hook to validate dependabot.yml (apache#14107) Add the ASF Allowlist check workflow (apache#14113) Revert "VMware to KVM: cold (VDDK) and warm (CBT) migration into Ceph/RBD and Linstor, plus importVm adoption of existing RBD root volumes and Linstor root/data volumes (apache#13656)" VMware to KVM: cold (VDDK) and warm (CBT) migration into Ceph/RBD and Linstor, plus importVm adoption of existing RBD root volumes and Linstor root/data volumes (apache#13656) engine-schema: upgrade path for 24.0.0 (apache#14033) xenserver/xcpng: do not bypass secondary storage when copy volumes between pools (apache#13750) backport: Incremental NAS backup support for KVM (apache#13074) (apache#13877) storage: KVM - enable RBD/Ceph volume encryption support (apache#13556) backup: remove the powered-off precondition for all backup providers (apache#14094) Nas backup: Fix mount/unmount error handling and timeout in LibvirtRestoreBackupCommandWrapper (apache#14006) kvm: apply rbd_default_data_pool when creating volumes from templates on RBD (apache#13361) server: copy template details to vm instance details when import VM (apache#12793) marvin: use pycryptodome instead of PyCrypt (apache#13594) Packaging: accept normalized Marvin archive names (apache#13793) SharedFS: fix NPE when accessing systemvm templates for unsupported hypervisors (apache#13930) ...
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.