GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — smoke test: support ceph storage pools by weizhouapache · Pull Request #11931 · apache/cloudstack Conversation
|
|
@blueorangutan package |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@ ## main #11931 +/- ## ============================================ + Coverage 19.78% 20.67% +0.88% - Complexity 19995 21726 +1731 ============================================ Files 6371 6492 +121 Lines 575909 590737 +14828 Branches 70509 72448 +1939 ============================================ + Hits 113950 122120 +8170 - Misses 449526 455343 +5817 - Partials 12433 13274 +841
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15584 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan test matrix |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@weizhouapache a [SL] Trillian-Jenkins matrix job (EL8 mgmt + EL8 KVM, Ubuntu22 mgmt + Ubuntu22 KVM, EL8 mgmt + VMware 7.0u3, EL9 mgmt + XCP-ng 8.2 ) has been kicked to run smoke tests |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[SF] Trillian test result (tid-14741)
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[SF] Trillian test result (tid-14742)
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan package |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15597 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan test |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[SF] Trillian test result (tid-14744)
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[SF] Trillian test result (tid-14753)
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[SF] Trillian test result (tid-14743)
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
ddfdacf to
844de76
Compare
|
ready for review and testing |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan package |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
When a host has no explicit tags initially, cls.host.hosttags is None. Passing None to the API does not set the hosttags parameter, so tags are never cleared during cleanup. Convert None to empty string to properly clear tags that were set during the test.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Storage selection can incorrectly run or skip tests, and failed restore tests can leave backups behind.
Review effort: Balanced
Findings: 6
Extends CloudStack’s smoke tests to support Ceph/RBD storage while skipping unsupported operations.
Changes:
| File | Description |
|---|---|
| test/integration/smoke/test_volumes.py | Adjusts zone-wide pool assertions and encryption skips. |
| test/integration/smoke/test_vm_snapshots.py | Adds an RBD snapshot skip. |
| test/integration/smoke/test_vm_life_cycle.py | Skips live migration between RBD pools. |
| test/integration/smoke/test_snapshots.py | Allows RBD snapshot coverage. |
| test/integration/smoke/test_over_provisioning.py | Includes RBD overprovisioning tests. |
| test/integration/smoke/test_host_tags.py | Sends empty strings to clear host tags. |
| test/integration/smoke/test_direct_download.py | Includes RBD in shared-storage coverage. |
| test/integration/smoke/test_backup_recovery_nas.py | Adds Ceph support, repository selection, incremental skips, and restore coverage. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| storage_pools = StoragePool.list(cls.api_client) | ||
| usable_pools = [p for p in storage_pools if getattr(p, 'state', 'Up') == 'Up'] | ||
| cls.storage_pool = usable_pools[0] if usable_pools else storage_pools[0] | ||
| if cls.storage_pool.type.lower() not in SUPPORTED_PRIMARY_STORAGE_POOL_TYPES: | ||
| cls.skipTest(cls, reason="Test can be run only if the primary storage is of type NFS or RBD (Ceph)") |
| if self.storage_pool.type.lower() == 'rbd': | ||
| self.skipTest("Incremental backups are not supported on RBD/Ceph primary Storage") |
| Backup.delete(self.apiclient, backup.id) | ||
| finally: |
| ) | ||
| for pool in storage_pools: | ||
| if not cls.nfsStorageFound and pool.type == "NetworkFilesystem": | ||
| if not cls.nfsStorageFound and pool.type in ("NetworkFilesystem", "RBD"): |
| list_volume_pool_response = list_storage_pools(cls.apiclient) | ||
| volume_pool = list_volume_pool_response[0] | ||
| if volume_pool.type == "RBD": | ||
| cls.skipTest(cls, reason="VM snapshot is unsupported for VMs on RBD storage pool") |
| list_volume_pool_response = list_storage_pools(cls.apiclient) | ||
| volume_pool = list_volume_pool_response[0] | ||
| if volume_pool.type == "RBD": | ||
| cls.skipTest(cls, reason="Volume encryption is unsupported for volumes on RBD storage pool") |
Zone-wide storage pools (scope=ZONE) are not bound to clusters or pods. When filtering by clusterid/podid, these pools may be included in results with clusterid=None and podid=None. Updated tests to: - test_01: Accept zone-wide pools in clusterid filter results - test_07: Accept zone-wide pools in podid filter results - test_12: New test validating zone-wide vs cluster-scoped pools Both invalid clusterid/podid filters now correctly return zone-wide pools, and assertions skip None values for zone-wide pools while validating cluster-scoped pools have their expected scope identifiers. Validated on two environments: - Zone-wide only (10.1.112.155): 12/12 tests passed - Cluster-scoped only (10.0.112.202): 12/12 tests passed
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
BackupOffering.listExternal(...) returns None or an empty list, indexing [0] will raise.… · NewassertEqual(len(backups), 1) will fail if the environment already has… · NewassertEqual(len(backups), 1) will fail if the environment already has… · Newcluster_scoped_found is set but never asserted, so the test can pass vacuously even if the filter… · Newpod_scoped_found is never asserted, so the test may pass… · Newscope attribute are silently ignored and won’t be validated by either branch,… · Newvolume_pool.type.lower(), suggesting… · Newvolume_pool.type.lower(), suggesting… · Newlist_storage_pools()[0], which can be the wrong pool in… · New🧠 Review effort: Lite
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| cls.provider_offerings = BackupOffering.listExternal(cls.api_client, cls.zone.id) | ||
| cls.backup_offering = BackupOffering.importExisting(cls.api_client, cls.zone.id, cls.provider_offerings[0].externalid, | ||
| cls.provider_offerings[0].name, cls.provider_offerings[0].description) | ||
| matching_offerings = [o for o in cls.provider_offerings if o.externalid == cls.backup_repository.id] | ||
| provider_offering = matching_offerings[0] if matching_offerings else cls.provider_offerings[0] | ||
| cls.backup_offering = BackupOffering.importExisting(cls.api_client, cls.zone.id, provider_offering.externalid, | ||
| provider_offering.name, provider_offering.description) |
| Backup.create(self.apiclient, self.vm.id, "restore_attach_backup") | ||
|
|
||
| backups = Backup.list(self.apiclient, self.vm.id) | ||
| self.assertEqual(len(backups), 1, "There should exist only one backup for the VM") | ||
| backup = backups[0] |
| Backup.delete(self.apiclient, backup.id) | ||
| finally: | ||
| if target_vm is not None: | ||
| try: | ||
| target_vm.delete(self.apiclient) | ||
| except Exception: | ||
| pass | ||
| self.backup_offering.removeOffering(self.apiclient, self.vm.id) |
| self.cluster.id, | ||
| "Cluster id should be equal to the cluster id passed in the filter" | ||
| ) | ||
|
|
| self.cluster.podid, | ||
| "Pod id should be equal to the pod id passed in the filter" | ||
| ) | ||
|
|
| zone_wide_pools = [p for p in storage_pools if hasattr(p, 'scope') and p.scope == "ZONE"] | ||
| cluster_scoped_pools = [p for p in storage_pools if hasattr(p, 'scope') and p.scope != "ZONE"] | ||
|
|
| id=vol_res[0].storageid) | ||
| volume_pool = volume_pool_response[0] | ||
| if volume_pool.type.lower() != 'networkfilesystem': | ||
| if volume_pool.type not in ("NetworkFilesystem", "RBD"): |
| self.assertTrue( | ||
| storage.type in ("NetworkFilesystem", "RBD"), | ||
| "Check storage pool type" |
| list_volume_pool_response = list_storage_pools(cls.apiclient) | ||
| volume_pool = list_volume_pool_response[0] | ||
| if volume_pool.type == "RBD": | ||
| cls.skipTest(cls, reason="VM snapshot is unsupported for VMs on RBD storage pool") |
| list_volume_pool_response = list_storage_pools(cls.apiclient) | ||
| volume_pool = list_volume_pool_response[0] | ||
| if volume_pool.type == "RBD": | ||
| cls.skipTest(cls, reason="Volume encryption is unsupported for volumes on RBD storage pool") |
TestVolumeEncryption previously skipped entirely on RBD storage pools. Now that RBD encryption is supported (PR apache#13556), extend it instead of skipping: - setUpClass no longer hard-skips RBD pools; records storage_pool_type for use by the assertions below. - check_volume_encryption() branches on pool type: RBD volumes are encrypted natively by librbd, so the domain XML carries <encryption format='luks2' engine='librbd'>; other pool types keep the existing qemu-native <encryption format='luks'> check. - does_host_with_encryption_support_exists() gained an RBD-specific path: the API's "encryptionsupported" field only reflects qemu-native LUKS support, so librbd capability is read from the host_details table (host.volume.encryption.rbd), joined on the host's uuid. - get_expected_guest_visible_size() centralizes the existing PowerFlex size-overhead adjustment and adds the equivalent for RBD: encrypted RBD volumes reserve a 16 MiB LUKS2 header out of the volume's own reported size, so the guest sees size - header. - test_03_root_and_data_volume_encryption now also exercises resize of the attached encrypted data volume, reusing its VM instead of spinning up a separate one: - online (VM stays running): verified via the volume's own API state and size, since picking up a live virtio-blk capacity change in the guest depends on the guest OS/kernel (the CentOS 5.5 template used here predates reliable live-resize notification) and is orthogonal to whether CloudStack/libvirt/qemu actually grew the volume. - offline (detach, resize, reattach): verified against the guest's fdisk output, matched by the expected byte count rather than a device letter derived from deviceid, since libvirt is not guaranteed to reuse the same target letter after a detach then reattach.
Deploy a VM from the direct download template on each type of storage found (NFS, RBD, SharedMountPoint and local) in a single test, using the storage pool type as the storage tag.
A disk-only VM snapshot does not contain the guest page cache, so the file written by the test may be missing after the revert.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
len(backups) == 1 is brittle if prior tests (or a previously interrupted run) left… · Newcluster_scoped_found is set but never asserted, so this test can pass even if the filter returns… · Newpod_scoped_found is never asserted, so the test may pass… · Newlist_storage_pools(cls.apiclient) always returns a non-empty list. If it returns… · Newlist_storage_pools()[0], which can be the wrong pool in…volume_pool.type.lower(), suggesting…volume_pool.type.lower(), suggesting…scope attribute are silently ignored and won’t be validated by either branch,…pod_scoped_found is never asserted, so the test may pass…cluster_scoped_found is set but never asserted, so the test can pass vacuously even if the filter…assertEqual(len(backups), 1) will fail if the environment already has…assertEqual(len(backups), 1) will fail if the environment already has…BackupOffering.listExternal(...) returns None or an empty list, indexing [0] will raise.…And 2 more that still need to be addressed.
🧠 Review effort: Lite
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| Backup.create(self.apiclient, self.vm.id, "restore_attach_backup") | ||
|
|
||
| backups = Backup.list(self.apiclient, self.vm.id) | ||
| self.assertEqual(len(backups), 1, "There should exist only one backup for the VM") | ||
| backup = backups[0] |
| # Filter may return zone-wide pools (with clusterid=None) alongside cluster-scoped pools | ||
| cluster_scoped_found = False | ||
| for storage_pool in storage_pools: | ||
| self.assertEqual( | ||
| storage_pool.clusterid, | ||
| self.cluster.id, | ||
| "Cluster id should be equal to the cluster id passed in the filter" | ||
| ) | ||
| if storage_pool.clusterid is not None: | ||
| # Only assert clusterid for cluster-scoped pools | ||
| cluster_scoped_found = True | ||
| self.assertEqual( | ||
| storage_pool.clusterid, | ||
| self.cluster.id, | ||
| "Cluster id should be equal to the cluster id passed in the filter" | ||
| ) |
| storage_pool.podid, | ||
| self.cluster.podid, | ||
| "Pod id should be equal to the pod id passed in the filter" | ||
| ) |
| list_volume_pool_response = list_storage_pools(cls.apiclient) | ||
| volume_pool = list_volume_pool_response[0] | ||
| if volume_pool.type == "RBD": | ||
| cls.skipTest(cls, reason="VM snapshot is unsupported for VMs on RBD storage pool") |
| id=vol_res[0].storageid) | ||
| volume_pool = volume_pool_response[0] | ||
| if volume_pool.type.lower() != 'networkfilesystem': | ||
| if volume_pool.type not in ("NetworkFilesystem", "RBD"): |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
BackupOffering.listExternal(...) can return None/empty in some environments or transiently;… · Newpool_type.lower() as the temporary storage tag is very generic (e.g., rbd,… · Newcluster_scoped_found is set but never asserted/used, so the test can pass even when the cluster… · Newpod_scoped_found is never asserted/used, so the test can… · Newtarget_pools[0]. If the first candidate happens to be an RBD… · Newtype.lower() == ...) and case-sensitive… · NewElementTree.findall(...) always returns a list (possibly empty), so… · Newlist_storage_pools(cls.apiclient) always returns a non-empty list. If it returns…pod_scoped_found is never asserted, so the test may pass…cluster_scoped_found is set but never asserted, so this test can pass even if the filter returns…len(backups) == 1 is brittle if prior tests (or a previously interrupted run) left…list_storage_pools()[0], which can be the wrong pool in…volume_pool.type.lower(), suggesting…volume_pool.type.lower(), suggesting…scope attribute are silently ignored and won’t be validated by either branch,…pod_scoped_found is never asserted, so the test may pass…cluster_scoped_found is set but never asserted, so the test can pass vacuously even if the filter…assertEqual(len(backups), 1) will fail if the environment already has…And 10 more that still need to be addressed.
🧠 Review effort: Lite
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Hi @weizhouapache can you please check if the Copilot comments are relevant? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Assert that the encryption format and passphrase secret lists of TestVolumeEncryption are not empty, instead of asserting they are not None, which findall() never returns. Compare the storage pool type case-insensitively, give the direct download test a unique storage tag, skip the live VM migration test for every RBD source pool, and drop an unused variable in test_list_storage_pools.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
addressed |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@blueorangutan package |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19799 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…e test PRs Add the commits for apache#14386, apache#14320 and apache#11931 and update the notes.
|
@blueorangutan test |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
[SF] Trillian test result (tid-17426)
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.