GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||
GeezerVison is showing github.com — [pipeline-manager] Check memory and storage autoscaling limits in the api-server by xiasongh · Pull Request #7819 · feldera/feldera Conversation
|
| Runtime configuration | Refused when | New api-server flag | Helm value |
|---|---|---|---|
resources.memory_min, memory_max |
above the maximum | --upper-limit-memory-megabyte |
kubernetesRunner.upperLimitMemoryMegabyte |
both storage_min and storage_max |
storage autoscaling disabled | --enable-storage-autoscaling |
kubernetesRunner.enableStorageAutoscaling |
With #7712 (storage size and class allowlists) and #7656 (the resize PATCH checks them), every field that can change while a pipeline runs is checked against the runner's limits. CPU has no runner limit.
memory_min above memory_max is refused, as storage_min above storage_max already is. A memory refusal reads "unsupported memory: ...".Settings that cannot change while a pipeline runs, such as resources.namespace, service_account_name, or env, are left to the runner, which refuses them at start with a message that names the Helm value.
The Helm side is feldera/cloud#2165, which must merge after this.
Testing: unit tests for each refusal and for parsing the flags as the chart passes them, and endpoint tests for POST, PUT, PATCH of runtime_config (including the deprecated memory_mb_max and an installation without limits), and PATCH of deployment_config. Each check was removed in turn, and a test failed. Platform tests were not run.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Review summary
(true, "POST", "/pipelines", with_env("e"), 400),
+ (true, "PUT", "/pipelines/p", pipeline("p", json!({"memory_max": "1000M"})), 200), // boundary
+ (true, "PATCH", "/pipelines/p", patch(json!({"memory_max": "1001M"})), 400),
+ (true, "PATCH", "/pipelines/p", patch(json!({"memory_mb_max": 2000})), 400), // legacy field
+ (false, "PATCH", "/pipelines/p", patch(json!({"memory_min": "2G", "memory_max": "1G"})), 400),
+ (true, "PATCH", "/pipelines/p", json!({"runtime_config": {"env": {"A": "1"}}}), 400),
+ (true, "PUT", "/pipelines/p", with_env("p"), 400),
+ (true, "PATCH", "/pipelines/p", json!({"runtime_config": {"env": {}}}), 200), // empty env allowedPlatform tests were not run here (no k8s). Otherwise the logic, the actionable error messages, and the docs look good. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
dbc9d3c to
e9013fb
Compare
e9013fb to
607d9d1
Compare
|
LGTM. What I ran (debug, merge commit):
Gates: tests cover the changed behaviour, nothing flaky (no sleeps, and each case builds its own state), docs and changelog are updated, the Missing tests (low, optional): at the endpoint level, the storage-autoscaling refusal is only tested for |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
607d9d1 to
17c8b4f
Compare
17c8b4f to
1b2fb3f
Compare
… api-server POST, PUT, and PATCH return 400 InvalidRuntimeConfig for a runtime_config with memory above the Kubernetes runner's maximum, or with both storage_min and storage_max when the runner does not enable storage autoscaling. A PATCH of deployment_config refuses memory above the maximum. Before, the next start failed, and a refused resize only went to the pipeline log. Both flags are unset by default and then not checked. On all installations, memory_min above memory_max is now refused.
1b2fb3f to
3bc6736
Compare
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.