GeezerVison is showing github.com — Cannot import a dumpdir containing constrained indices · Issue #1064 · devpi/devpi
Cannot import a dumpdir containing constrained indices #1064DescriptionHello,
I created a dumpdir using devpi-export. My devpi server currently runs with constrained index. The devpi-export command fails with:
Traceback (most recent call last):
File "/usr/local/bin/devpi-import", line 8, in <module>
sys.exit(import_())
File "/usr/local/lib/python3.10/site-packages/devpi_server/importexport.py", line 154, in import_
do_import(config.args.directory, xom)
File "/usr/local/lib/python3.10/site-packages/devpi_server/importexport.py", line 111, in do_import
importer.import_all(path)
File "/usr/local/lib/python3.10/site-packages/devpi_server/importexport.py", line 482, in import_all
stage = user.create_stage(index, type=indexconfig['type'])
File "/usr/local/lib/python3.10/site-packages/devpi_server/model.py", line 430, in create_stage
return self.parent.create_stage(self, index, type=type, **kwargs)
File "/usr/local/lib/python3.10/site-packages/devpi_server/keyfs.py", line 503, in create_stage
self.model_cache[key] = super().create_stage(
File "/usr/local/lib/python3.10/site-packages/devpi_server/model.py", line 173, in create_stage
stage._modify(**kwargs)
File "/usr/local/lib/python3.10/site-packages/devpi_server/model.py", line 1044, in _modify
self.customizer.validate_config(oldconfig, newconfig)
File "/usr/local/lib/python3.10/site-packages/devpi_constrained/main.py", line 62, in validate_config
raise self.InvalidIndexconfig(errors)
devpi_server.model.InvalidIndexconfig: ['A constrained index requires at least one base']
Extract of the dataindex.json for a test/dev index failing to be created:
"test/dev": {
"indexconfig": {
"acl_upload": [
"test"
],
"mirror_whitelist": [
"xgboost"
],
"volatile": true,
"mirror_whitelist_inheritance": "union",
"description": "test",
"acl_toxresult_upload": [":ANONYMOUS:"],
"title": "test",
"bases": [
"test/upload",
"root/pypi-wheels"
],
"constraints": [
"virtualenv<v20.24.6",
"pytest<=8.1.2",
"twine!=5.1.0"
],
"type": "constrained"
},
"projects": {},
"files": []
}
If I am not mistaken, the issue lies in the user.create_stage being called before bases are set on the newly created indices:
|
bases = indexconfig.pop('bases', None) |
|
stage = None |
|
if stagename == "root/pypi": |
|
# see if the stage was already created |
|
stage = self.xom.model.getstage(stagename) |
|
if self.xom.config.no_root_pypi: |
|
continue |
|
if stage is None: |
|
# we create a stage without any config |
|
stage = user.create_stage(index, type=indexconfig['type']) |
|
if stage is not None: |
|
# we use modify with _keep_unknown, so data from |
|
# removed plugins is preserved |
|
stage.modify(**indexconfig, _keep_unknown=True) |
|
if "bases" in import_index["indexconfig"]: |
|
# we are changing bases directly to allow import with |
|
# removed bases without changing the data from the export |
|
with stage.user.key.update() as userconfig: |
|
indexconfig = userconfig['indexes'][stage.index] |
|
indexconfig["bases"] = tuple(bases) |
|
stages.append(stage) |
Thanks for this great product ❤️ Activityfschulze commented on Oct 24, 2024 I think it is better to change the validation in devpi-constrained. A constrained index doesn't really require a base, it just doesn't do anything useful without one. Changing the import semantics would be way harder because of the ways bases work. amoutaux commented on Oct 24, 2024 amoutaux commented on Oct 28, 2024
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|