GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — feat(cli): allow options from CLI args or environment variables by nejch · Pull Request #1296 · python-gitlab/python-gitlab

Conversation

@nejch

@nejch nejch commented Feb 14, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Closes #960
Closes #893

Makes it easier to later introduce #715 hopefully, and maybe #1195.

BREAKING-CHANGE: The gitlab CLI will now accept CLI arguments
and environment variables for its global options in addition
to configuration file options. This may change behavior for
some workflows such as running inside GitLab CI and with
certain environment variables configured.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@codecov-io

codecov-io commented Feb 14, 2021

Copy link
Copy Markdown

Codecov Report

Merging #1296 (cab343c) into master (9d6c188) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1296   +/-   ##
=======================================
  Coverage   78.26%   78.26%           
=======================================
  Files          12       12           
  Lines        2894     2894           
=======================================
  Hits         2265     2265           
  Misses        629      629           
Flag Coverage Δ
unit 78.26% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d6c188...cab343c. Read the comment docs.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nejch nejch mentioned this pull request Apr 25, 2021
chore: bump version to 2.7.0 #1408
Merged
JohnVillalovos
JohnVillalovos reviewed Apr 25, 2021
View reviewed changes
Comment thread gitlab/__init__.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@nejch nejch added this to the v3.0.0 milestone Apr 27, 2021
@nejch nejch mentioned this pull request Jun 14, 2021
Breaking changes in 3.0.0 #1520
Closed
15 tasks done
phillid
phillid reviewed Jul 4, 2021
View reviewed changes
Comment thread gitlab/cli.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@phillid

phillid commented Jul 4, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

We found this PR after we tried including gitlab-cli in a GitLab CI/CD pipeline job. This PR looks like it will achieve what we wanted to do (drive the GitLab API without creating a config file in each job) 🎉

One outstanding wishlist item we had was if the tool could look for the same environment variables that are set inside a GitLab CI/CD job, i.e. CI_JOB_TOKEN, CI_SERVER_URL and so on 1. If the tool shared these varaibles, it could authenticate with the GitLab instance without any extra code written in CI/CD jobs - is that something that has already been considered?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nejch

nejch commented Jul 4, 2021

Copy link
Copy Markdown
Member Author

We found this PR after we tried including gitlab-cli in a GitLab CI/CD pipeline job. This PR looks like it will achieve what we wanted to do (drive the GitLab API without creating a config file in each job) 🎉

One outstanding wishlist item we had was if the tool could look for the same environment variables that are set inside a GitLab CI/CD job, i.e. CI_JOB_TOKEN, CI_SERVER_URL and so on 1. If the tool shared these varaibles, it could authenticate with the GitLab instance without any extra code written in CI/CD jobs - is that something that has already been considered?

Yes, part of this was meant to implement zero-config for CI jobs. But I need to maybe tweak and add tests to really check all scenarios (for some endpoints, a PAT is needed, so we should allow overriding in a sensible precedence between cli args/env/config files/CI env). Currently the precedence planned is:

python-gitlab/gitlab/__init__.py

Lines 224 to 232 in cab343c

1. Explicitly provided CLI arguments,
2. Environment variables,
3. Configuration files:
a. explicitly defined config files:
i. via the `--config-file` CLI argument,
ii. via the `PYTHON_GITLAB_CFG` environment variable,
b. user-specific config file,
c. system-level config file,
4. Environment variables always present in CI (CI_SERVER_URL, CI_JOB_TOKEN).

I'll try to get back to this PR in the coming days :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@phillid

phillid commented Jul 6, 2021

Copy link
Copy Markdown

Yes, part of this was meant to implement zero-config for CI jobs. But I need to maybe tweak and add tests to really check all scenarios (for some endpoints, a PAT is needed, so we should allow overriding in a sensible precedence between cli args/env/config files/CI env). Currently the precedence planned is:

python-gitlab/gitlab/__init__.py

Lines 224 to 232 in cab343c

1. Explicitly provided CLI arguments,
2. Environment variables,
3. Configuration files:
a. explicitly defined config files:
i. via the `--config-file` CLI argument,
ii. via the `PYTHON_GITLAB_CFG` environment variable,
b. user-specific config file,
c. system-level config file,
4. Environment variables always present in CI (CI_SERVER_URL, CI_JOB_TOKEN).

Aha. I'm blind 👀

That does indeed look like exactly the behaviour we wanted.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nejch nejch mentioned this pull request Sep 11, 2021
Improve helper example section with env var usage #1580
Closed
@nejch
nejch force-pushed the feat/merge-cli-env-file-config branch from cab343c to 13467c5 Compare September 12, 2021 19:40
@codecov-commenter

codecov-commenter commented Sep 12, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Codecov Report

Merging #1296 (13467c5) into master (ce4bc0d) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1296      +/-   ##
==========================================
+ Coverage   91.57%   91.61%   +0.03%     
==========================================
  Files          74       74              
  Lines        4263     4281      +18     
==========================================
+ Hits         3904     3922      +18     
  Misses        359      359              
Flag Coverage Δ
cli_func_v4 81.63% <100.00%> (+0.07%) ⬆️
py_func_v4 80.35% <26.31%> (-0.23%) ⬇️
unit 83.36% <84.21%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
gitlab/cli.py 87.50% <100.00%> (+1.29%) ⬆️
gitlab/client.py 89.04% <100.00%> (+0.18%) ⬆️
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nejch
nejch force-pushed the feat/merge-cli-env-file-config branch from 13467c5 to a998970 Compare December 7, 2021 23:28
@JohnVillalovos

JohnVillalovos commented Dec 9, 2021

Copy link
Copy Markdown
Member

What do you think about using argparse.ArgumentParser..add_mutually_exclusive_group() ?

For example:

private_token_group = parser.add_mutually_exclusive_group()
private_token_group.add_argument(
        "--private-token",
        help=("GitLab private token"),
        required=False,
    )
private_token_group.add_argument(
        "--private-token-env-var",
        help=("Name of environment variable containing GitLab private token"),
        required=False,
    )

But now that I look at the code I'm not sure where to do the processing to load the environment variable 😟

I'll just leave this comment anyway and maybe it gives you an idea 🤷‍♂️

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nejch

nejch commented Dec 10, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

.add_mutually_exclusive_group()

Thanks! Yes, I think it might be useful (not for arg/var exclusive I think but maybe for different types of tokens). But yes sometimes we want to validate those a bit later, because things like CI_JOB_TOKEN might always be present 🤦

I think we just need to write a bunch of tests for expected behaviors in all the possible combinations, and write the code based on that.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nejch nejch mentioned this pull request Dec 12, 2021
feat(cli): do not require config file to run CLI #1743
Merged
@nejch
nejch force-pushed the feat/merge-cli-env-file-config branch 2 times, most recently from a6390bf to c60ae1f Compare December 14, 2021 00:45
@nejch
nejch force-pushed the feat/merge-cli-env-file-config branch 5 times, most recently from 14487b3 to c6de2f5 Compare January 2, 2022 17:09
nejch
nejch commented Jan 2, 2022
View reviewed changes
Comment thread gitlab/cli.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread gitlab/client.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread gitlab/cli.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@nejch
nejch marked this pull request as ready for review January 2, 2022 17:19
@nejch
nejch requested a review from JohnVillalovos January 2, 2022 17:20
@nejch

nejch commented Jan 2, 2022

Copy link
Copy Markdown
Member Author

@JohnVillalovos took almost a year to get back to this, but ready for another look 😅

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nejch
nejch force-pushed the feat/merge-cli-env-file-config branch from c6de2f5 to fe964dd Compare January 2, 2022 17:55
JohnVillalovos
JohnVillalovos approved these changes Jan 2, 2022
View reviewed changes

@JohnVillalovos JohnVillalovos left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits. But overall looks good to me. Thanks!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread docs/cli-usage.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread gitlab/cli.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread gitlab/client.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread gitlab/client.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread tests/unit/test_gitlab_auth.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread gitlab/client.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread tests/unit/test_gitlab_auth.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@nejch
feat(cli): allow options from args and environment variables
1158be3
BREAKING-CHANGE: The gitlab CLI will now accept CLI arguments
and environment variables for its global options in addition
to configuration file options. This may change behavior for
some workflows such as running inside GitLab CI and with
certain environment variables configured.
@nejch
nejch force-pushed the feat/merge-cli-env-file-config branch from fe964dd to 1158be3 Compare January 2, 2022 21:11
@nejch nejch mentioned this pull request Jan 2, 2022
Revamp docs for CLI and env-based options #1795
Closed
@nejch
nejch requested a review from JohnVillalovos January 2, 2022 21:21
JohnVillalovos
JohnVillalovos approved these changes Jan 2, 2022
View reviewed changes

@JohnVillalovos JohnVillalovos left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nejch . Looks good.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@JohnVillalovos
JohnVillalovos merged commit ca58008 into main Jan 2, 2022
@JohnVillalovos
JohnVillalovos deleted the feat/merge-cli-env-file-config branch January 2, 2022 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JohnVillalovos JohnVillalovos JohnVillalovos approved these changes
+1 more reviewer
@phillid phillid phillid left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

v3.0.0

Development

Successfully merging this pull request may close these issues.

cli command line arguments for url and token CLI usage without configuration file

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@nejch @codecov-io @phillid @codecov-commenter @JohnVillalovos
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.