Description

rafaelcs

opened on Sep 11, 2019

Issue body actions

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

I'm running a test case on two different devices, but when the allure report is generated, the test case is shown only once.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Command I'm using to run the tests

1. pytest --dist=each -n 2 --alluredir=allure-results tests/login/test_request_code_without_phone_number.py
2. allure generate -c allure-results -o allure-results/allure-results-html
3. allure serve allure-results/allure-results-html or allure open allure-results/allure-results-html

Report generated:

What is the expected behavior?

Since I am running a test case on two different devices, the report should show repeated scenarios, like this pytest-html report:

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

• Allure version: 2.9.0
• allure-pytest: 2.8.0
• allure-python-commons: 2.8.0
• Appium-Python-Client: 0.47
• pytest: 5.1.2
• pytest-forked: 1.0.2
• pytest-xdist: 1.29.0
• selenium: 3.141.0

Other information

Another question related to this: As I am running a test case on two different devices, how can I display in the report the version of the device that the scenario ran?

Reacted by siarheimalaichuk

Activity

1. rafaelcs commented on Sep 19, 2019

   

   rafaelcs

   on Sep 19, 2019

   Author

   More actions

   Any update here?
2. rafaelcs commented on Oct 10, 2019

   

   rafaelcs

   on Oct 10, 2019

   Author

   More actions

   Update?
3. landhu commented on Dec 6, 2019

   

   landhu

   on Dec 6, 2019

   More actions

   Any update here?NO fix??
4. misanthropicat commented on Nov 2, 2020

   

   misanthropicat

   on Nov 2, 2020

   More actions

   What's current status of this issue? Is it still no solution to avoid this?
5. delatrie

   added

   theme:pytest

   bugSomething isn't workingSomething isn't working

   on Dec 5, 2022
6. delatrie commented on Dec 20, 2022

   

   delatrie

   on Dec 20, 2022

   Contributor

   More actions

   This happens because historyId of a test result is calculated based on the nodeid of a test, i.e doesn`t depend on environment in which the test was run.

   A historyId value is used to track test retries and history. If multiple tests share the same historyId, allure considers the earliest one as a basic test result and the other ones as its retries. You can see the rest of your tests in the Retries tab:

   
   (Here I`ve run the tests with -n 4)

   Allure TestOps (and future allure3) ignores historyId from a test result and calculates it itself based on testCaseId and test parameter values. So, to resolve this issue we should:

   1. Detect xdist and add worker id as a hidden dynamic parameter.
   2. Also add a python version, platform, hostname if --dist=each as excluded dynamic parameters to show them in the report (excluded means that the parameter is not used when calculating historyId).
   3. Make historyId dependent on non-excluded dynamic parameter values to also fix allure2 report.
7. delatrie

   added

   contributeCall for a contributionCall for a contribution

   on Dec 20, 2022
8. delatrie

   added

   priority:critical

   on Jun 20, 2023
9. delatrie

   removed

   priority:critical

   on Jul 8, 2023
10. HillsAbide commented on Aug 19, 2024

    

    HillsAbide

    on Aug 19, 2024

    More actions

    Oh my, this bug hasn't been completely fixed after 5 years, but I'm glad to see some progress.
    When will it be fully resolved? I'm still troubled by this issue while using python-xdist.
    @delatrie And if you have any workaround methods, please let me know.
11. HillsAbide commented on Aug 19, 2024

    

    HillsAbide

    on Aug 19, 2024

    More actions

    Update, I found a workaround to solve this issue.
    According to PR with ID 745, we can call allure.dynamic.parameter('workid', workid) in Setup\Fixture\Hook, which will allow the Allure report to differentiate parameters and display the results of multiple cases. I have already tested it successfully.

Issue actions