GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Exponential Growth in Allure Results with Each Iteration in runner.py Script · Issue #858 · allure-framework/allure-python

Exponential Growth in Allure Results with Each Iteration in runner.py Script #858

New issue
Copy link
New issue
Copy link
Closed
Bug
#860
Closed
Exponential Growth in Allure Results with Each Iteration in runner.py Script#858
Bug
#860
Copy link
Assignees
delatrie
Labels
theme:behave

Description

@MadhurSharma033
MadhurSharma033
opened on Apr 29, 2025
Issue body actions

Environment:

Script: runner.py
Dependencies: Python (3.12.6),
allure-behave==2.14.0,
allure-python-commons==2.13.3 ,
behave==1.2.6
Platform: MAC OS

Summary: While executing a runner.py script using a for loop to iterate over a list of devices, I observe unexpected behavior in the generated Allure results. Specifically, for each iteration through device_list, the count of test cases reported by Allure results grows exponentially, despite running a single test for each device.

Description:
Code snippet:

for device in devices:
device_serial = (device.get("serial") if isinstance(device, dict) else device)
allure_device_results_dir, report_device_path = create_device_folders(str(PLATFORM.value), str(device_serial))

if os.path.exists("allure-results"):
    shutil.rmtree("allure-results")
os.makedirs("allure-results", exist_ok=True)

runner.main(
    f"./features/ -f allure_behave.formatter:AllureFormatter "
    f"-o allure-results --no-capture --no-capture-stderr --no-skipped "
    f"--tags={TAG} --tags=~@ignore "
    f"--define board={BOARD.value} "
    f"--define product={PRODUCT} --define solution={SOLUTION} "
    f"--define device={device_serial} "
)

Observed Behaviour:
During first iteration (device 1), one result.json is produced, showing 1 test case count in Allure serve.
During second iteration (device 2), two result.json files are produced, showing 2 test cases count in Allure serve, although only one test occurs.
This pattern repeats for subsequent devices, with counts increasing exponentially.

Expected Behaviour:
For each device in the list, only one result.json should be created corresponding to the single test executed, reflecting accurate test case counts.

Steps to Reproduce:
Setup the script with a list of devices.
Execute runner.py to iterate over each device.
Observe the contents of allure-results directory and run allure serve.

Impact:
Incorrect test case count presentation.
Potential confusion during test result analysis.

Below is the screenshot of first iteration:

Image

and below is the screenshot of 4th iteration:

Image

Activity

  1. delatrie
    added the
    Bug
    issue type on Apr 29, 2025
  2. delatrie
    added
    theme:behave
    on Apr 29, 2025
  3. delatrie commented on Apr 30, 2025

    @delatrie
    delatrie
    on Apr 30, 2025
    Contributor
    More actions

    Hi, @MadhurSharma033

    It looks like the Allure formatter doesn't perform the cleanup properly.

    Could you please tell me in more detail what the runner object is in your script and how you got it? It will help write a test to reproduce the issue.

  4. MadhurSharma033 commented on May 2, 2025

    @MadhurSharma033
    MadhurSharma033
    on May 2, 2025
    Author
    More actions

    Hi @delatrie runner is nothing but imported "from behave import main as runner"

  5. delatrie
    self-assigned this
    on May 5, 2025
  6. delatrie commented on May 5, 2025

    @delatrie
    delatrie
    on May 5, 2025
    Contributor
    More actions

    @MadhurSharma033

    Given behave==1.2.6 the import you've provided gives the error:

    ImportError: cannot import name 'main' from 'behave'
    

    Could you please double check the Behave version and the import line? Or, better, create a public repository with the code to reproduce the behavior and share the link?

    Update: fixed in PR #860. It will be merged and released soon.

  7. delatrie
    added a commit that references this issue on May 5, 2025

    test(behave): add test to reproduce #858

    e293b8a
  8. delatrie
    closed this as completedin #860on May 6, 2025
  9. delatrie commented on May 6, 2025

    @delatrie
    delatrie
    on May 6, 2025
    Contributor
    More actions

    Released in 2.14.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

  • @delatrie
    delatrie

Labels

theme:behave

Type

Bug

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402503 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.