GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Some CI tests are flaky or real failures · Issue #2824 · lcompilers/lpython

Some CI tests are flaky or real failures #2824

New issue
Copy link
New issue
Copy link
Open
Open
Some CI tests are flaky or real failures#2824
Copy link

Description

@ubaidsk
ubaidsk
opened on Feb 17, 2025
Issue body actions

There are some CI tests that seem to be flaky and fail.

  1. Failure in ctest.
Test project /home/runner/work/lpython/lpython
    Start 1: test_stacktrace
1/2 Test #1: test_stacktrace ..................   Passed    0.00 sec
    Start 2: test_lpython
2/2 Test #2: test_lpython .....................Subprocess aborted***Exception:   0.86 sec
[doctest] doctest version is "2.4.8"
[doctest] run with "--help" for options
0 0 0 0 0 0 0 0 0 0 �
test_lpython: /home/runner/micromamba/envs/lp/include/llvm/IR/DataLayout.h:656: uint64_t llvm::StructLayout::getElementOffset(unsigned int) const: Assertion `Idx < NumElements && "Invalid element idx!"' failed.
===============================================================================
/home/runner/work/lpython/lpython/src/lpython/tests/test_llvm.cpp:[15](https://github.com/lcompilers/lpython/actions/runs/13378363981/job/37362241953?pr=2823#step:6:16)56:
TEST CASE:  PythonCompiler classes

/home/runner/work/lpython/lpython/src/lpython/tests/test_llvm.cpp:1556: FATAL ERROR: test case CRASHED: SIGABRT - Abort (abnormal termination) signal

===============================================================================
[doctest] test cases:  55 |  54 passed | 1 failed | [17](https://github.com/lcompilers/lpython/actions/runs/13378363981/job/37362241953?pr=2823#step:6:18) skipped
[doctest] assertions: 486 | 486 passed | 0 failed |
[doctest] Status: FAILURE!


50% tests passed, 1 tests failed out of 2

Total Test time (real) =   0.87 sec

The following tests FAILED:
	  2 - test_lpython (Subprocess aborted)
Errors while running CTest
Error: Process completed with exit code 8.
  1. Failure in reference tests
compiler_tester.tester.RunException: Testing with reference output failed.
runtime_errors/test_assert_01.py * run_dbg
The JSON metadata differs against reference results
Reference JSON: tests/reference/run_dbg-test_assert_01-2f34744.json
Output JSON:    tests/output/run_dbg-test_assert_01-2f34744.json
Omitting 9 identical items
Differing items:
{'stderr_hash': '32b0a24f111e577fe4fc5b3f4a5994b951e34dde7986b3fb750c5f5e'} != {'stderr_hash': '4811af471c73572b285e9ea01c8689abdd3cb32c717b3cd4876d2669'}
{'returncode': 134} != {'returncode': 1}
Diff against: tests/reference/run_dbg-test_assert_01-2f34744.stderr
1,7c1,2
<   File "tests/runtime_errors/test_assert_01.py", line 1
<     def test():
<   File "tests/runtime_errors/test_assert_01.py", line 4
<     test()
<   File "tests/runtime_errors/test_assert_01.py", line 2
<     assert False
< AssertionError
---
> *** buffer overflow detected ***: terminated
> Aborted (core dumped)

Error: Process completed with exit code 1.

Activity

  1. ubaidsk commented on Feb 17, 2025

    @ubaidsk
    ubaidsk
    on Feb 17, 2025
    CollaboratorAuthor
    More actions

    The above tests kind of fail randomly. I will probably comment out these two tests for now and get the CI to pass (which will unblock merging PRs). We can then fix these tests iteratively in subsequent PRs.

  2. ubaidsk commented on Feb 17, 2025

    @ubaidsk
    ubaidsk
    on Feb 17, 2025
    CollaboratorAuthor
    More actions

    Failure in reference tests

    Seems like all the run_with_dbg reference tests are failing/flaky as of now. Following the steps at the CI, they work fine for me locally.

  3. ubaidsk commented on Feb 17, 2025

    @ubaidsk
    ubaidsk
    on Feb 17, 2025
    CollaboratorAuthor
    More actions

    Also integration_tests/test_str_01.py fails with the above (> *** buffer overflow detected ***: terminated).

    I think these are real failures that we need to fix.

  4. ubaidsk
    changed the title [-]Some CI tests are flaky[/-] [+]Some CI tests are flaky or real failures[/+] on Feb 17, 2025
  5. kmr-srbh commented on Feb 18, 2025

    @kmr-srbh
    kmr-srbh
    on Feb 18, 2025
    Contributor
    More actions

    I think a fix for this buffer overflow error might be the same as lfortran/lfortran#6003.

    The CI failures started occurring when the GCC compiler was updated on the CI. The PRs which fixed them incrementally were:

    • ci: resolve failures lfortran/lfortran#5983
    • fix: allocate space for null terminator in string allocations lfortran/lfortran#6003
    • fix: initialize variables to avoid undefined behavior lfortran/lfortran#6004
  6. swamishiju commented on Feb 20, 2025

    @swamishiju
    swamishiju
    on Feb 20, 2025
    Collaborator
    More actions

    lpython/src/libasr/runtime/lfortran_intrinsics.c

    Lines 2306 to 2308 in 7eb2bea

    char* dest_char = (char*)malloc(s_len);
    strcpy(dest_char, s);

    It does look like the same issue for test_str_01

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      You are visitor 01402503 — GeezerVison does not store the pages you look up.
      Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.