GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-120501: Fix reference leak in JIT build by Eclips4 · Pull Request #120649 · python/cpython

Conversation

@Eclips4

@Eclips4 Eclips4 commented Jun 17, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: A lot of leaks in the test suite on the JIT build #120501

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Eclips4 added 2 commits June 17, 2024 14:59
@Eclips4
First try to fix..
871dc4c
@Eclips4
Forgot to commit codeobject.c
08c26d3
@Eclips4
Eclips4 requested a review from markshannon as a code owner June 17, 2024 15:39
@bedevere-app bedevere-app Bot mentioned this pull request Jun 17, 2024
A lot of leaks in the test suite on the JIT build #120501
Open
@bedevere-app bedevere-app Bot added the awaiting review label Jun 17, 2024
@Eclips4
Eclips4 marked this pull request as draft June 17, 2024 15:39
@bedevere-app bedevere-app Bot removed the awaiting review label Jun 17, 2024
@Eclips4 Eclips4 added the skip news label Jun 17, 2024
@Eclips4

Eclips4 commented Jun 17, 2024

Copy link
Copy Markdown
Member Author

@JeffersGlass can you apply this patch locally and verify that this solves the refleak problem?
FYI there's a another one problem(😢) that was introduced after f6fab21, but firstly in this PR I would like to solve initial problem 🙂

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4

Eclips4 commented Jun 17, 2024

Copy link
Copy Markdown
Member Author

The second problem is bisected to 72867c9

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JeffersGlass

JeffersGlass commented Jun 17, 2024

Copy link
Copy Markdown
Contributor

On Linux/X86_64, this seems to have fixed the refleak issue! 🎉

Building on Windows_x64, though, I get an error when starting the interpreter. Assertion failed: Py_SIZE(value), file D:\cpython\Python\executor_cases.c.h, line 405. Same error whether dropping into the REPL, or running with -c, or -m test, etc.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner
Fidget-Spinner requested a review from brandtbucher June 18, 2024 09:07
@vstinner vstinner changed the title gh-120501: Fix reference leak gh-120501: Fix reference leak in JIT build Jun 18, 2024
vstinner
vstinner reviewed Jun 18, 2024
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirm that the change fix the reference leak for test_colorsys (I picked on test to check the PR).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/optimizer.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Eclips4 added 3 commits June 23, 2024 10:06
@Eclips4
Merge branch 'main' into issue-120501
e9301f3
@Eclips4
Add a newline
08cb747
@Eclips4
Add a newline (2)
9926e1c
@Eclips4

Eclips4 commented Jun 24, 2024

Copy link
Copy Markdown
Member Author

So, here's the second problem:
Reproducer:

def foo():
    a = [1, 2, 3]
    exhit = iter(a)
    for _ in exhit:
        pass
    a.append("this should'be in exhit")
    print(f"got {list(exhit)}, should be []")

foo()
foo()
foo()
foo()
foo()
foo()

Output:

got [], should be []
got [], should be []
got [], should be []
got [], should be []
got [], should be []
got ["this should'be in exhit"], should be []

Obviously, the last line is incorrect.

Output with a PYTHON_LLTRACE=2 env:

got [], should be []
got [], should be []
got [], should be []
got [], should be []
got [], should be []
Optimizing foo (/home/eclips4/programming-languages/cpython/example.py:1) at byte offset 42
   1 ADD_TO_TRACE: _START_EXECUTOR (0, target=21, operand=0x7f4646e59832)
21: JUMP_BACKWARD(5)
   2 ADD_TO_TRACE: _CHECK_VALIDITY_AND_SET_IP (0, target=21, operand=0x7f4646e59832)
   3 ADD_TO_TRACE: _TIER2_RESUME_CHECK (0, target=21, operand=0)
18: FOR_ITER_LIST(3)
   4 ADD_TO_TRACE: _CHECK_VALIDITY_AND_SET_IP (0, target=18, operand=0x7f4646e5982c)
   5 ADD_TO_TRACE: _ITER_CHECK_LIST (3, target=18, operand=0)
   6 ADD_TO_TRACE: _GUARD_NOT_EXHAUSTED_LIST (3, target=18, operand=0)
   7 ADD_TO_TRACE: _ITER_NEXT_LIST (3, target=18, operand=0)
20: STORE_FAST(2)
   8 ADD_TO_TRACE: _CHECK_VALIDITY_AND_SET_IP (0, target=20, operand=0x7f4646e59830)
   9 ADD_TO_TRACE: _STORE_FAST (2, target=20, operand=0)
21: JUMP_BACKWARD(5)
  10 ADD_TO_TRACE: _CHECK_VALIDITY_AND_SET_IP (0, target=21, operand=0x7f4646e59832)
  11 ADD_TO_TRACE: _JUMP_TO_TOP (0, target=0, operand=0)
Created a proto-trace for foo (/home/eclips4/programming-languages/cpython/example.py:1) at byte offset 36 -- length 11
Optimized trace (length 10):
   0 OPTIMIZED: _START_EXECUTOR (0, jump_target=7, operand=0x7f4646e59e80)
   1 OPTIMIZED: _TIER2_RESUME_CHECK (0, jump_target=7, operand=0)
   2 OPTIMIZED: _ITER_CHECK_LIST (3, jump_target=8, operand=0)
   3 OPTIMIZED: _GUARD_NOT_EXHAUSTED_LIST (3, jump_target=9, operand=0)
   4 OPTIMIZED: _ITER_NEXT_LIST (3, target=18, operand=0)
   5 OPTIMIZED: _STORE_FAST_2 (2, target=20, operand=0)
   6 OPTIMIZED: _JUMP_TO_TOP (0, target=0, operand=0)
   7 OPTIMIZED: _DEOPT (0, target=21, operand=0)
   8 OPTIMIZED: _EXIT_TRACE (0, exit_index=0, operand=0)
   9 OPTIMIZED: _EXIT_TRACE (0, exit_index=1, operand=0)
got ["this should'be in exhit"], should be []

It's definitely related to this part of code:

cpython/Python/optimizer.c

Lines 1027 to 1033 in e4a97a7

if (is_for_iter_test[opcode]) {
/* Target the POP_TOP immediately after the END_FOR,
* leaving only the iterator on the stack. */
int extended_arg = inst->oparg > 255;
int32_t next_inst = target + 1 + INLINE_CACHE_ENTRIES_FOR_ITER + extended_arg;
jump_target = next_inst + inst->oparg + 1;
}

I guess the culprit is there. If we remove the _GUARD_NOT_EXHAUSTED_LIST from is_for_iter_test, the problem will go away (although it can still be reproduced in other ways using other (range, tuple) iterators).

JeffersGlass reacted with eyes emoji
All reactions
  • 👀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Jun 25, 2024

Copy link
Copy Markdown
Member

So, here's the second problem: ...

This is unrelated to this PR, but it is a real bug.
Can you make a separate issue for it (and assign it to me)? Thanks.

Eclips4 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4

Eclips4 commented Jun 25, 2024

Copy link
Copy Markdown
Member Author

So, here's the second problem: ...

This is unrelated to this PR, but it is a real bug. Can you make a separate issue for it (and assign it to me)? Thanks.

Done, see #121012.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4
Resolve merge conflict
1874cf1
@Eclips4

Eclips4 commented Sep 6, 2024

Copy link
Copy Markdown
Member Author

FYI: I came back to this and found out that there is a new leak 😢 (maybe there are more than one leak... who knows..) which I bisected to 9621a7d.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented Sep 6, 2024

Copy link
Copy Markdown
Member

Do you have a reproducer for the leak?

And just to clarify, it's a "real" leak (objects that aren't cleaned up at shutdown)? It's normal for the JIT to allocate some new objects during hot loops, but their reference counts and everything should still be correct.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4

Eclips4 commented Sep 7, 2024

Copy link
Copy Markdown
Member Author

Do you have a reproducer for the leak?

And just to clarify, it's a "real" leak (objects that aren't cleaned up at shutdown)? It's normal for the JIT to allocate some new objects during hot loops, but their reference counts and everything should still be correct.

It's look like a real leak since ./python -X showrefcount IIRC reports the state of reference counter at the shutdown of the interpeter.
MRE:

cases = [
    tuple(i for i in range(3)) for _ in range(39)
]


for i, j, k in cases:
    for lo in range(4):
        for hi in range(3,8):
            pass
./python -X showrefcount example.py
[1 refs, 1 blocks]
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented Sep 7, 2024

Copy link
Copy Markdown
Member

More minimal reproducer. This creates a trace, and warms up a single side exit:

for _ in range(82):
    for _ in range(0):
        pass

The issue is that we aren't clearing "child" executors from side exits when deallocating an executor.

This quick fix seems to solve the issue for me locally. Haven't dug too deep into whether it's totally correct, but it seems right.

diff --git a/Python/optimizer.c b/Python/optimizer.c
index 9198e410627..8ba2782c2c3 100644
--- a/Python/optimizer.c
+++ b/Python/optimizer.c
@@ -257,6 +257,10 @@ uop_dealloc(_PyExecutorObject *self) {
     _PyObject_GC_UNTRACK(self);
     assert(self->vm_data.code == NULL);
     unlink_executor(self);
+    for (uint32_t i = 0; i < self->exit_count; i++) {
+        self->exits[i].temperature = initial_unreachable_backoff_counter();
+        Py_CLEAR(self->exits[i].executor);
+    }
 #ifdef _Py_JIT
     _PyJIT_Free(self);
 #endif
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented Sep 7, 2024

Copy link
Copy Markdown
Member

Thanks for digging this up @Eclips4!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4

Eclips4 commented Sep 8, 2024

Copy link
Copy Markdown
Member Author

More minimal reproducer. This creates a trace, and warms up a single side exit:

for _ in range(82):
    for _ in range(0):
        pass

The issue is that we aren't clearing "child" executors from side exits when deallocating an executor.

This quick fix seems to solve the issue for me locally. Haven't dug too deep into whether it's totally correct, but it seems right.

diff --git a/Python/optimizer.c b/Python/optimizer.c
index 9198e410627..8ba2782c2c3 100644
--- a/Python/optimizer.c
+++ b/Python/optimizer.c
@@ -257,6 +257,10 @@ uop_dealloc(_PyExecutorObject *self) {
     _PyObject_GC_UNTRACK(self);
     assert(self->vm_data.code == NULL);
     unlink_executor(self);
+    for (uint32_t i = 0; i < self->exit_count; i++) {
+        self->exits[i].temperature = initial_unreachable_backoff_counter();
+        Py_CLEAR(self->exits[i].executor);
+    }
 #ifdef _Py_JIT
     _PyJIT_Free(self);
 #endif

Thanks, Brandt!
Don't you mind if I add this code to this PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4

Eclips4 commented Sep 8, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Another one problem, which seems to be introduced in f6fab21:

./python -m test -R 3:3 test_descr
Using random seed: 315532800
0:00:00 load avg: 38.54 Run 1 test sequentially
0:00:00 load avg: 38.54 [1/1] test_descr
beginning 6 repetitions
123456
.test test_descr failed -- Traceback (most recent call last):
  File "/home/eclips4/programming/programming-languages/cpython/Lib/test/test_descr.py", line 1294, in test_slots
    self.assertEqual(orig_objects, new_objects)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 16891 != 16892

test_descr failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_descr

Total duration: 1.5 sec
Total tests: run=156 failures=1 skipped=1
Total test files: run=1/1 failed=1
Result: FAILURE

Bisected to 7b21403

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented Sep 17, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Don't you mind if I add this code to this PR?

Nope, go ahead (it should probably just be a Py_XDECREF, not a Py_CLEAR, and I don't think we need to touch the temperature).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented Sep 17, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Another one problem, which seems to be introduced in f6fab21:

This is expected: the JIT allocates a new executor (which is a GC-tracked object) in the loop immediately above this assertEqual call. I'm not sure that the test is correct anyways, since a GC could have happened in the loop, changing the number of GC objects in the process.

Basically, it seems like a buggy test. But in general the JIT needs to be able to perform allocations in places where maybe they didn't happen before... so our leak checks should be resilient to that.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4
Eclips4 force-pushed the issue-120501 branch from c623d20 to 492807c Compare October 25, 2024 18:01
@Eclips4 @brandtbucher
Clear child executors from side exits
326401b
Co-authored-by: Brandt Bucher <[email protected]>
@Eclips4
Eclips4 force-pushed the issue-120501 branch from 492807c to 326401b Compare October 25, 2024 18:03
@Eclips4

Eclips4 commented Oct 25, 2024

Copy link
Copy Markdown
Member Author

Okay.. let's continue digging into this.. 😄
so, there's another failure (you can get it by running ./python -m test -R 3:3 test_capi)

======================================================================
FAIL: test_guard_type_version_executor_invalidated (test.test_capi.test_opt.TestUopsOptimization.test_guard_type_version_executor_invalidated)
Verify that the executor is invalided on a type change.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/eclips4/programming/programming-languages/cpython/Lib/test/test_capi/test_opt.py", line 1458, in test_guard_type_version_executor_invalidated
    self.assertEqual(list(iter_opnames(ex)).count("_GUARD_TYPE_VERSION"), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 1

======================================================================
FAIL: test_guard_type_version_removed (test.test_capi.test_opt.TestUopsOptimization.test_guard_type_version_removed)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/eclips4/programming/programming-languages/cpython/Lib/test/test_capi/test_opt.py", line 1353, in test_guard_type_version_removed
    self.assertEqual(guard_type_version_count, 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 1

======================================================================
FAIL: test_guard_type_version_removed_inlined (test.test_capi.test_opt.TestUopsOptimization.test_guard_type_version_removed_inlined)
Verify that the guard type version if we have an inlined function
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/eclips4/programming/programming-languages/cpython/Lib/test/test_capi/test_opt.py", line 1379, in test_guard_type_version_removed_inlined
    self.assertEqual(guard_type_version_count, 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 1

======================================================================
UNEXPECTED SUCCESS: test_guard_type_version_not_removed_escaping (test.test_capi.test_opt.TestUopsOptimization.test_guard_type_version_not_removed_escaping)
Verify that the guard type version is not removed if have an escaping function
----------------------------------------------------------------------
Ran 924 tests in 11.506s

FAILED (failures=3, skipped=4, unexpected successes=1)
test test_capi failed
test_capi failed (3 failures)

== Tests result: FAILURE ==

1 test failed:
    test_capi

Total duration: 23.5 sec
Total tests: run=924 failures=3 skipped=4
Total test files: run=1/1 failed=1
Result: FAILURE
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4
Merge branch 'main' into issue-120501
f18739b
@Eclips4

Eclips4 commented Oct 25, 2024

Copy link
Copy Markdown
Member Author

UPD: There are many more failures:

17 tests failed:
    test.test_asyncio.test_taskgroups test.test_inspect.test_inspect
    test.test_multiprocessing_forkserver.test_processes
    test.test_multiprocessing_spawn.test_processes test_argparse
    test_ast test_capi test_clinic test_collections test_ctypes
    test_descr test_difflib test_enum test_itertools test_monitoring
    test_tarfile test_zoneinfo
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented Oct 26, 2024

Copy link
Copy Markdown
Member

Okay.. let's continue digging into this.. 😄
so, there's another failure (you can get it by running ./python -m test -R 3:3 test_capi)

Let's keep this PR in scope. This new issue you found isn't a refleak. It's just a byproduct of the fact that when we run out of version numbers, we literally can't perform certain optimizations anymore.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Eclips4

Eclips4 commented Nov 1, 2024

Copy link
Copy Markdown
Member Author

Okay.. let's continue digging into this.. 😄
so, there's another failure (you can get it by running ./python -m test -R 3:3 test_capi)

Let's keep this PR in scope. This new issue you found isn't a refleak. It's just a byproduct of the fact that when we run out of version numbers, we literally can't perform certain optimizations anymore.

Should I create an separate issue for that?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Eclips4 added 7 commits January 9, 2025 13:34
@Eclips4
Merge branch 'main' into issue-120501
45e0c3a
@Eclips4
Remove unnecessary decref
a7957fa
@Eclips4
Fix incorrect conversion specifier
7d90844
@Eclips4
Steal reference instead of increfing
f2cb39d
@Eclips4
Revert changes because counter optimizer has been removed
09f7314
@Eclips4
Merge branch 'main' into issue-120501
7d291ae
@Eclips4
Revert unnecessary changes
b10a0c9
@Eclips4

Eclips4 commented Jan 20, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

So far, the latest observable leaks are memory block leaks:

eclips4@nixos ~/p/p/cpython (issue-120501)> ./python -m test -R 3:3 test_random
Using random seed: 315532800
0:00:00 load avg: 2.11 Run 1 test sequentially in a single process
0:00:00 load avg: 2.11 [1/1] test_random
beginning 6 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
123:456
XX. 121
test_random leaked [1, 2, 1] memory blocks, sum=4
test_random failed (reference leak)

== Tests result: FAILURE ==

1 test failed:
    test_random

Total duration: 13.5 sec
Total tests: run=103
Total test files: run=1/1 failed=1
Result: FAILURE

Though, for a larger number of repetitions it doesn't fail:

eclips4@nixos ~/p/p/cpython (issue-120501) [2]> ./python -m test -R 6:6 test_random
Using random seed: 315532800
0:00:00 load avg: 2.35 Run 1 test sequentially in a single process
0:00:00 load avg: 2.35 [1/1] test_random
beginning 12 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
123456:789012
XX.121 1.12..
test_random leaked [1, 0, 1, 2, 0, 0] memory blocks, sum=4 (this is fine)

== Tests result: SUCCESS ==

1 test OK.

Total duration: 27.5 sec
Total tests: run=103
Total test files: run=1/1
Result: SUCCESS

So.. it doesn't look like a real leaks, because ./python -Xshowrefcount always reports that there are no leaks at all:

./python -Xshowrefcount -m test -R 3:3 test_random
Using random seed: 315532800
0:00:00 load avg: 1.99 Run 1 test sequentially in a single process
0:00:00 load avg: 1.99 [1/1] test_random
beginning 6 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
123:456
XX. 121
test_random leaked [1, 2, 1] memory blocks, sum=4
test_random failed (reference leak)

== Tests result: FAILURE ==

1 test failed:
    test_random

Total duration: 13.6 sec
Total tests: run=103
Total test files: run=1/1 failed=1
Result: FAILURE
[0 refs, 0 blocks] 

I understand that with Tier 2, there could be allocations that didn't happen before. But I have no idea where they are. Any hints?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Eclips4 added 2 commits April 13, 2025 15:24
@Eclips4
Merge branch 'main' into issue-120501
cc17195
@Eclips4
Resolve merge conflict
7ef5009
@github-actions

github-actions Bot commented Apr 17, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner left review comments
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner
@brandtbucher brandtbucher Awaiting requested review from brandtbucher
@Fidget-Spinner Fidget-Spinner Awaiting requested review from Fidget-Spinner Fidget-Spinner will be requested when the pull request is marked ready for review Fidget-Spinner is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news stale Stale PR or inactive for long period of time.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@Eclips4 @JeffersGlass @markshannon @brandtbucher @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.