GeezerVison is showing github.com — Improve code coverage · Issue #68 · Technologicat/unpythonic
Improve code coverage #68DescriptionCoverage is currently at 85%. At least some of the final 15% would be nice to add.
A few percent could be false positives from the use of block macros in the tests for those block macros, or trivial error handling code that's not triggered by the tests, but there can be some exotic cases that are genuinely missing.
There is now a a CI setup that runs the test suite on Pythons 3.4, 3.5, 3.6, 3.7 and PyPy3, as well as runs coverage.py and uploads the coverage report to codecov.io.
Significant improvements in code coverage are slow work, and may have to wait until 0.14.4 or later.
See #5 for history. ActivityTechnologicat commented on Aug 28, 2020 Doing some of this already in 0.14.3.
It's been useful, a number of small bugs have been discovered and fixed. Technologicat commented on Aug 30, 2020 Currently at 89%.
The remaining missing (i.e. non-covered) lines fall into a few categories:
- False negatives, mostly related to macro expansion of block macros (Support (or suggested workaround) for syntactic macros (macropy/mcpy) coveragepy/coveragepy#1004), plus some module docstrings.
- Python version specific code, not run under CPython 3.6, where coverage is currently measured.
- Code specific to 3.4 or 3.5 has been tagged
# pragma: no cover. Code for 3.7 and 3.8 needs to be re-checked (grep: Python 3.7, Python 3.8) once we bump the minimum Python version.
- Code specific to PyPy3 has at least mostly been tagged
# pragma: no cover, but this could be revisited. Have to first check PyPy's plans for 3.7+ support, to know how relevant it will likely be in the future...
- Code whose tests are currently disabled due to incompatibility with Python 3.4 syntax (which would crash the whole test suite while compiling the relevant module, if running under 3.4).
- E.g. there is a case in
namedlambda, in unpythonic.syntax.lambdatools, which would be triggered by the tests, but the syntax used the tests (to trigger that code) requires the * and ** unpacking generalizations added in 3.5 and 3.6.
- There should be a
warn[] in the corresponding tests, so it can be fixed later when we bump the minimum Python version.
- Code whose tests are currently disabled due to MacroPy crashes on
complex and bytes azazel75/macropy#26. Basically anything dealing with bytestrings or complex numbers. Affects especially unpythonic.net.* and unpythonic.typecheck.
- Test framework (
unpythonic.test.fixtures, unpythonic.syntax.testingtools), particularly code handling test failures and errors. Cannot use the test framework to test itself.
- Network components (
unpythonic.net). Need to think of something here. Not really robust without automated tests, but it's the kind of interactive stuff that's hard to test compactly.
- Tricky (often corner) cases:
- Two in the continuations macro in
unpythonic.syntax.tailtools. Still have to figure out what exactly I was thinking when I coded that.
- The multithreading paranoia in
unpythonic.singleton and unpythonic.symbol. The code seems correct to me. I can't think of a reason that the final check shouldn't be there, but haven't found a way to cause it to trigger, either.
- One in
unpythonic.arity._kwargs.
- A
__setstate__ in unpythonic.test.test_singleton.
Notable exclusions:
- Any code that emits a macro expansion error (
assert False, "...message..." in macro code).
- Import failure handlers for optional dependencies. We require those dependencies for development in
requirements.txt.
- Failure code paths in tests.
- The "if main" idiom in the individual test modules. (Allows them to run individually as main programs through the
macropy3 wrapper, but runtests.py imports them as modules.)
- Most of the custom
__repr__.
- Anything that catches
UnknownArity.
- Quote blocks in macros. (Quoted code is not even supposed to run at that point, obviously.)
- Some paranoia code.
- In
mogrify: is there any realistic MutableSequence that has no clear(), or MutableSet that has no update()? The API for those ABCs allows that, so it's checked and worked around if it happens. But e.g. the builtin list has clear() and set has update(), so it's unlikely any client code will ever run into that case.
- In
isoftype: the implementation failure guards.
- A
__del__ in unpythonic.dynassign. It certainly runs, but isn't reported as covered for some reason. So it's excluded to stop coverage.py from complaining about it.
- A MacroPy 1.1.0b2 fallback in
unpythonic.syntax.lambdatools.
- Some backward compatibility aliases, e.g.
escape (now throw) and setescape (now catch).
Technologicat commented on Sep 6, 2020 Now at 91%.
- Beside just improving test coverage, we should aim at high coverage (as high as reasonable) in each module's own unit tests.
- Currently (esp. in low-level utilities) there's some code that's only exercised by tests of other modules. Hence a module, when tested individually, may have lower test coverage than it has when the full test suite runs. If that module breaks, this may make bugs harder to find than necessary.
- In some cases, such as
lazyutil, that's just fine: that module only exists for integration with lazify. But if a module has a reason to exist on its own, then it should also be tested on its own, as much as reasonable.
Technologicat commented on Sep 8, 2020 Now at 92%.
Basically unpythonic.syntax.letdoutil and unpythonic.syntax.simplelet are the last ones that are easy-ish (but somewhat tedious) to get fully covered. Then it's just the REPL/networking and test framework code, and meta-utilities such as setup.py and countlines.py. Technologicat commented on Sep 10, 2020 Now at 92.67%.
That's it for the tedious but low-hanging fruit.
Now just to layer the tests (issue #69) so that we can test unpythonic.syntax.testingtools, and then leave adding tests for the REPL/networking code to a future release (since this is already much better, and this release is already getting big). Technologicat commented on Sep 10, 2020 Ah, but before that:
- Go through all tests one more time, and:
- Use
the[] as appropriate to auto-print something useful for debugging if the tests ever fail
- Measure per-unit coverage, improve where necessary
- Done for regular code as of 66fd6dd.
- Done for macro code as of c2577d0.
- Figure out and add a test for the corner case in
continuations.
In the future:
- Improve naming in all test examples (no
foo, bar).
Technologicat commented on May 22, 2021 As of 0.15.0-pre, we're at 94% coverage. Technologicat commented on Apr 24, 2026 Status update (2026-04-24, post-2.1.0):
unpythonic.net now has test coverage — tier-1 REPL client tests in unpythonic/net/tests/test_client.py (18 tests, in-process), plus direct unit tests for ReceiveBuffer and decodemsg / ReuseAddrThreadingTCPServer. Also shipped in 2.1.0: Windows support for net.ptyproxy via socket.socketpair().
The 2021 notes above are otherwise out of date — the code has churned considerably through the 2.0.0 / 2.1.0 modernization. Current coverage overall is good; keeping this open at ∞ as a standing item to capture future gaps. New concrete coverage issues, if any, can be filed as subissues of this one rather than as fresh top-level issues.
— Claude (Opus 4.7)
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|