GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-130167: Minor `textwrap.dedent()` optimization by Marius-Juston · Pull Request #131925 · python/cpython Conversation
|
| Benchmark | raw | new |
|---|---|---|
| raw_text: "abc \t" | 2.91 ms | 3.33 ms: 1.15x slower |
| raw_text: " " | 4.31 ms | 4.17 ms: 1.03x faster |
| raw_text: " \t abc" | 4.04 ms | 4.40 ms: 1.09x slower |
| raw_text: " \t abc \t " | 4.08 ms | 4.40 ms: 1.08x slower |
| raw_text: 1000 spaces | 34.9 ms | 27.9 ms: 1.25x faster |
| Basic indented text with empty lines | 1.88 us | 1.82 us: 1.03x faster |
| Text with mixed indentation and blank lines | 1.85 us | 1.79 us: 1.03x faster |
| No indentation (edge case) | 1.24 us | 1.14 us: 1.09x faster |
| Only blank lines | 958 ns | 641 ns: 1.49x faster |
| Edge case: No common prefix to remove | 1.09 us | 993 ns: 1.10x faster |
| Edge case: Single indented line | 998 ns | 855 ns: 1.17x faster |
| Edge case: Single indented line only | 631 ns | 432 ns: 1.46x faster |
| Edge case: Empty text | 89.3 ns | 81.0 ns: 1.10x faster |
| no_indent | 2.57 us | 2.50 us: 1.03x faster |
| mixed | 10.4 us | 10.9 us: 1.05x slower |
| large_text | 77.7 us | 85.7 us: 1.10x slower |
| whitespace_only | 859 ns | 599 ns: 1.43x faster |
| Geometric mean | (ref) | 1.08x faster |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
It seems for the benchmarks handling large text, there's a consistent slowdown. Could you plot a chart of running the function with inputs of different sizes? Perhaps take the unicodeobject file and profile the first 500 lines, then the first 1000, 1500, etc, for this PR and the version using min/max. A |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
It stays a consistently slower by 5-8% except for long lines test
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
From additional testing, this way is slightly faster in the ranges where the lines are between 1-18.
after analyzing all the function documentation in the Python codebase the average documentation length seems to be the following https://gist.github.com/Marius-Juston/08c574901317ee40837ab87ce0855685:
( excluding all 1 line comments ) so technically, this implementation is slightly faster for the average documentation length ( now is this performance increase really worth it, no probably not lol ) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| l1 = min(non_blank_lines, default='') | ||
| l2 = max(non_blank_lines, default='') | ||
| margin = 0 | ||
| l1 = None |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
| l1 = None | |
| l1 = '' |
You can then skip the is None check in the loop. Similar for l2 (but you have to pick the default with care)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| if line and not line.isspace(): | ||
| if l1 is None or line < l1: | ||
| l1 = line | ||
| if l2 is None or line > l2: |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Once you skip the if None check, this if can be an elseif
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
I'm against this change not only because it's harder to read but also because it only offers an overall <10% speed-up. In general, we accept optimizations exceeding that threshold. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@eendebakpt with the new changes:
Small values
Large file
So on the large files we get 0 improvement, a 13% slight improvement on the synthetic test benches and on smaller values |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Agreed that this optimization is most likely not necessary, though this is very slightly more efficient memory wise since it does not have a |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| val = False | ||
| for i, line in enumerate(lines): | ||
| # Compute min + max concurrently + normalize others | ||
| if line and not line.isspace(): | ||
| if val: | ||
| if line < l1: | ||
| l1 = line | ||
| elif line > l2: | ||
| l2 = line | ||
| else: | ||
| val = True | ||
| l1 = l2 = line | ||
|
|
||
| else: | ||
| lines[i] = '' | ||
|
|
||
| if not val or not l1: | ||
| return '\n'.join(lines) | ||
|
|
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
| val = False | |
| for i, line in enumerate(lines): | |
| # Compute min + max concurrently + normalize others | |
| if line and not line.isspace(): | |
| if val: | |
| if line < l1: | |
| l1 = line | |
| elif line > l2: | |
| l2 = line | |
| else: | |
| val = True | |
| l1 = l2 = line | |
| else: | |
| lines[i] = '' | |
| if not val or not l1: | |
| return '\n'.join(lines) | |
| l1 = 'z' | |
| l2 = '' | |
| for i, line in enumerate(lines): | |
| # Compute min + max concurrently + normalize others | |
| if line and not line.isspace(): | |
| if line < l1: | |
| l1 = line | |
| if line > l2: | |
| l2 = line | |
| else: | |
| lines[i] = '' | |
| if not l1: | |
| return '\n'.join(lines) | |
| margin = 0 | |
| for margin, c in enumerate(l2): | |
| if c != l1[margin] or c not in ' \t': | |
| break | |
This saves chechking the val. Performance gain is only a bit though.
I agree with the other reviewers: unless we find out a way to make this much faster, the small gains are not worth the change.
@Marius-Juston Thanks for putting in the work to get all the performance numbers!
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Thanks for getting the benchmarks, sometimes its hard to get contributors to quote numbers, and you've gone above and beyond with benchmarking :).
But unfortunately, the numbers make me -1 on this.
min/max are builtins that are written in C; they're pretty fast, especially compared to multiple lines of Python code. If we're concerned about those being slow, let's try to optimize those on the C level instead.I've learned that it can be hard to know when to stop painting. I think we should put down the brush, and enjoy the optimization that's already landed 🙂.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Makes sense. I am just having so much fun doing this it is definitely hard to stop lol! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Closing this down then! ( I will be putting a feature request for the |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
I think it's already been proposed but rejected so search through issues and https://discuss.python.org/c/ideas/6 first. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
I think I found it, the https://discuss.python.org/t/minmax-function-alongside-min-and-max/2834/73 issue got closed down. It seems like it got stale and just ignored? A shame. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Feel free to open a new thread referencing the one you found. Personally I'd suggest A |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.