GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Rewrite indentation code by axvr · Pull Request #31 · clojure-vim/clojure.vim

Conversation

@axvr

@axvr axvr commented Apr 27, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Tip

As of the latest change, indentation is 2–3x faster with the potential to surpass 10x with Vim9script and Lua.

This PR contains a rewrite of the entire Clojure indentation code that aims to:

  1. Improve the (currently abysmal) indent performance and unlock yet more improvements in future,
    • Improve indent performance #6
    • Port performance improvements to Neovim/Lua #32
    • Create Vim 9 script variant of the new indentation algorithm #35
  2. Change the defaults to be more similar to the Clojure Style Guide, cljfmt, Emacs and VS Code defaults,
    • cljfmt-compatible ns form formatting #22
  3. Enable greater configuration of indent rules (such that various indentation styles are possible),
    • Question: is there a way to configure Tonsky's indentation rule? #21
  4. Simplify the indentation code as much as possible (currently so complex, it is near impossible to work on),
  5. Improve accuracy of the indentation algorithms, eliminating most (if not all) previous indentation bugs,
    • [Bug] incorrect indentation for curly braces with = #34
    • Rewrite indentation code #31 (comment)
  6. Eliminate reliance on syntax highlighting to be compatible with alternate syntax highlighting systems (e.g. Tree-sitter),
    • Compatibility with tree-sitter #33
  7. Create a rich indentation test suite,
    • Add unit tests #1
    • Run indentation tests on Neovim too #39
    • Rewrite indentation testing code #26
  8. Make quality of life improvements (when the (Neo)Vim version supports them), such as preventing = operator from altering the indentation of multi-line strings and setting the 'lisp' option in Clojure buffers.
    • [Feature Request] Set lisp option #40

To do

  • New indentation algorithm.
  • More indentation tests + test the different config options.
  • Run indentation tests on Vim and Neovim.
  • Update documentation (doc file and README).
  • Replace existing configuration options.
  • Resolve most remaining TODOs.
  • Create a GitHub issue for the remaining missing indent features:
    • Indentation rules for reader conditionals (i.e. #?@(...), #?(...)).
    • Complex indentation rules (e.g. letfn, extend-protocol, etc.).
  • Maybe write blog post about this update?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

garrett-hopper reacted with rocket emoji
All reactions
  • 🚀 1 reaction
axvr added 15 commits April 21, 2023 01:29
@axvr
Begin rewrite of indentation code
7d6bdd7
The objectives are:

    1. Simplify the indentation code; previous implementation has become
       so complex it is impossible to maintain,

    2. Significantly improve performance; previous indentation code was
       painfully slow, (see issue #6)

    3. Maximum configurability; should be configured similarly to cljfmt
       and make previously impossible things possible (e.g. issue #21).

As of this commit, objectives 1 and 2 have been met, but work on
objective 3 has not yet begun.  There will continue to be further
improvements, particularly around performance and the "what if syntax
highlighting is disabled?" scenario.

These changes will unfortunately be backwards incompatible, but
hopefully the improved performance and API will make up for it.
@axvr
Double (or better) indent performance!
5706ea3
By utilising some ugly mutable state, this change more than doubles the
performance of the indentation code.  You can expect even further
benefits in larger code blocks.  This should completely eliminate any
need for the `clojure_maxlines` configuration option.
@axvr
A bit of code clean up and minor optimisations
5f083ff
@axvr
Check for maps before vectors for further performance enhancement
f4e0689
Since maps are far more likely to across multiple lines than vectors
(and when they are they tend to be much larger, e.g. EDN files!), we can
further optimise indentation by checking if the line is inside a map
*before* checking it is inside a vector.

(This happens because of the performance improvement made with the
addition of the optimised `CheckPair` function.)
@axvr
Add fallback for when Vim was not compiled with searchpairpos
2627e1c
@axvr
Fix string detection when entering a newline while in insert mode
4a3cab7
@axvr
Minor performance improvement when file contains multi-line strings
d1ec02c
@axvr
Mimic multi-line string/regex indent behaviour of VS Code and Emacs
c3047be
The `=` operator will no longer alter the indent of lines within
a Clojure multi-line string or regular expression.

The previous behaviour was annoying for writing detailed doc-strings, as
it made reformatting the file with `gg=G` not possible as it would screw
up the indentation within the doc-strings.

Now the behaviour matches that of VS Code and Emacs.
@axvr
When clojure_align_multiline_strings is -1, no indentation
eb0463e
By setting the `clojure_align_multiline_strings` option to `-1` it will
switch to traditional Lisp multi-line string indentation, of no indent.
@axvr
Update indentation options sections of the README and vim help file
d69008f
@axvr
Improve some of the comments in the indentation code
d73e408
@axvr
Move string indent picker to a separate function
d177b3b
@axvr
Fix false positive multi-line string detection
51e6a43
Sometimes it seems that when `=`ing over a block, Vim will sometimes
not re-highlight strings correctly until we have already ran
`searchpairpos`.  In this case, we implement a hacky workaround which
detects the false-positive and recovers from it.
@axvr
EDN files are unlikely to contain many lists, so check them last
e573da7
@axvr
Yet more string detection fixes and slight code clean up
242dc9d
@axvr
axvr force-pushed the indent-forms branch from 8279a89 to 242dc9d Compare April 28, 2023 14:21
axvr added 2 commits April 28, 2023 16:30
@axvr
Initial work on list indentation
342f35f
Still lots to do here, this is just the initial work.
@axvr
More indentation code clean up and readability improvements
51f5dcb
@axvr

axvr commented Apr 30, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

I've been experimenting with a custom algorithm as per the check list item above. This version eliminates all syntax highlight checks (and therefore no longer requires the various hacks to make those checks work) plus it is approx the same speed as my first rewrite attempt and uses about the same amount of code.

However the core benefits of this version is that it works when syntax highlighting is switched off, and eliminates the performance bottle neck of syntax highlight checks. This means that it will be possible to write a version of the algorithm in Vim9 script, to get another significant performance improvement! (Maintaining both a legacy Vim script and Vim9 script versions should be feasible due to how much simpler this is than the original implementation.)

TL;DR: I'll push a new version to this branch soon which will be:

  1. Simpler with no hacks,
  2. Can be become even faster (via Vim9 script),
  3. Works even when syntax highlighting is switched off.

Once this branch is feature complete, I'll share benchmarks comparing it to the original as is on master.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

axvr added 4 commits May 1, 2023 00:58
@axvr
Start of a small Clojure reader for indentation w/o syntax highlighting
cc9cda7
Not complete yet, but getting there.
@axvr
Completed the mini Clojure reader as the core of the indentation system
ee2acc2
Some refactoring should be possible here and further optimisations.
Once all optimisations I can think of have been implemented, I'll try
writing an alternate Vim9 script version.

(The syntax highlight group checks used in previous implementations of
the indentation code was the core bottleneck, so a Vim9 script version
would not have been much faster.)
@axvr
Reader indent algorithm perf has surpassed the prev syntax highlight one
bdbc281
The performance of the new reader-like indentation algorithm has now
surpassed the performance of my previous syntax highlighting approach
with better accuracy and no hacky code required.
@axvr
Small refactor to improve code clarity in indentation code
8ee73c5
@axvr
axvr force-pushed the indent-forms branch from 437e727 to 8ee73c5 Compare May 1, 2023 13:26
@axvr

axvr commented May 1, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Performance benchmarks as of current change (+ unpushed Vim9 version). This is measuring the time to reindent a whole file on a Macbook Pro 2021, so expect much more noticable performance gains on slower hardware. 🚀

Code version 200 lines of Clojure 600 lines of EDN
Original 0.96s 4.6s (clojure_maxlines = 0), 3.5s (clojure_maxlines = 300)
New (legacy) 0.33s 2.6s
New (Vim9) 0.06s 0.46s

(I'll share new benchmarks (and the full reports generated by Vim) once this branch is feature complete, although I don't expect it to change much.)

Even with the Vim9 implementation in the code too (not pushed yet, still figuring out how best to integrate it in a maintainable way), we can expect the file size of indent/clojure.vim to be ~150 LOC less than the original!

Regarding Neovim, until Vim9 script support is added, the "New (legacy)" code will be used. If anyone would like to volunteer to write a Lua implementation of this code for Neovim please open an issue and we can discuss how to integrate it.

kawas44 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@NoahTheDuke NoahTheDuke mentioned this pull request May 2, 2023
Port performance improvements to Neovim/Lua #32
Open
@axvr
Fix indentation bug during comment detection
a4beb52
@axvr axvr mentioned this pull request May 30, 2023
Compatibility with tree-sitter #33
Open
@axvr
Fix accidental detection of backslashes as tokens
3d8197e
Previously backslashes were accidentally detected as tokens by the
indentation tokeniser.  This meant that character literals, would break
indentation of everything after them.
@axvr axvr mentioned this pull request Jun 22, 2023
[Bug] incorrect indentation for curly braces with = #34
Open
@axvr
Indent with- macros and functions like cljfmt default
78ecad5
@axvr
axvr force-pushed the indent-forms branch from f6667a5 to 78ecad5 Compare January 19, 2025 12:10
axvr added 2 commits January 25, 2025 21:35
@axvr
Run indentation tests against Vim AND Neovim
6ce7017
Closes: #39
@axvr
Install latest Vim and Neovim in GitHub Actions
d2f39f9
@axvr axvr linked an issue Jan 25, 2025 that may be closed by this pull request
Run indentation tests on Neovim too #39
Open
axvr added 2 commits January 25, 2025 21:48
@axvr
Merge latest changes from 'origin/master'
8e9ed28
@axvr
New indentation test runner
e5f86af
@axvr
axvr force-pushed the indent-forms branch from e04d1c0 to e5f86af Compare January 26, 2025 01:53
@axvr
Improve new test runner output and detect script errors
7659541
@axvr
axvr force-pushed the indent-forms branch from 80ec263 to 7659541 Compare January 26, 2025 02:06
axvr added 7 commits January 26, 2025 02:10
@axvr
Speed up indentation tests on Vim
1ea333a
@axvr
Move indenttime script to dev folder as time-indent
aae9ae8
@axvr
Update GitHub Actions
1883215
@axvr
Kaocha not needed anymore, so removing dependency
5f995a1
@axvr
Bump copyright year
f3358f3
@axvr
Remove unneeded config from clj/profile.clj
13ca6fe
@axvr
Run indentation tests in Vim's silent Ex-improved mode
5b277db
Avoids screen flickering during test execution.
@axvr
axvr marked this pull request as ready for review February 3, 2025 22:26
axvr added 2 commits February 4, 2025 22:58
@axvr
Reduce indentation test runner noise and add colour output
d8dc8a7
@axvr
GitHub Actions "annotations" feature doesn't work with colour output
e50122e
@axvr
axvr force-pushed the indent-forms branch from 01d8513 to e50122e Compare February 4, 2025 23:20
@axvr
Simplify indentation test runner output on GitHub Actions
c4b8d48
@garrett-hopper garrett-hopper mentioned this pull request May 8, 2025
Question: is there a way to configure Tonsky's indentation rule? #21
Open
@garrett-hopper

garrett-hopper commented May 8, 2025

Copy link
Copy Markdown

FYI, the "uniform" indent_style isn't quite following Tonsky's Better Clojure formatting

Multi-line lists that start with a symbol are always indented with two spaces,
Other multi-line lists, vectors, maps and sets are aligned with the first element (1 or 2 spaces).

Multi-line lists that start with a non-symbol are currently indented with 2 instead of 1 space.

E.g.

(ns namespace
  (:require
   [clojure.set :as s]))

The final line is indented 1 space because :require is keyword and not a symbol.

if indent_style ==# 'uniform' | return base_indent + 1 | endif

Not sure the best way way for this to check if the first list item is a symbol or not. 🤔

I wrote this for #21 before I realized this rewrite was being done. It's a pattern that will match symbols, adapted from the tree-sitter grammar. (Though it may be easier to just special case non-symbols as first list item.)

local suffixonly = ":#'0-9\\/" -- not allowed as first character
local forbidden = {
  [[\[\]\n\r\t \\]], -- brackets, whitespace, slashes
  '(){}"@~^;`,', -- parens, braces, special characters
  suffixonly,
}
local prefix = string.format("[^%s]", table.concat(forbidden))
local suffix = string.format([[\(%s\|[%s]\)*]], prefix, suffixonly)
local symbol = prefix .. suffix
[^\[\]\n\r\t \\(){}"@~^;`,:#'0-9\/]\([^\[\]\n\r\t \\(){}"@~^;`,:#'0-9\/]\|[:#'0-9\/]\)*
axvr reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@axvr axvr mentioned this pull request Jul 12, 2025
how to disable completely? #43
Open
@axvr

axvr commented Jul 12, 2025

Copy link
Copy Markdown
Member Author

Thanks for raising this @garrett-hopper. I'll add it to my list and try to fix it soon.

(Yet more mildly irritating rules in the common Clojure indentation styles 😆 😭)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@axvr

axvr commented Jul 12, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Apologies to anyone waiting patiently for this to be merged. I haven't managed to find much time to dedicate to this and each time I come back to it, it requires a lot of context reloading as the standard Clojure indentation styles are shockingly all very complicated (contrary to Clojure itself) and to implement them efficiently in Vim script is forever painful.

Currently what exists in this branch is IMO already far superior to what is currently on the master branch, so my current plan is:

  1. fix a few of the more major outstanding issues,
  2. document the clojure_indent_rules option with warnings that it may change in the future,
  3. merge, 🥳
  4. open issues for the remaining known problems that I don't consider to be dealbreakers,
  5. gather more feedback on other issues with wider use,
  6. fix issues critical to the core algorithm,
  7. later complete Create Vim 9 script variant of the new indentation algorithm #35 and Port performance improvements to Neovim/Lua #32 once the core algorithm is perfected to avoid needing to implement the same fixes 3 times.

I hope to get round to the first 4 soon, but can't make any promises on when I will do so.

garrett-hopper and NoahTheDuke reacted with heart emoji
All reactions
  • ❤️ 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@NoahTheDuke

NoahTheDuke commented Jul 14, 2025

Copy link
Copy Markdown

Pretty exciting! Thanks for shepherding this! I've been using this branch exclusively since you opened the PR and I'm quite pleased with it.

axvr and dharrigan reacted with heart emoji
All reactions
  • ❤️ 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@NoahTheDuke

NoahTheDuke commented Apr 15, 2026

Copy link
Copy Markdown

Bug report: defining protocol methods in defrecord under "standard" still indents the body to match the arg vector:

(defrecord Example [a b c]
  MyProto
  (method1 [this d e f]
    (+ a b c d e f)))

becomes

(defrecord Example [a b c]
  MyProto
  (method1 [this d e f]
           (+ a b c d e f)))

It looks like there's tests covering this, so I'm not sure what's wrong.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@NoahTheDuke NoahTheDuke mentioned this pull request Jul 14, 2026
Reflow comments #44
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

indentation Affects indentation performance Affects performance

Projects

None yet

Milestone

Indentation overhaul

Development

Successfully merging this pull request may close these issues.

[Bug] incorrect indentation for curly braces with = Compatibility with tree-sitter Question: is there a way to configure Tonsky's indentation rule? Improve indent performance Add unit tests
Run indentation tests on Neovim too [Feature Request] Set lisp option

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@axvr @NoahTheDuke @SavageMessiah @garrett-hopper
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.