MacVim is a fork of Vim in order to provide a GUI, and as such it routinely needs to merge from upstream. The process is not completely trivial as MacVim is an intrusive fork, in that it doesn't simply wraps a Vim executable in a GUI with minimal change, but instead has source code changes sprinkled all over Vim. There are a few reasons for that, but in particular:

• Vim isn't designed to be embeddable (which NeoVim helps resolve). In particular, the GUI code is quite intermingled with the core source code. Vim doesn't have a native GUI frontend for macOS, as such there needs to be a fair amount of code added to support it.
• MacVim makes modifications to the documentation and runtime files to provide a more seamless experience rather than simply slapping a raw Vim binary inside terminal emulator.
• Certain features like trackpad gesture support or Dark Mode support in scripting relies modifying core source code.

Note: Merging from upstream is usually done by the code maintainers. This is necessary because each merge is usually an involved process and requires going through a lot of files/merge conflicts, and isn't very conductive to a pull-request model that are hard to properly review. Please do not file a pull request to merge from upstream. If you notice that MacVim hasn't merged from upstream in a while (2-3 weeks), file an issue.

To merge from upstream, add both the Vim (use the remote name vim to be consistent with version history) and MacVim remote to your Git repository, then run git merge vim/master. Resolve conflicts, quickly test locally, and commit.

MacVim's master branch is protected behind branch protection which requires status checks to have passed before you are allowed to push commits to it or merge PRs. While it's possible for a maintainer to override that by directly pushing new commits to master, the normal procedure is to first push the new merge commit to a "staging" branch first, and let CI checks run to completion.

Once CI has passed for that branch, push the same commit to master branch. There should be no GitHub warnings about overriding branch protection rules if staging has passed CI with said commit. This makes sure that we only push tested commits to master branch and prevents breakage. While it's possible to set up a CI step to automatically merge from staging once CI has passed, generally we prefer to have read-only CI for security reasons, and so we do this step manually for now.

Some merges will result in merge conflicts and need some cleanups. Some caveats to pay attention to:

• While annoying, the committer should review at least the descriptions of every single change in Vim to watch out for potential gotchas. Sometimes there are important changes that introduce subtle bugs while not generating merge conflicts, and human eyes is still the best way to catch such issues. It also helps you notice any interesting changes that you may want to add to release notes, or features that MacVim should integrate. A good command to run is to git log --reverse origin/master..vim/master | vim - to get a list of commits to review.
• A lot of MacVim-specific code are wrapped under FEAT_GUI_MACVIM, and merge conflicts tend to be changes around those places as the contexts around the code blocks have changed. Watch out for refactored code as Git doesn't do a good job tracking those so if you see mass deletions/additions, make sure the MacVim-specific changes make it across as well.
• Files that need special handling:
  • README.md in Vim's repo is moved to README_vim.md (which is linked to from MacVim's README.md). Because Git doesn't support native rename tracking, it tends to get confused by any change in Vim's README file. Usually you just want to use the MacVim's version (git checkout --ours -- README.md), and copy the file to README_vim.md instead (git show vim/master:README.md >! README_vim.md).
  • src/configure.ac / src/auto/configure: If these were changed, src/auto/configure needs to be re-generated using autoconf. The version of autoconf should match the upstream version. Easiest way to do it is to install the correct version using Homebrew (brew install autoconf), then do cd src; make AUTOCONF=/opt/homebrew/Cellar/autoconf/2.71/bin/autoconf autoconf.
  • src/ex_cmdidxs.h: This file is auto-generated from all the registered Vim Ex commands. Since MacVim has unique commands (e.g. :macmenu) this file is likely to generate a conflict every time Vim adds a command. Use cd src; make cmdidxs to regenerate this file.
  • .github/workflows are quite different from upstream. Usually, we don't merge from upstream and you can directly pick MacVim's version, but it's useful to see if there are interesting changes that could be applied to MacVim as well.
    • Vim's own CI YAML files are mostly moved to a separate vim_unused folder. During normal Git merge, rename detection should successfully resolve any changes to the upstream files. It could be interesting to see changes to the upstream ci-macos.yml file and incorporate changes to our own ci-macvim.yml if they look useful.
  • .github/pull_request_template.md is removed in MacVim.
  • src/Makefile: At the end of this file are a list of auto-generated file dependencies. This list is different between Vim and MacVim. If you see merge conflicts here, just delete all of them and run cd src; make depend which will recreate the dependencies correctly.

It's also sometimes easier to merge small amounts of Vim commits one at a time (e.g. 100-200 commits) instead of all at once if MacVim is a little behind.

GitHub allows commits to automatically close issues via the "close #XXXX" markers in commit messages. Unfortunately, this feature is not configurable at all as of this writing and cannot be turned off. If upstream Vim used this to close an issue, it would be referring to an issue in Vim, but the moment we merge the commit and push to GitHub will try to close a random unrelated issue of the same number in MacVim as well.

There is no easy way to work around this issue, because we don't want to rewrite Vim's history to remove the issue markers (as that will change the commit hash). After you have pushed a merge, you should visit https://github.com/macvim-dev/macvim/issues?q=sort%3Aupdated-desc+ to see if there are any recently updated issues, and if some issues were accidentally closed, just go and manually re-open.

Usually, Vim issue numbers are much larger than MacVim issue numbers and we will be fine. You could manually check what issues are referenced when doing an upstream merge by doing the following:

git log master..vim/master | grep -o "#\d\+" | sort -Vr

Then compare that with the largest issue number in MacVim:

curl -sL "https://api.github.com/repos/macvim-dev/macvim/issues?per_page=1" | jq ".[0].number"

If the referenced issues are all larger than MacVim's largest issue number, then we will be fine.

See https://github.com/orgs/community/discussions/17308

Interesting changes in Vim should be included in MacVim release notes for users who don't pay close attention to Vim development to be able to tell what changed. Generally, features and important bug fixes should be included. It's also useful to mention most newly included options, commands, functions, color schemes, and bundled plugins.

The list of new options, commands, and functions can be obtained by diff'ing runtime/doc/quickref.txt (under option-list), runtime/doc/index.txt (under ex-cmd-index) and runtime/doc/usr_41.txt (under function-list). New color schemes can be found in runtime/colors. New bundled plugins can be found by looking at runtime/pack/dist/opt.

Long term, MacVim should try to minimize the change from Vim, to make each merge easier and less error-prone. As of this writing (2020-01-14) there are ~2000 lines of diff for all of Vim's C code (src/*.c). Possible ways to improve the situation:

• Some of the changes are because Vim's macOS support is a little lacking and MacVim added macOS-specific features. Integrate some of MacVim's changes back upstream to avoid having to maintain them locally on MacVim's side.
• Refactor all the FEAT_GUI_MACVIM code blocks to be more merge friendly by moving them to individual lines.
• If possible, avoid changes in core Vim and prefer doing them in src/MacVim.

Suggestions are welcome for improvements to make the code easier to merge and maintain.