Commit Graph

101 Commits

Author SHA1 Message Date
Lewis Russell
688b961d13 feat(treesitter): add support for wasm parsers
Problem: Installing treesitter parser is hard (harder than
climbing to heaven).

Solution: Add optional support for wasm parsers with `wasmtime`.

Notes:

* Needs to be enabled by setting `ENABLE_WASMTIME` for tree-sitter and
  Neovim. Build with
  `make CMAKE_EXTRA_FLAGS=-DENABLE_WASMTIME=ON
  DEPS_CMAKE_FLAGS=-DENABLE_WASMTIME=ON`
* Adds optional Rust (obviously) and C11 dependencies.
* Wasmtime comes with a lot of features that can negatively affect
  Neovim performance due to library and symbol table size. Make sure to
  build with minimal features and full LTO.
* To reduce re-compilation times, install `sccache` and build with
  `RUSTC_WRAPPER=<path/to/sccache> make ...`
2024-08-26 16:44:03 +02:00
Christian Clason
fa79a8ad6d build(deps): vendor libvterm at v0.3.3
Problem: Adding support for modern Nvim features (reflow, OSC 8, full
utf8/emoji support) requires coupling libvterm to Nvim internals
(e.g., utf8proc).

Solution: Vendor libvterm at v0.3.3.
2024-08-10 10:26:07 +02:00
bfredl
1247684ae1 build(deps): remove msgpack-c dependency 2024-08-05 12:22:12 +02:00
Christian Clason
51d85f7ea5
build(deps): drop unused bundled bash, python parsers and queries
Problem: Neovim bundles treesitter parsers for bash and python but does
not use them by default. This dilutes the messaging about the bundled
parsers being required for functionality or reasonable out-of-the-box
experience. It also increases the risk of query incompatibilities for no
gain.

Solution: Stop bundling bash and python parser and queries.
2024-07-09 15:26:48 +02:00
dundargoc
32e16cb0b6 build: add utf8proc as dependency
utf8proc contains all the data which is currently in
unicode_tables.generated.h internally, but in quite a different format.
Ideally unicode_tables.generated.h should be removed as well so we rely
solely on utf8proc. We want to avoid a situation where the possibility
of unicode mismatch occurs, e.g a distro using both unicode 12 and
unicode 13.
2024-06-28 19:47:39 +02:00
dundargoc
3d3717de4e build: use libvterm commit with generated *.inc files
This allows us to skip generating them during our build process.
2024-05-05 21:38:36 +02:00
dundargoc
383018dbb8 build: simplify policy-setting for dependencies
Passing `CMAKE_POLICY_DEFAULT_CMP0092=NEW` to all dependencies is
simpler than setting it manually in each file.
2024-05-03 18:22:20 +02:00
dundargoc
a89ce89742
docs: fix typos (#27868)
Co-authored-by: ite-usagi <77563904+ite-usagi@users.noreply.github.com>
Co-authored-by: v-sim <56476039+v-sim@users.noreply.github.com>
Co-authored-by: Evgeni Chasnovski <evgeni.chasnovski@gmail.com>
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
Co-authored-by: Quico Augustijn <quico.public@gmail.com>
Co-authored-by: nhld <nahnera@gmail.com>
Co-authored-by: francisco souza <108725+fsouza@users.noreply.github.com>
2024-03-28 09:32:32 +08:00
dundargoc
036f86feac ci: provide separate macos releases for intel and arm
This will immensely reduce the complexity required to support both
architectures, reduce overall lines of code and unblock follow-up
simplifications.
2024-03-24 20:58:30 +01:00
dundargoc
2c1e8f7e96 build: use GIT_REPOSITORY for local URLs
`GIT_REPOSITORY` will cause cmake to rebuild if local dependency
changes, which isn't the case for `URL`.

Also document how to test a different commits of a dependency.
2024-03-23 13:36:12 +01:00
dundargoc
187ae67735 build: introduce variable DEPS_IGNORE_SHA for skipping dependency hash check
This will reduce friction as developers no longer need to provide a hash
when testing out different commits.

To skip the hash check, set `DEPS_IGNORE_SHA` to `TRUE` in
`cmake.deps/CMakeLists.txt`.
2024-03-19 08:52:03 +01:00
dundargoc
4788abf2da build: stop installing parser.h from treesitter 2024-02-09 12:08:25 +01:00
dundargoc
ae3eed53d6 build: various build improvements
- remove "ran-" prefix from touch files as it's redundant since the
  they're already in the directory named `touches`.
- Include `contrib` when formatting with `make formatlua`.
- Use TARGET_FILE generator expression instead of assuming the
  executable location.
- reuse logic that determines whether to use lua or luajit.
- add translations to the `nvim` target.

Makefile improvements:
- rename variable `CMAKE_PRG` to `CMAKE` to make it more consistent with
  the builtin `MAKE` variable.
- stop propagating flags to generator. Users should use cmake for
  non-standard use cases.
- remove `+` prefix from targets. If the user for whatever reason wants
  to dry-run a target then they should be able to.
2024-01-24 12:41:09 +01:00
dundargoc
404fdb0f36 build: cmake fixes
- add EXTERNALPROJECT_OPTIONS variable to main build
- use `REQUIRED` keyword for IWYU.
- remove check_c_compiler_flag checks when `ENABLE_COMPILER_SUGGESTIONS`
  is `ON`. If we explicitly enable it then we probably want it to give
  an error if it doesn't exist, rather than silently skip it.
- Move dependency interface libraries to their find module and use them
  as a pseudo-imported target.
- Remove BUSTED_OUTPUT_TYPE. It's not used and we can reintroduce it
  again if something similar is needed.
- Use LINK_OPTIONS intead of LINK_FLAGS when generating the `--version`
  output.
2023-12-16 21:06:28 +01:00
dundargoc
7840760776 build: bump minimum cmake version to 3.13
The benefits are primarily being able to use FetchContent, which allows
for a more flexible dependency handling. Other various quality-of-life
features such as `-B` and `-S` flags are also included.

This also removes broken `--version` generation as it does not work for
version 3.10 and 3.11 due to the `JOIN` generator expression.

Reference: https://github.com/neovim/neovim/issues/24004
2023-12-16 17:17:24 +01:00
dundargoc
896b400bff build: fix universal mac builds
Cmake 3.28+ will fail if two projects download the same file to prevent
scheduling problems. This can be circumvented by downloading luajit to a
unique location for each target. This is theoretically non-optimal since
we need to download the same file three times for universal builds, but
universal builds are rare and the convenience of this method outweighs
setting up the dependencies optimally. This fixes the currently broken
release workflow for mac.

Closes https://github.com/neovim/neovim/issues/26526.
2023-12-16 16:47:22 +01:00
dundargoc
404043e74c build: vendor libtermkey
This is a proof of concept/WIP to evaluate the viability of vendoring
libtermkey as it's been deprecated.
2023-11-30 12:02:10 -06:00
dundargoc
99b8a343e1 fixup: quick update, squash later 2023-11-20 15:21:55 +01:00
dundargoc
7a80e169c5 build: disable all compiler warnings from dependencies 2023-11-20 15:21:55 +01:00
dundargoc
bec2ebebda build: various cmake fixes
- Correct MSVC warning suppression. The C4003 warning is issued during
  file generation and not for the actual source files.
- Remove non-existent "scripts/pvscheck.sh" file from `lintsh` target.
- Remove spaces inside for loops with uncrustify.
- Point dependencies to use a git tag rather than releases, as releases
  might have changes that deviate from the actual source code.
- Automatically update uncrustify config before formatting or linting.
2023-11-18 18:38:45 +01:00
dundargoc
5cefec7349 build: various cmake fixes
- silence false warnings on MSVC
- merge `clang-tidy` cmake target into `lintc` and remove the
  corresponding make target
- use cmake's built-in endianness detection
2023-11-04 18:32:47 +01:00
dundargoc
8405649f92 build: use neovim/libvterm instead of neovim/deps for libvterm
Using a mirror makes it easier to test patches as well as reducing
maintenance when a new version is released.
2023-10-29 12:14:19 +01:00
dundargoc
4f526b9fd8 build: use neovim/libtermkey instead of neovim/deps for libtermkey
Using a mirror makes it easier to test patches as well as reducing
maintenance when a new version is released.
2023-10-28 22:32:52 +02:00
dundargoc
b80a8e2c16 build: adjust how find order is prioritized
Ensure bundled libraries and include directories are always searched
first before any others. This will provide a more consistent experience
as the search order of the builtin find_ functions can vary depending on
system. This should make the build process faster when building with
bundled deps as we limit the search to only the .deps directory.
Separating the search between .deps and everything makes debugging
find_-related problems simpler if you need to check how dependencies are
found.

For libraries, we divide the search process into the following order:
1. Only search in .deps directory and only search for static libraries.
2. Only search in .deps directory and search for all libraries.
3. Search everywhere and search for all libraries.

Make an exception for FindLibintl.cmake as changing the search order
seems to break some tests on macos.
2023-10-16 20:15:41 +02:00
dundargoc
c50951a4d0
build: various fixes
- simplify lua interpreter search
- fix incorrect variable name in BuildLua.cmake
- build PUC Lua with -O2
- silence non-mandatory find_package search for libuv
- simplify Find modules
- Prefer using the explicitly set CI_BUILD over relying on the
  environment variable "CI".
2023-09-04 00:00:26 +02:00
Christian Clason
3e80b39a8e build(deps): bump luajit to HEAD - 03c31124c
Switch to a rolling release, so the `0-beta3` suffix is dropped in favor
of the date. Remove the custom UNIX command as the symlink is now
created by the LuaJIT Makefile.
2023-08-23 11:23:11 +09:00
Christian Clason
cb0a1a10b2 feat(treesitter): add bash parser and queries 2023-07-01 11:28:32 +02:00
Christian Clason
88c8803aa1 feat(treesitter): add python parser and queries 2023-07-01 11:28:32 +02:00
Christian Clason
11844dde81
feat(treesitter): bundle markdown parser and queries (#22481)
* bundle split Markdown parser from https://github.com/MDeiml/tree-sitter-markdown
* add queries from https://github.com/nvim-treesitter/nvim-treesitter/tree/main
* upstream `#trim!` and `#inject-language!` directives

Co-authored-by: dundargoc <gocdundar@gmail.com>
2023-07-01 11:08:06 +02:00
dundargoc
8b8e607284
build: move luarocks and rocks installation to main build
This will ensure luacheck and busted are only installed when they're
actually needed. This cuts total build time by over 50%.

Closes https://github.com/neovim/neovim/issues/22797.
2023-05-21 20:57:39 +02:00
dundargoc
1086016477
build: cmake cleanup
- Simplify error checking when using execute_process.
- Set BUILD_SHARED_LIBS to OFF when building dependencies.
  This is normally not needed, but msgpack interprets an unset
  BUILD_SHARED_LIBS to build a shared library, which is the opposite of
  the cmake behavior.
- Move function check_lua_module to Util.cmake.
- Remove unnecessary code.
- Make variable naming more consistent
2023-05-13 12:12:29 +02:00
dundargoc
82bb7bbc48
build: add luajit runtime files when installing (#23514)
Closes https://github.com/neovim/neovim/issues/15543.
2023-05-09 17:25:49 +02:00
dundargoc
88366641ad
build: remove USE_BUNDLED_BUSTED option
The previous logic made it possible to install bundled luarocks, but
also use external rocks, making the luarocks installation unnecessary.
Instead, let's assume that if the user wants to use the bundled
luarocks, then they also want to use it to install necessary rocks.
2023-05-08 17:24:19 +02:00
Enan Ajmain
1bf29a0ae1
build: add system lua include dir for lpeg
Problem: Lpeg requires Lua headers.  Currently the include directories
for Lpeg are set only to the bundled deps folder, so if the user wants
to use system Lua/Luajit, Lpeg will not find the system headers and will
fail to build.

Solution: Add system Lua/Luajit include directories when USE_BUNDLED_LUA
and USE_BUNDLED_LUAJIT are turned off.

Fixes #23469
2023-05-07 00:47:45 +02:00
bfredl
45bcf83869 refactor(build): include lpeg as a library 2023-04-27 11:40:00 +02:00
bfredl
1e60e8c040 refactor(build): use vendored versions of mpack and luabitop 2023-04-19 10:44:25 +02:00
dundargoc
c8667c8756
build: various cmake fixes
- Remove unused function argument from GetBinaryDeps
- Remove unused variable LUA_LOAD_PACKAGE_MODULE_SOURCE
- Add LUA_FS_MODULE_SOURCE as a dependency of VIM_MODULE_FILE
2023-04-15 22:39:30 +02:00
dundargoc
5a7280ba68
build: create helper function for simplifying luarocks installation
The function keeps track of the previously installed rock, meaning we no
longer need to manually keep track of the dependency chain. This will
make adding or removing rocks much easier.
2023-04-12 21:26:06 +02:00
dundargoc
2612930a09
build(Windows): allow building without custom md5sum
Follow-up to eb1da498d6. The workaround in
that case only works if md5sum is in users path. We work around this by
adding the directory with the md5sum shipped with luarocks to PATH.

Co-authored-by: erw7 <erw7.github@gmail.com>
2023-04-05 19:49:16 +02:00
dundargoc
81f2bce775
build: cmake cleanup
- Change libtermkeyCMakeLists.txt to LibtermkeyCMakeLists.txt
- Remove duplicate mark_as_advanced calls in FindLibuv.cmake
- Fix "Enabling Clang sanitizer" messages as it's no longer clang-only
- Simplify parser installation syntax
- Rename tree-sitter to treesitter
2023-04-04 19:27:21 +02:00
dundargoc
eb1da498d6
build(windows): work around luarocks not finding its own md5sum
Luarocks is unable to find its own md5sum due to these reasons listed in
the comment https://github.com/luarocks/luarocks/issues/1443.

The pull request https://github.com/luarocks/luarocks/pull/1498 resolves
this issue, but in the meantime we can work around it by resetting the
value of MD5sum to "md5sum".
2023-04-03 21:12:42 +02:00
Christian Clason
d7f7450017 refactor(treesitter)!: rename help parser to vimdoc 2023-04-01 15:07:16 +02:00
dundargoc
81abd5c5c7 build: download wintools executables separately
The wintools executables are stored in a zip file, making it
inconvenient to bump these during releases. Instead, unpack the
executables in the deps repository and download each executable
separately.
2023-03-28 16:18:13 +02:00
dundargoc
b155608bff
build: set CMAKE_C_STANDARD to 99 for all dependencies
Older gcc versions (4.x) require passing --std=c99 compiler flag to
prevent warnings from being emitted. Instead of setting it for each
dependency, it's easier to just pass the CMAKE_C_STANDARD flag to all
dependencies. This also prevents the scenario of us forgetting to set it
if we add new dependencies.
2023-03-24 10:22:15 +01:00
Justin M. Keyes
5726f33e8c
Merge #22691 build!: sanitizers for gcc 2023-03-19 13:42:25 -04:00
ii14
17ce634b8f build!: rename sanitizer options from CLANG_* to ENABLE_* 2023-03-17 03:40:57 +01:00
Christian Clason
a5cf62e81c build(deps): bump mpack to 1.0.10 2023-03-17 01:03:56 +01:00
Christian Clason
8408405cb3 build(deps): bump luacheck to 1.1.0-1
also ignore two new warnings showing false positives
2023-03-15 19:23:27 +01:00
Christian Clason
17ff65f59a build(deps): bump coxpcall to 1.17.0-1 2023-03-15 18:37:08 +01:00
red
64d3f68c07
build: fix build warning when using gcc 4.9.2
Problem:
When building with gcc 4.9.2, tree-sitter throws warning: "for loop initial
declarations are only allowed in C99 or C11 mode"

Solution:
set CMAKE_C_STANDARD option to 99
2023-03-09 16:46:34 +01:00