GeezerVison

Thank you Hugo! do you have comments on the content itself, or parts that were not clear enough?

Thank you Hugo! do you have comments on the content itself, or parts that were not clear enough?

Content looks okay to me, but let's wait for someone who is more familiar with adding extension modules to CPython to also review.

By the way, we don't need to add gh- prefixes here to PR titles, they're only needed on the CPython for Bedevere to link issues to PRs and backports.

Yes, that's what I observed afterwards. I thought that maybe there would be Bedevere that would update the issue automatically but in the end I needed to add the "Fix #..." myself.

I was reading what I wrote and I forgot something about the configure script and bootstrapping so I'll update the tutorial.

EDIT: won't have time today anymore!
EDIT 2: converting to a draft until I resolved my own thinking
EDIT 3: updated

OTOH, the only c prefix I can find is cmathmodule.c.

And that c stands for "complex", not "implemented in C". Best to continue the underscore-prefixed convention (new C modules will typically be accelerators for pure Python modules anyway).

And that c stands for "complex", not "implemented in C".

Oh, TIL! I never looked in that corner of the code base (yet) :)

I don't know a lot about the limited API and I think the tutorial should not necessarily have too much things as an introduction.

It may make sense to separate out some concepts, such as the limited API, into how-to guides instead of baking them into the tutorial.

I don't know a lot about the limited API and I think the tutorial should not necessarily have too much things as an introduction.

@vstinner has been converting modules to limited API, maybe he's a better person to argue that it should be in the initial example, as a default to start with :)
For starters, it makes the module friendlier to non-CPython implementations, as the limited API is (nowadays) designed to avoid CPython-specific details.
Removing the define gives you access to more API.

Py_BUILD_CORE_MODULE is kind of the opposite: gives you access to internals, which might change frequently. That coupling makes both the module and the internals harder to maintain, but you get more power and speed.

Thank you for the explanation on the Limited API! IIUC:

• With Limited API + #include "Python.h" -> only expose a partial set of the API
• only #include "Python.h" -> do not expose internals
• Py_BUILD_CORE_MODULE + #include "Python.h" -> expose internals

Ideally, I don't want to duplicate the explanation on the macros themselves (I don't remember where it is) but I can a small paragraph summarizing the above? What remains to add on the page is (in separate PRs):

• configuration variables (without explaining how to use autoconf syntax, I'll just link a reference to the autoconf page). Typically useful when adding stuff related to maths.
• frozen modules, but I don't really know about them so I'll leave it to someone else (I didn't find any definition of them...)

The limited API is documented publicly, but Py_BUILD_CORE_MODULE is internal. I think this would be a good place to mention it, since it's only useful for core modules. But, it doesn't need much more than that bullet point.

I don't really want to add it in the main section because it could disrupt the reading flow.

So I think I can add something in the Troubleshooting section when the compiler complains with Py_BUILD_CORE being missing when you include some headers for some functions (and you don't know that you actually needed that macro). That way, people wouldn't add the Py_BUILD_CORE macro directly (the compiler message is literal "this requires Py_BUILD_CORE and I think it's a bit misleading since it does not need Py_BUILD_CORE itself but rather Py_BUILD_CORE_MODULE).

Actually, there is this page: https://docs.python.org/3/using/configure.html#c-extensions which explains somewhat those macros. Should it be moved to the devguide? it also explains the difference with the Py_BUILD_CORE_BUILTIN macro.

(By the way, I don't like the naming "module" and "built-in" and I was confused a lot at first! I would have preferred something like "static" and "shared" because it's much more explanatory... but I think we cannot change this =/ (unless there are some other conditions I am not aware of)).

Yes, I think mentioning Py_BUILD_CORE_BUILTIN in the user-facing documentation is a mistake. Users could read that page and think that they can define Py_BUILD_CORE_MODULE for their own dynamic libraries.
@vstinner, would you agree with moving the mentions of Py_BUILD_CORE* macros from configure docs to the devguide?

I don't like the naming "module" and "built-in" and I was confused a lot at first! I would have preferred something like "static" and "shared" because it's much more explanatory...

Maybe some of the confusion remains?
Most core extension modules can be either linked statically or be a shared library. The person who builds Python, not the one that writes the module code, makes that choice.

IMO, the distinction isn't all that important, and it could moved all the way down to “Configuring the CPython project”, with Setup.stdlib.in treated as the normal and Setup.bootstrap.in as a less important special case.

The reasons for putting a module Modules/Setup.bootstrap.in are:

• the module defines C APIs that are used in core
• the module is used by importlib, deepfreeze, freeze, runpy, or sysconfig
• it's a “commonly used core module”

If you're adding your first module, these aren't very likely to apply.

A similar special case, then, is modules that must be built as shared libraries -- that's mostly modules that test the loading of shared libraries.

@vstinner, would you agree with moving the mentions of Py_BUILD_CORE* macros from configure docs to the devguide?

I would prefer to keep the doc in the configure docs. You can duplicate the doc in the devguide if you want.

Maybe some of the confusion remains?

No.... the confusing is because I'm tired and I forgot about the *shared* and *static* markers... (I even wrote a section about them!!)

I'll add the paragraph on the macros. I'm waiting for python/cpython#122544 to be decided because the troubleshooting section will need to be updated (and I'm taking holidays next week so it's better to wait for the correct image to be chosen before merging (it saves us 1 PR if we merge it before)).