GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Some builtin single phase modules do get loaded into subinterpreters which do not support them · Issue #119450 · python/cpython

Some builtin single phase modules do get loaded into subinterpreters which do not support them #119450

New issue
Copy link
New issue
Copy link
Closed
Closed
Some builtin single phase modules do get loaded into subinterpreters which do not support them#119450
Copy link
Labels
topic-subinterpreterstype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@sebsura
sebsura
opened on May 23, 2024
Issue body actions

Bug description:

If a subinterpreter is started with

PyInterpreterConfig cfg = {
...
    .check_multi_interp_extensions = 1,
};
Py_NewInterpreterFromConfig(&ts, &cfg);

I expect that no single phase module is allowed to be loaded. This works for all self build modules and with most builtin ones, but not all. Some modules i found to be misbehaving are e.g. datetime, tracemalloc.

This can easily cause a crash:

#include <Python.h>
#include <assert.h>

// make subinterpreter from config
PyThreadState* make_sub(PyInterpreterConfig* config)
{
  PyThreadState* sub = NULL;
  PyStatus status = Py_NewInterpreterFromConfig(&sub, config);

  if (PyStatus_Exception(status)) {
    return NULL;
  }

  return sub;
}

// create an isolated subinterpreter, then load a module with name 'name'
void sub_import(PyThreadState* main, const char* name)
{
  PyEval_AcquireThread(main);

  PyInterpreterConfig config = {
      .use_main_obmalloc = 0,
      .allow_fork = 0,
      .allow_exec = 0,
      .allow_threads = 1,
      .allow_daemon_threads = 0,
      .check_multi_interp_extensions = 1,
      .gil = PyInterpreterConfig_OWN_GIL,
  };

  PyThreadState* sub = make_sub(&config);
  assert(sub);

  PyObject* module = PyImport_ImportModule(name);
  
  printf("Loaded: %s\n", module ? "yes" : "no");

  Py_XDECREF(module);

  Py_EndInterpreter(sub);
}

int main()
{
  Py_InitializeEx(0);

  PyThreadState* interp = PyEval_SaveThread();

  const char* module_name = "datetime";

  sub_import(interp, module_name);
  sub_import(interp, module_name); // <- crash

  PyEval_AcquireThread(interp);
  Py_FinalizeEx();
}

This is because the module will get loaded a second time, which will cause python to try and cleanup the old state from the old interpreter inside the new interpreter.

// obmalloc.c
void
_PyObject_Free(void *ctx, void *p) // p will point to an object allocated by pymalloc from the old interpreter here
{
    /* PyObject_Free(NULL) has no effect */
    if (p == NULL) {
        return;
    }

    OMState *state = get_state();
    if (UNLIKELY(!pymalloc_free(state, ctx, p))) { // <- this returns false, since the object does not belong to the new interpreter
        /* pymalloc didn't allocate this address */
        PyMem_RawFree(p); // <- this will crash, since the pointer was not allocated directly through malloc
        raw_allocated_blocks--;
    }
}

This may be related to #117953 and #104621 but its important to note that this does not happen with the readline module as it correctly refuses to load.

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Activity

  1. sebsura
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    on May 23, 2024
  2. Eclips4
    added
    topic-subinterpreters
    on May 23, 2024
  3. github-project-automation
    added this to Subinterpreterson May 23, 2024
  4. github-project-automation
    moved this to Todo in Subinterpreterson May 23, 2024
  5. sebsura commented on May 23, 2024

    @sebsura
    sebsura
    on May 23, 2024
    Author
    More actions

    The deallocation is caused by this:

    // _tetimemodule.c:6755
    static int
    _datetime_exec(PyObject* module)
    {
        ...
        PyObject *d = PyDateTime_DeltaType.tp_dict; // <- this is a global variable
        DATETIME_ADD_MACRO(d, "resolution", new_delta(0, 0, 1, 0));
  6. ericsnowcurrently commented on Jul 8, 2024

    @ericsnowcurrently
    ericsnowcurrently
    on Jul 8, 2024
    Member
    More actions

    This should be resolved now for 3.13+, as of gh-121060. Sadly, the fix is a bit too complex to backport to 3.12.

  7. ericsnowcurrently
    added
    pendingThe issue will be closed if no feedback is providedThe issue will be closed if no feedback is provided
    on Jul 8, 2024
  8. sebsura commented on Jul 9, 2024

    @sebsura
    sebsura
    on Jul 9, 2024
    Author
    More actions

    Thanks for the heads up. I can now run the sample code i gave above without crashing!

  9. sebsura
    closed this as completedon Jul 9, 2024
  10. github-project-automation
    moved this from Todo to Done in Subinterpreterson Jul 9, 2024
  11. erlend-aasland
    removed
    pendingThe issue will be closed if no feedback is providedThe issue will be closed if no feedback is provided
    on Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic-subinterpreterstype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

    Projects

    • Subinterpreters
      Status
      Done
      Show more project fields

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402503 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.