Description

altendky

opened on Dec 24, 2021

Issue body actions

Creating and binding a socket outside of a call to memory_profiler.memory_usage() causes it to not close properly inside the call. After calling .close() the port is still claimed and a new socket is unable to be bound to the same port. Without memory_profiler in the middle it works. I haven't gotten to digging into this yet but will share anything I find.

Expand the collapsed sections at the bottom for a full set of commands to recreate and a full terminal session.

For linkage, I initially reported this at CFMTech/pytest-monitor#53.

$ venv/bin/python x.py
Traceback (most recent call last):
  File "/home/altendky/repos/chia-blockchain/tmp/x.py", line 29, in <module>
    main()
  File "/home/altendky/repos/chia-blockchain/tmp/x.py", line 18, in main
    memory_profiler.memory_usage((profiled, [sock3]))
  File "/home/altendky/repos/chia-blockchain/tmp/venv/lib/python3.9/site-packages/memory_profiler.py", line 377, in memory_usage
    returned = f(*args, **kw)
  File "/home/altendky/repos/chia-blockchain/tmp/x.py", line 25, in profiled
    sock2.bind(address)
OSError: [Errno 98] Address already in use

import socket

import memory_profiler


address = ("127.0.0.1", 33125)


def main():
    sock = socket.socket()
    sock.bind(address)

    profiled(sock)

    sock3 = socket.socket()
    sock3.bind(address)

    memory_profiler.memory_usage((profiled, [sock3]))


def profiled(sock):
    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)
    sock2.close()


main()

commands

cat > x.py << EOF
import socket

import memory_profiler


address = ("127.0.0.1", 33125)


def main():
    sock = socket.socket()
    sock.bind(address)

    profiled(sock)

    sock3 = socket.socket()
    sock3.bind(address)

    memory_profiler.memory_usage((profiled, [sock3]))


def profiled(sock):
    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)
    sock2.close()


main()
EOF
cat x.py
python3.9 -m venv venv
venv/bin/python -m pip install --upgrade pip setuptools wheel
venv/bin/pip install memory-profiler==0.60.0 psutil==5.8.0
venv/bin/python x.py
venv/bin/python --version --version
venv/bin/pip freeze
uname -a
lsb_release -a

full console session

$ cat > x.py << EOF
> import socket
> 
> import memory_profiler
> 
> 
> address = ("127.0.0.1", 33125)
> 
> 
> def main():
>     sock = socket.socket()
>     sock.bind(address)
> 
>     profiled(sock)
> 
>     sock3 = socket.socket()
>     sock3.bind(address)
> 
>     memory_profiler.memory_usage((profiled, [sock3]))
> 
> 
> def profiled(sock):
>     sock.close()
> 
>     sock2 = socket.socket()
>     sock2.bind(address)
>     sock2.close()
> 
> 
> main()
> EOF

$ cat x.py
import socket

import memory_profiler


address = ("127.0.0.1", 33125)


def main():
    sock = socket.socket()
    sock.bind(address)

    profiled(sock)

    sock3 = socket.socket()
    sock3.bind(address)

    memory_profiler.memory_usage((profiled, [sock3]))


def profiled(sock):
    sock.close()

    sock2 = socket.socket()
    sock2.bind(address)
    sock2.close()


main()

$ python3.9 -m venv venv

$ venv/bin/python -m pip install --upgrade pip setuptools wheel
Requirement already satisfied: pip in ./venv/lib/python3.9/site-packages (21.1.1)
Collecting pip
  Using cached pip-21.3.1-py3-none-any.whl (1.7 MB)
Requirement already satisfied: setuptools in ./venv/lib/python3.9/site-packages (56.0.0)
Collecting setuptools
  Using cached setuptools-60.1.0-py3-none-any.whl (952 kB)
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: setuptools
    Found existing installation: setuptools 56.0.0
    Uninstalling setuptools-56.0.0:
      Successfully uninstalled setuptools-56.0.0
  Attempting uninstall: pip
    Found existing installation: pip 21.1.1
    Uninstalling pip-21.1.1:
      Successfully uninstalled pip-21.1.1
Successfully installed pip-21.3.1 setuptools-60.1.0 wheel-0.37.1

$ venv/bin/pip install memory-profiler==0.60.0 psutil==5.8.0
Collecting memory-profiler==0.60.0
  Using cached memory_profiler-0.60.0-py3-none-any.whl
Collecting psutil==5.8.0
  Using cached psutil-5.8.0-cp39-cp39-manylinux2010_x86_64.whl (293 kB)
Installing collected packages: psutil, memory-profiler
Successfully installed memory-profiler-0.60.0 psutil-5.8.0

$ venv/bin/python x.py
Traceback (most recent call last):
  File "/home/altendky/repos/chia-blockchain/tmp/x.py", line 29, in <module>
    main()
  File "/home/altendky/repos/chia-blockchain/tmp/x.py", line 18, in main
    memory_profiler.memory_usage((profiled, [sock3]))
  File "/home/altendky/repos/chia-blockchain/tmp/venv/lib/python3.9/site-packages/memory_profiler.py", line 377, in memory_usage
    returned = f(*args, **kw)
  File "/home/altendky/repos/chia-blockchain/tmp/x.py", line 25, in profiled
    sock2.bind(address)
OSError: [Errno 98] Address already in use

$ venv/bin/python --version --version
Python 3.9.5 (default, Jun  3 2021, 15:18:23) 
[GCC 9.3.0]

$ venv/bin/pip freeze
memory-profiler==0.60.0
psutil==5.8.0

$ uname -a
Linux p1 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal

Activity

1. altendky commented on Dec 25, 2021

   

   altendky

   on Dec 25, 2021

   CollaboratorAuthor

   More actions

   Mmm... multiprocessing is being used with the default start method of fork with which "All resources of the parent are inherited by the child process". Explicitly specifying spawn instead with multiprocessing.set_start_method("spawn") works. spawn is described in part with "unnecessary file descriptors and handles from the parent process will not be inherited".

   new commands that work

   cat > x.py << EOF
   import multiprocessing
   import socket
   
   import memory_profiler
   
   
   address = ("127.0.0.1", 33125)
   
   
   def main():
       multiprocessing.set_start_method("spawn")
   
       sock = socket.socket()
       sock.bind(address)
   
       profiled(sock)
   
       sock3 = socket.socket()
       sock3.bind(address)
   
       memory_profiler.memory_usage((profiled, [sock3]))
       print("really made it")
   
   
   def profiled(sock):
       sock.close()
   
       sock2 = socket.socket()
       sock2.bind(address)
       sock2.close()
   
   
   # auaughhghgh  multiprocessing!
   if __name__ == "__main__":
       main()
   EOF
   cat x.py
   python3.9 -m venv venv
   venv/bin/python -m pip install --upgrade pip setuptools wheel
   venv/bin/pip install memory-profiler==0.60.0 psutil==5.8.0
   venv/bin/python x.py
   venv/bin/python --version --version
   venv/bin/pip freeze
   uname -a
   lsb_release -a
   

   new working terminal session

   $ cat > x.py << EOF
   > import multiprocessing
   > import socket
   > 
   > import memory_profiler
   > 
   > 
   > address = ("127.0.0.1", 33125)
   > 
   > 
   > def main():
   >     multiprocessing.set_start_method("spawn")
   > 
   >     sock = socket.socket()
   >     sock.bind(address)
   > 
   >     profiled(sock)
   > 
   >     sock3 = socket.socket()
   >     sock3.bind(address)
   > 
   >     memory_profiler.memory_usage((profiled, [sock3]))
   >     print("really made it")
   > 
   > 
   > def profiled(sock):
   >     sock.close()
   > 
   >     sock2 = socket.socket()
   >     sock2.bind(address)
   >     sock2.close()
   > 
   > 
   > # auaughhghgh  multiprocessing!
   > if __name__ == "__main__":
   >     main()
   > EOF

   $ cat x.py
   import multiprocessing
   import socket
   
   import memory_profiler
   
   
   address = ("127.0.0.1", 33125)
   
   
   def main():
       multiprocessing.set_start_method("spawn")
   
       sock = socket.socket()
       sock.bind(address)
   
       profiled(sock)
   
       sock3 = socket.socket()
       sock3.bind(address)
   
       memory_profiler.memory_usage((profiled, [sock3]))
       print("really made it")
   
   
   def profiled(sock):
       sock.close()
   
       sock2 = socket.socket()
       sock2.bind(address)
       sock2.close()
   
   
   # auaughhghgh  multiprocessing!
   if __name__ == "__main__":
       main()

   $ python3.9 -m venv venv

   $ venv/bin/python -m pip install --upgrade pip setuptools wheel
   Requirement already satisfied: pip in ./venv/lib/python3.9/site-packages (21.1.1)
   Collecting pip
     Using cached pip-21.3.1-py3-none-any.whl (1.7 MB)
   Requirement already satisfied: setuptools in ./venv/lib/python3.9/site-packages (56.0.0)
   Collecting setuptools
     Using cached setuptools-60.1.0-py3-none-any.whl (952 kB)
   Collecting wheel
     Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
   Installing collected packages: wheel, setuptools, pip
     Attempting uninstall: setuptools
       Found existing installation: setuptools 56.0.0
       Uninstalling setuptools-56.0.0:
         Successfully uninstalled setuptools-56.0.0
     Attempting uninstall: pip
       Found existing installation: pip 21.1.1
       Uninstalling pip-21.1.1:
         Successfully uninstalled pip-21.1.1
   Successfully installed pip-21.3.1 setuptools-60.1.0 wheel-0.37.1

   $ venv/bin/pip install memory-profiler==0.60.0 psutil==5.8.0
   Collecting memory-profiler==0.60.0
     Using cached memory_profiler-0.60.0-py3-none-any.whl
   Collecting psutil==5.8.0
     Using cached psutil-5.8.0-cp39-cp39-manylinux2010_x86_64.whl (293 kB)
   Installing collected packages: psutil, memory-profiler
   Successfully installed memory-profiler-0.60.0 psutil-5.8.0

   $ venv/bin/python x.py
   really made it

   $ venv/bin/python --version --version
   Python 3.9.5 (default, Jun  3 2021, 15:18:23) 
   [GCC 9.3.0]

   $ venv/bin/pip freeze
   memory-profiler==0.60.0
   psutil==5.8.0

   $ uname -a
   Linux p1 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

   $ lsb_release -a
   No LSB modules are available.
   Distributor ID: Ubuntu
   Description:    Ubuntu 20.04.3 LTS
   Release:        20.04
   Codename:       focal

   Ok, so with the cause identified, what next? I'm not exactly sure. Probably some documentation commenting on this with references to each of multiprocessing, spawn, fork, inherit-ed/ing, file descriptors, etc so it is findable by searching. I hesitate to mess with the multiprocessing global state and auto-configure this since code using memory-profiler may choose something else. If we do set the start method then it should only happen if not otherwise configured by the 'outer' code. I'll think it over and see what I can come up with.
2. altendky commented on Dec 29, 2021

   

   altendky

   on Dec 29, 2021

   CollaboratorAuthor

   More actions

   Are multiprocessing features really being used? Or could we just use a regular subprocess instead?

Issue actions