GeezerVison is showing github.com — Allow waiting on a mock · Issue #61215 · python/cpython
Allow waiting on a mock #61215Description
| BPO |
17013 |
| Nosy |
@jcea, @pitrou, @rbtcollins, @cjw296, @ezio-melotti, @voidspace, @Kentzo, @lisroach, @mariocj89, @pablogsal, @Kentzo, @tirkarthi |
| PRs |
[WIP] bpo-17013: Implement WaitableMock to create Mock objects that can wait until called #12818gh-61215: New mock to wait for multi-threaded events to happen #16094bpo-17013: Extend Mock.called to allow waiting for calls #17133gh-61215: Add Mock.call_event to allow waiting for calls #20759 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2013-01-22.12:15:47.799>
labels = ['type-feature', 'library', '3.9']
title = 'Allow waiting on a mock'
updated_at = <Date 2020-06-10.14:37:15.926>
user = 'https://github.com/pitrou' bugs.python.org fields:
activity = <Date 2020-06-10.14:37:15.926>
actor = 'Kentzo'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2013-01-22.12:15:47.799>
creator = 'pitrou'
dependencies = []
files = []
hgrepos = []
issue_num = 17013
keywords = ['patch']
message_count = 18.0
messages = ['180377', '181398', '247777', '340146', '340153', '340158', '340164', '352211', '356898', '367446', '371133', '371165', '371175', '371193', '371195', '371196', '371202', '371207']
nosy_count = 13.0
nosy_names = ['jcea', 'pitrou', 'rbcollins', 'cjw296', 'ezio.melotti', 'michael.foord', 'Ilya.Kulakov', 'lisroach', 'mariocj89', 'lkollar', 'pablogsal', 'Kentzo', 'xtreak']
pr_nums = ['12818', '16094', '17133', '20759']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue17013'
versions = ['Python 3.9'] Linked PRs
Activitypitrou commented on Jan 22, 2013 In non-trivial tests, you may want to wait for a method to be called in another thread. This is a case where unittest.mock currently doesn't help. It would be nice to be able to write:
myobj.some_method = Mock(side_effect=myobj.some_method)
# launch some thread
myobj.some_method.wait_until_called() And perhaps
myobj.some_method.wait_until_called_with(...) (with an optional timeout?)
If we don't want every Mock object to embed a threading.Event, perhaps there could be a ThreadedMock subclass?
Or perhaps even:
WaitableMock(..., event_class=threading.Event) so that people can pass multiprocessing.Event if they want to wait on the mock from another process? voidspace commented on Feb 5, 2013 rbtcollins commented on Jul 31, 2015 tirkarthi commented on Apr 13, 2019 Is there still sufficient interest in this? I gave an initial try at this based on my limited knowledge of mock and Antoine's idea. I created a new class WaitableMock that inherits from Mock and accepts an event_class with threading.Event as default and stores an event object. When call is made then via CallableMixin _mock_call is called which I have overridden to set the event object. I have wait_until_called that waits on this event object for a given timeout to return whether it's called or not.
I am not sure of implementing wait_until_called_with since I set the event object for any call in _mock_call irrespective of argument and since the call params for which the event has to be set are passed to wait_until_called_with. Perhaps allow passing a call object to wait_until_called_with and and during _mock_call set event object only if the call is the same as one passed to wait_until_called_with ?
See also bpo-26467 to support asyncio with mock
Initial implementation
class WaitableMock(Mock):
def __init__(self, *args, **kwargs):
event_class = kwargs.pop('event_class', threading.Event)
_safe_super(WaitableMock, self).__init__(*args, **kwargs)
self.event = event_class()
def _mock_call(self, *args, **kwargs):
_safe_super(WaitableMock, self)._mock_call(*args, **kwargs)
self.event.set()
def wait_until_called(self, timeout=1):
return self.event.wait(timeout=timeout)Sample program :
import multiprocessing
import threading
import time
from unittest.mock import WaitableMock, patch
def call_after_sometime(func, delay=1):
time.sleep(delay)
func()
def foo():
pass
def bar():
pass
with patch('__main__.foo', WaitableMock(event_class=threading.Event)):
with patch('__main__.bar', WaitableMock(event_class=threading.Event)):
threading.Thread(target=call_after_sometime, args=(foo, 1)).start()
threading.Thread(target=call_after_sometime, args=(bar, 5)).start()
print("foo called ", foo.wait_until_called(timeout=2)) # successful
print("bar called ", bar.wait_until_called(timeout=2)) # times out
foo.assert_called_once()
bar.assert_not_called()
# Wait for the bar's thread to complete to verify call is registered
time.sleep(5)
bar.assert_called_once()# foo is called but waiting for call to bar times out and hence no calls to bar are registered though bar is eventually called in the end and the call is registered at the end of the program.
➜ cpython git:(master) ✗ time ./python.exe ../backups/bpo17013_mock.py
foo called True
bar called False
./python.exe ../backups/bpo17013_mock.py 0.40s user 0.05s system 7% cpu 5.765 total mariocj89 commented on Apr 13, 2019 I think this is REALLY interesting!, there are many situations where this has been useful, it would greatly improve multithreading testing in Python.
Q? I see you inherit from Mock, should it inherit from MagicMock?
I'd say send the PR and the discussion can happen there about the implementation, seems there is consensus in this thread.
I would find it OK to start with wait_until_called if you want and can be discussed in another issue/PR if you don't have the time/think it might not be worth. tirkarthi commented on Apr 13, 2019 Thanks Mario for the feedback.
I see you inherit from Mock, should it inherit from MagicMock?
yes, it can inherit from MagicMock to mock magic methods and to wait on their call.
I thought some more about waiting for function call with arguments. One idea would be to have a dictionary with args to function as key mapping to an event object and to set and wait on that event object. To have wait_until_called that is always listening on a per mock object and to have wait_until_called_with listening on event object specific to that argument. Below is a sample implementation and an example to show it working with wait_until_called and wait_until_called_with. wait_until_called_with is something difficult to model since it needs per call event object and also need to store relevant key mapping to event object that is currently args and doesn't support keyword arguments. But wait_until_called is little simpler waiting on a per mock event object.
I will open up a PR with some tests.
class WaitableMock(MagicMock):
def __init__(self, *args, **kwargs):
_safe_super(WaitableMock, self).__init__(*args, **kwargs)
self.event_class = kwargs.pop('event_class', threading.Event)
self.event = self.event_class()
self.expected_calls = {}
def _mock_call(self, *args, **kwargs):
ret_value = _safe_super(WaitableMock, self)._mock_call(*args, **kwargs)
for call in self._mock_mock_calls:
event = self.expected_calls.get(call.args)
if event and not event.is_set():
event.set()
# Always set per mock event object to ensure the function is called for wait_until_called.
self.event.set()
return ret_value
def wait_until_called(self, timeout=1):
return self.event.wait(timeout=timeout)
def wait_until_called_with(self, *args, timeout=1):
# If there are args create a per argument list event object and if not wait for per mock event object.
if args:
if args not in self.expected_calls:
event = self.event_class()
self.expected_calls[args] = event
else:
event = self.expected_calls[args]
else:
event = self.event return event.is_set() or event.wait(timeout=timeout)
# Sample program to wait on arguments, magic methods and validating wraps
import multiprocessing
import threading
import time
from unittest.mock import WaitableMock, patch, call
def call_after_sometime(func, *args, delay=1):
time.sleep(delay)
func(*args)
def wraps(*args):
return 1
def foo(*args):
pass
def bar(*args):
pass
with patch('__main__.foo', WaitableMock(event_class=threading.Event, wraps=wraps)):
with patch('__main__.bar', WaitableMock(event_class=threading.Event)):
# Test normal call
threading.Thread(target=call_after_sometime, args=(foo, 1), kwargs={'delay': 1}).start()
threading.Thread(target=call_after_sometime, args=(bar, 1), kwargs={'delay': 5}).start()
print("foo called ", foo.wait_until_called(timeout=2))
print("bar called ", bar.wait_until_called(timeout=2))
foo.assert_called_once()
bar.assert_not_called() # Test wraps works
assert foo() == 1
# Test magic method
threading.Thread(target=call_after_sometime, args=(foo.__str__, ), kwargs={'delay': 1}).start()
print("foo.__str__ called ", foo.__str__.wait_until_called(timeout=2))
print("bar.__str__ called ", bar.__str__.wait_until_called(timeout=2))
foo.reset_mock()
bar.reset_mock() # Test waiting for arguments
threading.Thread(target=call_after_sometime, args=(bar, 1), kwargs={'delay': 1}).start()
threading.Thread(target=call_after_sometime, args=(bar, 2), kwargs={'delay': 5}).start()
print("bar called with 1 ", bar.wait_until_called_with(1, timeout=2))
print("bar called with 2 ", bar.wait_until_called_with(2, timeout=2))
time.sleep(5)
print("bar called with 2 ", bar.wait_until_called_with(2))
$ ./python.exe ../backups/bpo17013_mock.py
foo called True
bar called False
foo.__str__ called True
bar.__str__ called False
bar called with 1 True
bar called with 2 False
bar called with 2 True mariocj89 commented on Apr 13, 2019 Kooning great! I would Add a test for the following (I think both fails with the proposed impl):
- The mock is called BEFORE calling wait_until_called_with
- I call the mock with arguments and then I call wait for call without arguments.
- using keyword arguments
Also, I don’t have a great solution for it but it might be worth prefixing time-out with something in the wait_untill_called_with. In situations where the mocked object has a timeout parameter (which is a common argument for multithreaded apps). mariocj89 commented on Sep 12, 2019 Spoke offline with @XTreak, I'll be sending a PR for this to take over the existing one.
@lisroach proposed a new name, EventMock to differentiate it from any awaitable async notation.
@michael.foord suggested using mock_timeout as the argument.
Discussed also to change the naming of the method to await_until_any_call to make the semantics similar to the one that Mock provides. As await_until_called_with might mislead the user that it applies only to the last call. Kentzo commented on Nov 18, 2019 I have submitted an alternative implementation of this feature heavily inspired by _AwaitEvent I wrote for asynctest [0].
There was recently an interest from the community towards asynctest to the point that got some of its measures merged into CPython [1].
The key features of my implementation [2]:
- Gives meaning to the existing Mock.called property, otherwise not much useful
- Does not require end users to do anything: change is automatically available in every Mock (and any subclass of mock that does not override
called)
- Use of conditionals provides API that allows much richer extension: instead of hardcoding conditions as events it allows to wait until arbitrary predicate becomes true
- Utilizes existing semantics of python conditionals (both asyncio and threading)
Accepting this PR will allow me to bring _AwaitEvent thereby completing mock.py with waiting mechanics with identical semantics for both threading-based and asyncio-based cases.
0: https://github.com/Martiusweb/asynctest/blob/4b1284d6bab1ae90a6e8d88b882413ebbb7e5dce/asynctest/mock.py#L428
1: #9296
2: #17133 mariocj89 commented on Apr 27, 2020 For the record, I have no strong preference over either implementation. @voidspace preferred offline the new mock class, but sadly the rationale is lost in the physical word. Kentzo commented on Jun 10, 2020 As far as I understand it introduces 3 methods that may clash. It's unlikely but (I speculate)
is still more likely than an identity check with called.
That being said, the PR can be redone as a subclass. But that implementation will not play
as nicely with the planned awaited property for asyncio mocks (see description in the PR). Kentzo commented on Jun 10, 2020
That is not true, is actually encouraged to check for singletons like True, False and None.
You're right, just never used it as I never needed an identity check against True / False
The PR is re-done to use an additional property call_event instead of extending called for backwards compatibility. added a commit that references this issue on Jul 4, 2023gh-61215: Rename `wait_until_any_call` to `wait_until_any_call_with` (#… added a commit that references this issue on Jul 17, 2023gh-61215: threadingmock: Improve test suite to avoid race conditions (#… terryjreedy commented on Feb 24, 2025 The 5 attached PRs seem to have already gone beyond the original issue. Let's open a new issue for anything else.
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|