Spring 2009 (updated June-July 2018)

Validating Function Arguments with Decorators

As a final example of the utility of decorators, this section develops a function decorator that automatically tests whether arguments passed to a function or method are within a valid numeric range. It's designed to be used either during development or production, and can be used as a template for similar tasks (e.g., argument type testing, if you must). As we'll see, supporting arbitrary argument passing modes leads us down a variety of Python introspection and tool-building paths, and underscores subtleties in Python's argument matching modes.

A Basic Range-Testing Decorator for Positional Arguments

Let's start with a basic range test implementation. In the chapter "A More Realistic Example", we wrote a class that gave a raise to objects based upon a passed-in percentage [on this website, this example is available on this page]:

class Person:  
    ...
    def giveRaise(self, percent):
        self.pay = int(self.pay * (1 + percent))

There, we noted that we should probably check the percentage to make sure it's not too large or small, if we want the code to be robust. Although we could do this with an if statement in the method, it might be more useful and interesting to develop a more general tool that can perform range tests for us automatically, for the arguments of any function or method we might code now or in the future.

The following function decorator implements this idea. To keep this simple at first, it works only for positional arguments, and assumes they always appear at the same position in every call; they cannot be passed by keyword name, and we don't support additional **args keywords in calls because this can invalidate the positions declared in the decorator:

def rangetest(*argchecks):                  # validate positional arg ranges
    def onDecorator(func):
        if not __debug__:                   # True if "python -O main.py args.."
            return func                     # no-op: call original directly
        else:                               # else wrapper while debugging
            def onCall(*args):
                for (ix, low, high) in argchecks:
                    if args[ix] < low or args[ix] > high:
                        errmsg = 'Argument %s not in %s..%s' % (ix, low, high)
                        raise TypeError(errmsg)
                return func(*args)
            return onCall
    return onDecorator

As is, this code is mostly a rehash of the coding patterns we explored for privacy decorators in the prior section [in the book], so I won't describe it in detail here. A few quick highlights:

• Function versus class decorators: This is a function, not class decorator: it wraps function and method calls, not instance construction calls. As such its innermost nesting is a function which dispatches to the saved original function, after validation. There's no need for a nested class here, because we are wrapping a function call, not controlling a wrapped class instance.
• Decorator arguments: We're using decorator arguments again, to specify the position and ranges of arguments to be validated; as before, decorator argument values are retained in enclosing function scope references, and the rangetest function returns the actual decorator to be applied to the subject function for the @ syntax.
• Debug switch: We test the built-in name __debug__ here, to see if we're running in development mode. Python sets this variable to True only if has not been run with a -O command line argument. If False, we return the onCall wrapper to validate arguments. If True, we simply return the original function, and skip the validation wrapper altogether, thereby avoiding the performance cost of the onCall wrapper. The __debug__ name is True if you launch the script with a system command-line like python -O main.py args...; doing so effectively turns off range tests, and optimizes your function call's speed.
• Wrapping methods: Wrapped class methods work the same as simple functions, except that the self (first) argument is passed automatically, so argument specification numbers for methods must begin with 1, not 0. When used on a class method, the method name is rebound to the onCall function in the decorator, which remembers both the original method function and the argchecks tuple in enclosing scopes. When the original class method name is later called, onCall receives the instance in argument 1 as usual in *args, and passes this along to self in the original method function.

To test this decorator, the following code applies it to functions, as well as a class method—the one from the prior chapter's example. Notice that we can pass argument specifications in any sequence object (e.g., tuple or list); onCall's *args bundles them in a tuple, and the inner for loop's header line extracts their three parts with sequence assignment. Trace through the decorated Person.giveRaise method call in this to see how the self argument is propagated to the original method naturally:

#if __name__ == '__main__':                 # to make this a library 
print(__debug__)                            # True if no -O python cmdline arg                   
                                            # False if "python -O main.py args" 

@rangetest((1, 0, 120))                     # person = rangetest(..)(person)
def person(name, age):
    print('%s is %s years old' % (name, age))

@rangetest([0, 1, 12], [1, 1, 31], [2, 0, 2009])
def birthday(M, D, Y):
    print('birthday = {0}/{1}/{2}'.format(M, D, Y))


class Person:  
    """
    revisit the Person class to validate argument
    """
    def __init__(self, name, job, pay):
        self.job  = job
        self.pay  = pay
                                          # arg 0 is the self instance here
    @rangetest([1, 0.0, 1.0])             # giveRaise = rangetest(..)(giveRaise)
    def giveRaise(self, percent):
        self.pay = int(self.pay * (1 + percent))


# comment lines raise TypeError unless "python -O" on shell command line

person('Bob Smith', 45)                   # really runs onCall(..) with state 
#person('Bob Smith', 200)                 # or person() if -O cmd line argument

birthday(5, 31, 1963)
#birthday(5, 32, 1963)

sue = Person('Sue Jones', 'dev', 100000)
sue.giveRaise(.10)                        # really runs onCall(self, .10)
print(sue.pay)                            # or giveRaise(self, .10) if -O
#sue.giveRaise(1.10)
#print(sue.pay)

When run, valid calls in this code produce the following output (all the code in this section works the same under Python 2.X and 3.X, because function decorators are supported in both, and we use 3.X-style print call syntax):

C:\misc> C:\Python30\python rangetest.py
True
Bob Smith is 45 years old
birthday = 5/31/1963
110000

Uncommenting any of the invalid calls cause a TypeError to be raised by the decorator; here's the result when the last two lines are allowed to run (I've omitted some of the error message text here to save space):

C:\misc> C:\Python30\python rangetest.py
True
Bob Smith is 45 years old
birthday = 5/31/1963
110000
...
TypeError: Argument 1 not in 0.0..1.0

Running Python with its -O flag at a system command-line will disable range-testing, but also avoid the performance overhead of the wrapping layer—we wind up calling the original undecorated function directly. Assuming this is a debugging tool only, you can use this flag to optimize your program for production use:

C:\misc> C:\Python30\python -O rangetest.py
False
Bob Smith is 45 years old
birthday = 5/31/1963
110000
231000

Supporting Keyword Arguments Too

The prior version works and illustrates the basics we need to employ, but it's fairly limited—it supports validating arguments passed by position only, and assumes that no keyword arguments are passed in such a way that makes validated argument position numbers incorrect (they won't be in *args if they were passed by name, and will generate an index error in the wrapping layer's code). That's fine if all your arguments are passed by position, but a less-than-ideal constraint in a general tool. Python supports more flexible argument-passing modes, which we're not yet addressing.

Our next version does better. It supports range validations for arguments passed by either position or keyword name. Arguments specified by position are assumed to be passed by position, and arguments specified by keyword are assumed always passed by name. When the original function name is called, we simply step through both the *args tuple and **kargs dictionaries, running the specified positional and keyword argument validations, respectively.

def rangetest(*argchecks, **kargchecks):    # validate keyword arg ranges too
    def onDecorator(func):
        if not __debug__:                   # True if "python -O main.py args.."
            return func                     # no-op: call original directly
        else:                               # else wrapper while debugging
            def onCall(*args, **kargs):
                for (ix, low, high) in argchecks:
                    if args[ix] < low or args[ix] > high:
                        errmor = 'Argument #%s not in %s..%s' % (ix, low, high)
                        raise TypeError(error)
                    
                for (key, (low, high)) in kargchecks.items():
                    if kargs[key] < low or kargs[key] > high:
                        error = 'Argument "%s" not in %s..%s' % (key, low, high)
                        raise TypeError(error)
                    
                return func(*args, **kargs)
            return onCall
    return onDecorator

Here this version's self-test code; at the @ decorator, arguments expected to be passed by position are specified by number as before, and expected keyword arguments are given by name now. Run this on your own to experiment; the commented-out lines here are invalid, and raise exceptions in the decorator as expected:

@rangetest((1, 0, 120))                     # person = rangetest(..)(person)
def person(name, age):
    print('%s is %s years old' % (name, age))

@rangetest(M=(1, 12), D=(1, 31), Y=(0, 2009))
def birthday(M, D, Y):
    print('birthday = {0}/{1}/{2}'.format(M, D, Y))

@rangetest((0, 0, 999999999), height=(0.0, 7.0))     # ssn always by position
def record(ssn, category, height):                   # height always by keyword
    print('record: %s, %s' % (ssn, category))        # category not checked

class Person:
    """
    revisit the Person class to validate argument
    """
    def __init__(self, name, job, pay):
        self.job  = job
        self.pay  = pay
                                          # giveRaise = rangetest(..)(giveRaise)
    @rangetest((1, 0.0, 1.0))             # assume percent passed by position
    def giveRaise(self, percent):
        self.pay = int(self.pay * (1 + percent))
                                          
    @rangetest(percent=(0.0, 1.0))        # same, but assume passed by keyword
    def giveRaise2(self, percent, bonus=.10):
        self.pay = int(self.pay * (1 + percent + bonus))


# comment lines raise TypError unless "python -O" on shell command line

person('Bob Smith', 45)                    # really runs onCall(..) with state 
#person('Bob Smith', 200)                  # or person() if -O cmd line argument

birthday(M=5, D=31, Y=1963)
birthday(Y=1963, M=5, D=31)
#birthday(M=5, D=32, Y=1963)

record(123456789, 'spam', height=5.5)
record(123456789, height=5.5, category='spam')
#record(1234567899, height=5.5, category='spam')
#record(123456789, height=99, category='spam')

sue = Person('Sue Jones', 'dev', 100000)
sue.giveRaise(.10)                        # really runs onCall(self, .10)
print(sue.pay)                            # or giveRaise(self, .10) if -O
sue.giveRaise2(percent=.10)
print(sue.pay)
#sue.giveRaise2(percent=1.10)
#print(sue.pay)

Here's an example run, with the last two lines uncommented to trigger an error:

C:\misc> C:\Python30\python rangetest.py
True
Bob Smith is 45 years old
birthday = 5/31/1963
birthday = 5/31/1963
record: 123456789, spam
record: 123456789, spam
110000
132000
...
TypeError: Argument "percent" not in 0.0..1.0

Although both positional and keyword arguments can be validated now, this version does not support an argument being passed in either positional or keyword modes at different calls. Once decorated, the argument to test must always appear at the same position in every call, or always be passed by keyword in every call. This limits the flexibility of argument passing in calls, and other arguments cannot be passed in a way that changes expected positions. This may be a reasonable constraint during development, since expected call patterns can often be predicted; it's still not ideal in a general tool, though.

The root of inflexibility in this version is a bit subtle, and stems from Python's argument matching algorithm; but argument position numbers may not be reliable if arguments may be passed arbitrarily, by name or position. Given just an argument's name, we don't have enough information to know which position it may show up in if it is passed by position instead. Similarly, a given argument's expected position may not be valid if other arguments are passed in arbitrary ways. To address this uncertainty, this version requires that checked arguments must always be passed by either name or a fixed position.

To be fully general, we might be able to do better by requiring information at decoration time for each argument, validated or not; this seems a less than user-friendly policy. Would could also try to mimic Python's argument matching logic in its entirety to see which names have been passed in which modes, but that seems like far too much complexity for our tool—and is prone to change arbitrarily over time.

It might be better if we could somehow match arguments passed by name against the set of all expected arguments' names, in order to determine in which position arguments actually appear during a given call. To this end, we may be able to leverage the introspection API available on function objects and their associated code objects (which gives access to expected arguments' names). The next section shows one way to code this plan.

Allowing Arguments to be Passed in Either Mode

As described in the prior section, in order to allow validated arguments to be passed by either position or name, we need to be able to match against the set of all expected argument names, in order to determine which argument was passed to which position. It turns out that Python provides exactly what we need, as part of the introspection tools available on function objects, and their associated code objects. The set of expected argument names is the first N variables names attached to a function's code object:

# In Python 2.6 and earlier:

>>> def func(a, b, c=3, d=4):
	x = 1
	y = 2
	
>>> dir(func)
['__call__', '__class__', '__closure__', '__code__', '__defaults__', 
...more omitted...
'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals',
'func_name']

>>> dir(func.func_code)
['__class__', '__cmp__', '__delattr__', '__doc__', '__eq__', '__format__',
...more omitted...
'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 
'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 
'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']
 
>>> func.func_code.co_varnames
('a', 'b', 'c', 'd', 'x', 'y')
 
>>> func.func_code.co_nlocals
6
>>> func.func_code.co_argcount
4
>>> code = func.func_code                    # same as func.__code__ in 2.6 and 3.0
>>> code.co_varnames[:code.co_argcount]      # argument names at the front
('a', 'b', 'c', 'd')

Although the API has changed slightly in 3.0, it's still straightforward to access—we use func.__code__ instead of func.func_code, and can use the Python version number in the sys module to choose one or the other (the new __code__ attribute is also available in 2.6 redundantly for portability, but func_code must be used in prior 2.X Pythons):

# In Python 3.0 (and 2.6 for compatibility):

>>> sys.version_info                         # [0] is major release number
(3, 0, 0, 'final', 0)

>>> def func(a, b, c, d):
...     x = 1
...     y = 2
...
>>> code = func.__code__                     # different attribute name
>>> code.co_varnames[:code.co_argcount]       
('a', 'b', 'c', 'd')

Now, we can put these to use to validate ranges for arguments passed in either mode, by matching against all expected arguments. The algorithm is as follows:

1. Arguments to be checked are passed only by keyword to the decorator, such that it knows checked arguments by name, not position.
2. Arguments passed in an actual call to the decorated function may be passed by either name or position.
3. Checked arguments passed by keyword are trivial to test, since they will be present in the wrapping layer's **kargs dictionary.
4. Checked arguments not in **kargs are assumed to have been passed by position, and must be mapped to their actual position in the call, as follows:
   1. When an actual call occurs, obtain the names of all arguments expected by the function from the function's code object.
   2. Remove all actual arguments passed by keyword in the call and present in **kargs, from the function's list of expected arguments.
   3. The remaining expected arguments are assumed to have been passed by position, and their relative position in the remaining expected arguments list gives their position in the actual call's *pargs tuple.
5. To validate arguments, we first look for them in **kargs by name, and then in *pargs by their relative position in the remaining expected arguments list.

Under this scheme, both passing modes are supported for arguments to be checked: arguments expected by position but passed by name will show up in **kargs and be processed by name; conversely, arguments expected by name but passed by position will be mapped to their positions automatically in the remaining expected arguments list. Here's how this translates to code:

def rangetest(**argchecks):                 # validate ranges for both arg types
    def onDecorator(func):                  # onCall remembers func and argchecks
        if not __debug__:                   # True if "python -O main.py args.."
            return func                     # wrap if debugging else use original  
        else:                               
            import sys
            code = func.__code__ if sys.version_info[0] == 3 else func.func_code
            allargs = code.co_varnames[:code.co_argcount]
            
            def onCall(*pargs, **kargs):
                positionals = list(allargs) 
                for argname in kargs:
                    # for all passed by name, remove from expected
                    positionals.remove(argname)
                    
                for (argname, (low, high)) in argchecks.items():
                    # for all args to be checked
                    if argname in kargs:
                        # was passed by name
                        if kargs[argname] < low or kargs[argname] > high:
                            errmsg = 'Argument "{0}" not in {1}..{2}'
                            errmsg = errmsg.format(argname, low, high)
                            raise TypeError(errmsg)
                    else:
                        # was passed by position
                        position = positionals.index(argname)
                        if pargs[position] < low or pargs[position] > high:
                            errmsg = 'Argument "{0}" not in {1}..{2}'
                            errmsg = errmsg.format(argname, low, high)
                            raise TypeError(errmsg)

                return func(*pargs, **kargs)    # okay: run original call
            return onCall
    return onDecorator

To test, the following makes sure that the prior calls still work in this scheme, but add an additional function to stress the code further. Notice how arguments to test are given only by name now, since the decorator automatically maps them to their expected positions if they are not passed by keyword in the actual call:

@rangetest(age=(0, 120))                    # person = rangetest(..)(person)
def person(name, age):
    print('%s is %s years old' % (name, age))

@rangetest(M=(1, 12), D=(1, 31), Y=(0, 2009))
def birthday(M, D, Y):
    print('birthday = {0}/{1}/{2}'.format(M, D, Y))

@rangetest(ssn=(0, 999999999), height=(0.0, 7.0))
def record(ssn, category, height):                   # args by name or position
    print('record: %s, %s' % (ssn, category))        # category not checked

class Person:
    """
    revisit the Person class to validate argument
    """
    def __init__(self, name, job, pay):
        self.job  = job
        self.pay  = pay
                                          # giveRaise = rangetest(..)(giveRaise)
    @rangetest(percent=(0.0, 1.0))        # percent passed by name or position
    def giveRaise(self, percent):
        self.pay = int(self.pay * (1 + percent))
                                          

# comment lines raise TypError unless "python -O" on shell command line

person('Bob Smith', 45)                    # really runs onCall(..) with state
person(age=45, name='Bob Smith')
#person('Bob Smith', 200)                  # or person() if -O cmd line argument
#person('Bob Smith', age=200)
#person(age=200, name='Bob Smith')

birthday(M=5, D=31, Y=1963)
birthday(Y=1963, M=5, D=31)
birthday(5, 31, Y=1963)
birthday(5, D=31, Y=1963)
birthday(5, Y=1963, D=31)
#birthday(5, 32, 1963)
#birthday(5, 32, Y=1963)
#birthday(5, D=32, Y=1963)
#birthday(5, Y=1963, D=32)
#birthday(M=5, D=32, Y=1963)

record(123456789, 'spam', height=5.5)
record(123456789, height=5.5, category='spam')
#record(1234567899, height=5.5, category='spam')
#record(123456789, height=99, category='spam')

sue = Person('Sue Jones', 'dev', 100000)
sue.giveRaise(.10)                        # really runs onCall(self, .10)
print(sue.pay)                            # or giveRaise(self, .10) if -O
sue.giveRaise(percent=.10)
print(sue.pay)
#sue.giveRaise(1.10)
#sue.giveRaise(percent=1.10)
#print(sue.pay)

Supporting Omitted Default Arguments Too

One final twist: the preceding version works as intended, allowing checked arguments to be passed either by name or position. Unfortunately, it still fails to address default arguments that are omitted in a call. As a last step, let's generalize the decorator to skip range tests for arguments that are defaulted and omitted.

The reason the prior version does not support defaults lies in its assumption that checked arguments not in the **kargs dictionary are assumed to have been passed by position. That's not true if an argument was defaulted—it won't be in **kargs, and may not be in the *pargs positionals list either, even if it's part of the remaining expected arguments list. Because the decorator does not skip testing for arguments that are allowed to default in the call, it can still fail by indexing off the end of the actual arguments list.

With the prior version, for instance, the valid call in the second last line of the following triggers an index error in the decorator's code, because argument d is not in the keyword **kargs, but is also not present in the positional *pargs since it was defaulted; indexing the actual arguments list on the relative position of d is out of bounds. The last line similarly fails, because c is in the expected list, but not in the actual *pargs:

@rangetest(a=(1,10), b=(1,10), c=(1, 10), d=(1, 10))
def omitargs(a, b=7, c=8, d=9):
    print(a, b, c, d)

omitargs(1, 2, 3, 4)       # works
omitargs(1, 2, 3)          # FAILS! - index out of range, pargs[position]
omitargs(1, 2, d=4)        # FAILS! - same reason: d in **, but c fails

On first glance, accounting for this is a potentially difficult problem. When the wrapper is invoked for an actual call, all we have access to are the sets of positional and keyword arguments passed; argument names in the decorated function have not been assigned yet, and indeed, we can't even be sure if the call is legal. Although we could try to emulate Python's argument matching mechanism at this point, we can get the same mileage by simply modifying the algorithm used by the prior version.

The simpler solution relies on two constraints on argument passing order imposed by Python (these still hold true both in 2.6 and 3.0):

1. At the call, all positional arguments appear before all keyword arguments
2. In the def, all non-default arguments appear before all default arguments

That is, a non-keyword argument cannot follow a keyword argument at a call, and a non-default argument cannot follow a default argument at a definition. All name=value must appear after any simple name in both places.

To simplify our task, we can also make the assumption that a call is valid in general, such that all arguments will receive values (by name or position), or were omitted to pick up defaults. This isn't quite true, because the function hasn't yet been actually called when the wrapper logic runs to test arguments—it may still fail later when invoked by the wrapper layer, due to incorrect argument passing. As long as that doesn't cause the wrapper to fail any more badly, though, we can finesse the validity of the call (and if the call will fail anyhow, we don't care much what it does to our decorator). This helps, because validating calls before the call is actually made would require us to emulate the argument-matching algorithm of Python exactly—a much too complex procedure for the purposes of our development tool.

Now, given these assumptions, we can allow for omitted default arguments in the call with a slightly different algorithm:

1. Rather than removing all the passed keyword arguments from the set of all expected arguments, we can instead assume that all the N passed positional arguments match the first N expected arguments. This is true per Python's call ordering rules given above, since all positionals precede all keywords.
2. To obtain the list of arguments actually passed by position, we can slice the list of all expected arguments up to the length of the *pargs positionals tuple.
3. Any arguments after the first N actual positionals, then, were either passed by keyword, or were defaulted by omission at the call.
4. For each argument to be validated, if it is in **kargs it was passed by name; if it is instead in the first N expected arguments, it was passed by position; otherwise, we can assume it was omitted in the call and defaulted, and need not be checked.

In other words, we can skip tests for arguments that were omitted in a call, by assuming that the first N actually passed positional arguments in *pargs must match the first N argument names in the list of all expected arguments. Any others must have been passed by keyword and thus be in **kargs, or have been defaulted. Under this scheme, if an argument to be checked was omitted between the rightmost positional argument and the leftmost keyword argument, between keyword arguments, or after the rightmost positional in general, it will simply be skipped by the decorator, and not trigger an index error. The following final form captures this approach:

trace = True

def rangetest(**argchecks):                 # validate ranges for both+defaults
    def onDecorator(func):                  # onCall remembers func and argchecks
        if not __debug__:                   # True if "python -O main.py args.."
            return func                     # wrap if debugging else use original
        else:
            import sys
            code = func.__code__ if sys.version_info[0] == 3 else func.func_code
            allargs  = code.co_varnames[:code.co_argcount]
            funcname = func.__name__
            
            def onCall(*pargs, **kargs):
                # all pargs match first N args by position
                # the rest must be in kargs or omitted defaults
                positionals = list(allargs)
                positionals = positionals[:len(pargs)]

                for (argname, (low, high)) in argchecks.items():
                    # for all args to be checked
                    if argname in kargs:
                        # was passed by name
                        if kargs[argname] < low or kargs[argname] > high:
                            errmsg = '{0} argument "{1}" not in {2}..{3}'
                            errmsg = errmsg.format(funcname, argname, low, high)
                            raise TypeError(errmsg)

                    elif argname in positionals:
                        # was passed by position
                        position = positionals.index(argname)
                        if pargs[position] < low or pargs[position] > high:
                            errmsg = '{0} argument "{1}" not in {2}..{3}'
                            errmsg = errmsg.format(funcname, argname, low, high)
                            raise TypeError(errmsg)
                    else:
                        # assume not passed: default
                        if trace:
                            print('Argument "{0}" defaulted'.format(argname))

                return func(*pargs, **kargs)    # okay: run original call
            return onCall
    return onDecorator

To test, we'll repeat prior cases again to verify no new breakage, and add a new function with defaults to verify our final mutation (I've omitted repeated test cases' code here for space; see the preceding version's self-test code, or the source file link at the end of this section):

...repeated tests omitted are coded and run the same...

@rangetest(a=(1, 10), b=(1, 10), c=(1, 10), d=(1, 10))
def omitargs(a, b=7, c=8, d=9):
    print(a, b, c, d)                                # skip omitted defaults
                                          

# comment lines raise TypeError unless "python -O" on shell command line

...repeated tests omitted are coded and run the same...

omitargs(1, 2, 3, 4)
omitargs(1, 2, 3)
omitargs(1, 2)
omitargs(1)
omitargs(1, 2, 3, d=4)
omitargs(1, 2, d=4)
omitargs(1, d=4)
omitargs(a=1, d=4)
omitargs(d=4, a=1)
omitargs(1, b=2)
omitargs(1, b=2, d=4)
omitargs(a=1)
omitargs(d=8, c=7, b=6, a=1)
omitargs(d=8, c=7, a=1)


#omitargs(1, 2, 3, 11)
#omitargs(1, 2, 11)
#omitargs(1, 11, 3)
#omitargs(11, 2, 3)
#omitargs(1, 11)
#omitargs(11)
#omitargs(1, 2, 3, d=11)
#omitargs(1, 2, 11, d=4)
#omitargs(1, 2, d=11)
#omitargs(1, 11, d=4)
#omitargs(1, d=11)
#omitargs(11, d=4)
#omitargs(d=4, a=11)
#omitargs(1, b=11, d=4)
#omitargs(1, b=2, d=11)
#omitargs(1, b=2, c=11)
#omitargs(1, b=11)
#omitargs(d=8, c=11, b=6, a=1)
#omitargs(d=8, c=7, a=11)

Open Issues

Two final caveats: First, calls to the original function that are not valid still fail in our final decorator. The following both trigger exceptions, for example:

omitargs()
omitargs(d=8, c=7, b=6)

These only fail, though, where we try to invoke the original, with func(*pargs, **kargs). While we could try to imitate Python's argument matching to avoid this, there's not much reason to do so—since the call would fail at this point anyhow, we might as well let Python's own argument matching logic detect the problem for us.

Lastly, although our final version handles positional arguments, keyword arguments, and omitted defaults, it still doesn't do anything explicit about *args and **args that may be used in a decorated function that accepts arbitrarily many arguments. We probably don't need to care for our purposes, though:

• If an extra keyword argument is passed, it will show up in **kargs and can be tested normally by name if mentioned to the decorator.
• If an extra keyword argument is not passed, it won't be in either **kargs or the sliced expected positionals list, and will not be checked—it is treated as though it were defaulted, even though it is really an optional extra argument.
• If an extra argument is passed by position instead, there's no way to reference it in the decorator anyhow—it won't be named in the sliced expected arguments list, and so will be simply skipped. Because such arguments are not listed in the function's definition, there's no way to map a name given to the decorator back to an expected relative position.

In other words, as is the code supports testing arbitrary keyword arguments by name, but not arbitrary positionals that are unnamed and hence have no set position in the function's argument signature.

For instance, consider the following simplified version of our decorator's code:

def func(a, b, *pargs, **kargs):
    print('func:', pargs, kargs)        # jun-2018: 3.X compatible prints

code = func.__code__
print('expected:', code.co_varnames[:code.co_argcount])

def wrap(func):
    def onCall(*pargs, **kargs):
        print('onCall:', pargs, kargs)
        func(*pargs, **kargs)
    return onCall

func = wrap(func)
func(1, 2, 3, 4, x=4, y=5)

When run, only a and b show up as expected argument names on the function object; kargs and pargs do not. Within the wrapped function itself, arguments that don't match a and b wind up in *pargs or **kargs. In the wrapping layer, however, all actual arguments show up in *pargs or **kargs:

expected: ('a', 'b')
onCall: (1, 2, 3, 4) {'y': 5, 'x': 4}
func: (3, 4) {'y': 5, 'x': 4}

Slicing the expected arguments list on the length of *pargs will leave it unchanged in this case, since there are more actual than expected. Extra keyword arguments can be range-tested as is, because their names will either appear in **kargs, or be unnamed in the sliced expected arguments list and thus skipped. Extra positionals will simply be skipped, because their names do not appear on the sliced expected arguments list. If we omit a default argument, we get the same net result—the expected arguments list is reduced to just a, so checks on name b will be skipped, though any keyword arguments can still be tested out of **kargs:

def func(a, b=8, *pargs, **kargs):
...
func(1, x=4, y=5)
...

expected: ('a', 'b')
onCall: (1,) {'y': 5, 'x': 4}
func: () {'y': 5, 'x': 4}

In principle, we could extend the decorator's interface to support *pargs in the decorated function too, for the very rare cases where this might be useful (e.g., a special argument name with a test to apply to all arguments in the wrapper's *pargs beyond the length of the expected arguments list). Since we've already exhausted the space allocation for this example, though, if you care about such improvements, you've officially crossed over into the realm of suggested exercise.

Why Decorators Here?

Now that we've worked so hard to carefully craft a general and flexible decorator solution, I should mention that we can often achieve the same effect by adding manual tests or using assert statements inside the function itself. For example, the following range testing using our final decorator code:

class Person:
    @rangetest(percent=(0.0, 1.0))               # use decorator to validate
    def giveRaise(self, percent):
        self.pay = int(self.pay * (1 + percent))

could be implemented by simply testing ranges manually inside the function:

class Person:
    def giveRaise(self, percent):                # validate with in-line code
        if percent < 0.0 or percent > 1.0:
            raise TypeError, 'percent invalid'
        self.pay = int(self.pay * (1 + percent))

or by using the Python assert statement, which is designed for this sort of development-time testing:

class Person:                                    # validate with asserts
    def giveRaise(self, percent):            
        assert percent >= 0.0 and percent <= 1.0, 'percent invalid'
        self.pay = int(self.pay * (1 + percent))

As we'll see in the next part of the book, the assert statement is like a conditional raise: it raises an exception if a test is not true. Like our decorator, assert statements also check the __debug__ flag and do nothing if the Python -O flag is used to run the script (e.g., python -O m.py); in fact assert goes a minor step further, and generates no code at all when -O is used.

So why bother with the decorator? In the simple case of testing one argument's range, assert may be just as good—it's one line of code versus one line of code. In less straightforward cases, though, asking programmers to in-line more complex logic in their functions is much less friendly than coding it in a reusable decorator (imagine in-lining the equivalent of the [book's] last section's private and public declarations, for example). For more complex requirements than those we've seen here, the decorator coding pattern may provide better support than in-line tests or assertions.

Moreover, if decoration logic must ever change, there is only one copy to update in a decorator; without one, there may be arbitrarily many in-lined copies to find and change. Although the @ syntax is not strictly required to leverage this benefit (we could rebind the function name with a manual assignment, and still avoid code redundancy with encapsulation), the special @ line can make our intent more obvious.

In addition, once you see the coding pattern we've arrived at for proceeding arguments in decorators, it can be applied in other contexts. Checking argument data types at development time, for example, is a straightforward extension:

def typetest(**argchecks):
    def onDecorator(func):
           ....
           def onCall(*pargs, **kargs):
                positionals = list(allargs)[:len(pargs)]
                for (argname, type) in argchecks.items():
                     if argname in kargs:
                        if not isinstance(kargs[argname], type):
                            ...
                            raise TypeError(errmsg)
                    elif argname in positionals:
                        position = positionals.index(argname)
                        if not isinstance(pargs[position], type):
                            ...
                            raise TypeError(errmsg)
                    else:
                        # assume not passed: default
                return func(*pargs, **kargs)
            return onCall
    return onDecorator

@typetest(a=int, c=float)
def func(a, b, c, d):                    # like func = typetest(...)(func)
    ...

func(1, 2, 3.0, 4)                       # okay
func('spam', 2, 99, 4)                   # triggers exception

As you should have learned in the book, though, this particular role is a generally bad idea in working code, and not at all "Pythonic"—type testing restricts your function to work on specific types only, instead of allowing it to operate on any types with compatible interfaces. In effect, it limits your code, and breaks its flexibility. On the other hand, every rule has exceptions; type checking may come in handy in isolated cases while debugging, and when interfacing with code written in more restrictive languages such as C++. This general pattern of argument processing might also be applicable in a variety of less controversial roles.

In the end, an author's role is to raise questions about the merits of alternative techniques, unless a best practice has already emerged. As usual, your role should always be to judge such tradeoffs for yourself.

For More Reading

This article's material was polished and published in this book's 4th and 5th editions. You can fetch its example code here: rangetest.py. The code was later adapted to function-parameter type testing in a 2015 post, and is presented alongside privacy declarations in the books.

For additional reading, try these other articles popular at learning-python.com:

• Using tkinter Programs on Android
• Strings in 3.X: Unicode and Binary Data
• The New Windows Launcher in Python 3.3
• A More Realistic OOP Example
• Python's New-Style Inheritance Algorithm
• When Pythons Attack (restored)
• Answer Me These Questions Three...
• Python Changes 2014+
• Teaching Python — 25 Years of Spam

These and more are available on the blog page.