History of Positional-Only Parameter Semantics in Python

Python originally supported positional-only parameters. Early versions of the language lacked the ability to call functions with arguments bound to parameters by name. Around Python 1.0, parameter semantics changed to be positional-or-keyword. Since then, users have been able to provide arguments to a function either positionally or by the keyword name specified in the function’s definition.

In current versions of Python, many CPython “builtin” and standard library functions only accept positional-only parameters. The resulting semantics can be easily observed by calling one of these functions using keyword arguments:

>>> help(pow)
...
pow(x, y, z=None, /)
...

>>> pow(x=5, y=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: pow() takes no keyword arguments

pow() expresses that its parameters are positional-only via the / marker. However, this is only a documentation convention; Python developers cannot use this syntax in code.

There are functions with other interesting semantics:

• range(), an overloaded function, accepts an optional parameter to the left of its required parameter. [4]
• dict(), whose mapping/iterator parameter is optional and semantically must be positional-only. Any externally visible name for this parameter would occlude that name going into the **kwarg keyword variadic parameter dict. [3]

One can emulate these semantics in Python code by accepting (*args, **kwargs) and parsing the arguments manually. However, this results in a disconnect between the function definition and what the function contractually accepts. The function definition does not match the logic of the argument handling.

Additionally, the / syntax is used beyond CPython for specifying similar semantics (i.e., [1] [2]); thus, indicating that these scenarios are not exclusive to CPython and the standard library.

Problems Without Positional-Only Parameters

Without positional-only parameters, there are challenges for library authors and users of APIs. The following subsections outline the problems encountered by each entity.

Benefits of Positional-Only Parameters

Positional-only parameters give more control to library authors to better express the intended usage of an API and allows the API to evolve in a safe, backward-compatible way. Additionally, it makes the Python language more consistent with existing documentation and the behavior of various “builtin” and standard library functions.

def as_my_type(x):
    ...

def add_to_queue(item: QueueItem):
    ...

def add_to_queue(items: Union[QueueItem, List[QueueItem]]):
    ...

def add_to_queue(*items: QueueItem):
    ...

io.FileIO.write(self=f, b=b"data")

>>> help(io.FileIO.write)
Help on method_descriptor:

write(self, b, /)
    Write buffer b to file, return number of bytes written.

Performance

In addition to the aforementioned benefits, the parsing and handling of positional-only arguments is faster. This performance benefit can be demonstrated in this thread about converting keyword arguments to positional: [11]. Due to this speedup, there has been a recent trend towards moving builtins away from keyword arguments: recently, backwards-incompatible changes were made to disallow keyword arguments to bool, float, list, int, tuple.

Maintainability

Providing a way to specify positional-only parameters in Python will make it easier to maintain pure Python implementations of C modules. Additionally, library authors defining functions will have the choice for choosing positional-only parameters if they determine that passing a keyword argument provides no additional clarity.

This is a well discussed, recurring topic on the Python mailing lists:

• September 2018: Anders Hovmöller: [Python-ideas] Positional-only parameters
• February 2017: Victor Stinner: [Python-ideas] Positional-only parameters, discussion continued in March
• February 2017: [9]
• March 2012: [8]
• May 2007: George Sakkis: [Python-ideas] Positional only arguments
• May 2006: Benji York: [Python-Dev] Positional-only Arguments

Logical ordering

Positional-only parameters also have the (minor) benefit of enforcing some logical order when calling interfaces that make use of them. For example, the range function takes all its parameters positionally and disallows forms like:

range(stop=5, start=0, step=2)
range(stop=5, step=2, start=0)
range(step=2, start=0, stop=5)
range(step=2, stop=5, start=0)

at the price of disallowing the use of keyword arguments for the (unique) intended order:

range(start=0, stop=5, step=2)

Compatibility for Pure Python and C Modules

Another critical motivation for positional-only parameters is PEP 399: Pure Python/C Accelerator Module Compatibility Requirements. This PEP states that:

This PEP requires that in these instances that the C code must pass the test suite used for the pure Python code to act as much as a drop-in replacement as reasonably possible

If the C code is implemented using the existing capabilities to implement positional-only parameters using the argument clinic, and related machinery, it is not possible for the pure Python counterpart to match the provided interface and requirements. This creates a disparity between the interfaces of some functions and classes in the CPython standard library and other Python implementations. For example:

$ python3 # CPython 3.7.2
>>> import binascii; binascii.crc32(data=b'data')
TypeError: crc32() takes no keyword arguments

$ pypy3 # PyPy 6.0.0
>>>> import binascii; binascii.crc32(data=b'data')
2918445923

Other Python implementations can reproduce the CPython APIs manually, but this goes against the spirit of PEP 399 to avoid duplication of effort by mandating that all modules added to Python’s standard library must have a pure Python implementation with the same interface and semantics.

Consistency in Subclasses

Another scenario where positional-only parameters provide benefit occurs when a subclass overrides a method of the base class and changes the name of parameters that are intended to be positional:

class Base:
    def meth(self, arg: int) -> str:
        ...

class Sub(Base):
    def meth(self, other_arg: int) -> str:
        ...

def func(x: Base):
    x.meth(arg=12)

func(Sub())  # Runtime error

This situation could be considered a Liskov violation — the subclass cannot be used in a context when an instance of the base class is expected. Renaming arguments when overloading methods can happen when the subclass has reasons to use a different choice for the parameter name that is more appropriate for the specific domain of the subclass (e.g., when subclassing Mapping to implement a DNS lookup cache, the derived class may not want to use the generic argument names ‘key’ and ‘value’ but rather ‘host’ and ‘address’). Having this function definition with positional-only parameters can avoid this problem because users will not be able to call the interface using keyword arguments. In general, designing for subclassing usually involves anticipating code that hasn’t been written yet and over which the author has no control. Having measures that can facilitate the evolution of interfaces in a backwards-compatible would be useful for library authors.

Optimizations

A final argument in favor of positional-only parameters is that they allow some new optimizations like the ones already present in the argument clinic due to the fact that parameters are expected to be passed in strict order. For example, CPython’s internal METH_FASTCALL calling convention has been recently specialized for functions with positional-only parameters to eliminate the cost for handling empty keywords. Similar performance improvements can be applied when creating the evaluation frame of Python functions thanks to positional-only parameters.

Syntax and Semantics

From the “ten-thousand foot view”, eliding *args and **kwargs for illustration, the grammar for a function definition would look like:

def name(positional_or_keyword_parameters, *, keyword_only_parameters):

Building on that example, the new syntax for function definitions would look like:

def name(positional_only_parameters, /, positional_or_keyword_parameters,
         *, keyword_only_parameters):

The following would apply:

• All parameters left of the / are treated as positional-only.
• If / is not specified in the function definition, that function does not accept any positional-only arguments.
• The logic around optional values for positional-only parameters remains the same as for positional-or-keyword parameters.
• Once a positional-only parameter is specified with a default, the following positional-only and positional-or-keyword parameters need to have defaults as well.
• Positional-only parameters which do not have default values are required positional-only parameters.

Therefore, the following would be valid function definitions:

def name(p1, p2, /, p_or_kw, *, kw):
def name(p1, p2=None, /, p_or_kw=None, *, kw):
def name(p1, p2=None, /, *, kw):
def name(p1, p2=None, /):
def name(p1, p2, /, p_or_kw):
def name(p1, p2, /):

Just like today, the following would be valid function definitions:

def name(p_or_kw, *, kw):
def name(*, kw):

While the following would be invalid:

def name(p1, p2=None, /, p_or_kw, *, kw):
def name(p1=None, p2, /, p_or_kw=None, *, kw):
def name(p1=None, p2, /):

Full Grammar Specification

A simplified view of the proposed grammar specification is:

typedargslist:
  tfpdef ['=' test] (',' tfpdef ['=' test])* ',' '/' [','  # and so on

varargslist:
  vfpdef ['=' test] (',' vfpdef ['=' test])* ',' '/' [','  # and so on

Based on the reference implementation in this PEP, the new rule for typedarglist would be:

typedargslist: (tfpdef ['=' test] (',' tfpdef ['=' test])* ',' '/' [',' [tfpdef ['=' test] (',' tfpdef ['=' test])* [',' [
        '*' [tfpdef] (',' tfpdef ['=' test])* [',' ['**' tfpdef [',']]]
      | '**' tfpdef [',']]]
  | '*' [tfpdef] (',' tfpdef ['=' test])* [',' ['**' tfpdef [',']]]
  | '**' tfpdef [',']] ] )| (
   tfpdef ['=' test] (',' tfpdef ['=' test])* [',' [
        '*' [tfpdef] (',' tfpdef ['=' test])* [',' ['**' tfpdef [',']]]
      | '**' tfpdef [',']]]
 | '*' [tfpdef] (',' tfpdef ['=' test])* [',' ['**' tfpdef [',']]]
 | '**' tfpdef [','])

and for varargslist would be:

varargslist: vfpdef ['=' test ](',' vfpdef ['=' test])* ',' '/' [',' [ (vfpdef ['=' test] (',' vfpdef ['=' test])* [',' [
        '*' [vfpdef] (',' vfpdef ['=' test])* [',' ['**' vfpdef [',']]]
      | '**' vfpdef [',']]]
  | '*' [vfpdef] (',' vfpdef ['=' test])* [',' ['**' vfpdef [',']]]
  | '**' vfpdef [',']) ]] | (vfpdef ['=' test] (',' vfpdef ['=' test])* [',' [
        '*' [vfpdef] (',' vfpdef ['=' test])* [',' ['**' vfpdef [',']]]
      | '**' vfpdef [',']]]
  | '*' [vfpdef] (',' vfpdef ['=' test])* [',' ['**' vfpdef [',']]]
  | '**' vfpdef [',']
)

Semantic Corner Case

The following is an interesting corollary of the specification. Consider this function definition:

def foo(name, **kwds):
    return 'name' in kwds

There is no possible call that will make it return True. For example:

>>> foo(1, **{'name': 2})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() got multiple values for argument 'name'
>>>

But using / we can support this:

def foo(name, /, **kwds):
    return 'name' in kwds

Now the above call will return True.

In other words, the names of positional-only parameters can be used in **kwds without ambiguity. (As another example, this benefits the signatures of dict() and dict.update().)

Origin of “/” as a Separator

Using / as a separator was initially proposed by Guido van Rossum in 2012 [8] :

Alternative proposal: how about using ‘/’ ? It’s kind of the opposite of ‘*’ which means “keyword argument”, and ‘/’ is not a new character.

Positional-or-Keyword Arguments

If / and * are not present in the function definition, arguments may be passed to a function by position or by keyword.

Positional-Only Parameters

Looking at this in a bit more detail, it is possible to mark certain parameters as positional-only. If positional-only, the parameters’ order matters, and the parameters cannot be passed by keyword. Positional-only parameters would be placed before a / (forward-slash). The / is used to logically separate the positional-only parameters from the rest of the parameters. If there is no / in the function definition, there are no positional-only parameters.

Parameters following the / may be positional-or-keyword or keyword-only.

Keyword-Only Arguments

To mark parameters as keyword-only, indicating the parameters must be passed by keyword argument, place an * in the arguments list just before the first keyword-only parameter.

Function Examples

Consider the following example function definitions paying close attention to the markers / and *:

>>> def standard_arg(arg):
...     print(arg)
...
>>> def pos_only_arg(arg, /):
...     print(arg)
...
>>> def kwd_only_arg(*, arg):
...     print(arg)
...
>>> def combined_example(pos_only, /, standard, *, kwd_only):
...     print(pos_only, standard, kwd_only)

The first function definition standard_arg, the most familiar form, places no restrictions on the calling convention and arguments may be passed by position or keyword:

>>> standard_arg(2)
2

>>> standard_arg(arg=2)
2

The second function pos_only_arg is restricted to only use positional parameters as there is a / in the function definition:

>>> pos_only_arg(1)
1

>>> pos_only_arg(arg=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: pos_only_arg() got an unexpected keyword argument 'arg'

The third function kwd_only_args only allows keyword arguments as indicated by a * in the function definition:

>>> kwd_only_arg(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: kwd_only_arg() takes 0 positional arguments but 1 was given

>>> kwd_only_arg(arg=3)
3

And the last uses all three calling conventions in the same function definition:

>>> combined_example(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: combined_example() takes 2 positional arguments but 3 were given

>>> combined_example(1, 2, kwd_only=3)
1 2 3

>>> combined_example(1, standard=2, kwd_only=3)
1 2 3

>>> combined_example(pos_only=1, standard=2, kwd_only=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: combined_example() got an unexpected keyword argument 'pos_only'

Recap

The use case will determine which parameters to use in the function definition:

def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):

As guidance:

• Use positional-only if names do not matter or have no meaning, and there are only a few arguments which will always be passed in the same order.
• Use keyword-only when names have meaning and the function definition is more understandable by being explicit with names.

Do Nothing

Always an option — the status quo. While this was considered, the aforementioned benefits are worth the addition to the language.

Decorators

It has been suggested on python-ideas [9] to provide a decorator written in Python for this feature.

This approach has the benefit of not polluting function definition with additional syntax. However, we have decided to reject this idea because:

• It introduces an asymmetry with how parameter behavior is declared.
• It makes it difficult for static analyzers and type checkers to safely identify positional-only parameters. They would need to query the AST for the list of decorators and identify the correct one by name or with extra heuristics, while keyword-only parameters are exposed directly in the AST. In order for tools to correctly identify positional-only parameters, they would need to execute the module to access any metadata the decorator is setting.
• Any error with the declaration will be reported only at runtime.
• It may be more difficult to identify positional-only parameters in long function definitions, as it forces the user to count them to know which is the last one that is impacted by the decorator.
• The / syntax has already been introduced for C functions. This inconsistency will make it more challenging to implement any tools and modules that deal with this syntax — including but not limited to, the argument clinic, the inspect module and the ast module.
• The decorator implementation would likely impose a runtime performance cost, particularly when compared to adding support directly to the interpreter.

Per-Argument Marker

A per-argument marker is another language-intrinsic option. The approach adds a token to each of the parameters to indicate they are positional-only and requires those parameters to be placed together. Example:

def (.arg1, .arg2, arg3):

Note the dot (i.e., .) on .arg1 and .arg2. While this approach may be easier to read, it has been rejected because / as an explicit marker is congruent with * for keyword-only arguments and is less error-prone.

It should be noted that some libraries already use leading underscore [12] to conventionally indicate parameters as positional-only.

Using “__” as a Per-Argument Marker

Some libraries and applications (like mypy or jinja) use names prepended with a double underscore (i.e., __) as a convention to indicate positional-only parameters. We have rejected the idea of introducing __ as a new syntax because:

• It is a backwards-incompatible change.
• It is not symmetric with how the keyword-only parameters are currently declared.
• Querying the AST for positional-only parameters would require checking the normal arguments and inspecting their names, whereas keyword-only parameters have a property associated with them (FunctionDef.args.kwonlyargs).
• Every parameter would need to be inspected to know when positional-only arguments end.
• The marker is more verbose, forcing marking every positional-only parameter.
• It clashes with other uses of the double underscore prefix like invoking name mangling in classes.

Group Positional-Only Parameters With Parentheses

Tuple parameter unpacking is a Python 2 feature which allows the use of a tuple as a parameter in a function definition. It allows a sequence argument to be unpacked automatically. An example is:

def fxn(a, (b, c), d):
    pass

Tuple argument unpacking was removed in Python 3 (PEP 3113). There has been a proposition to reuse this syntax to implement positional-only parameters. We have rejected this syntax for indicating positional only parameters for several reasons:

• The syntax is asymmetric with respect to how keyword-only parameters are declared.
• Python 2 uses this syntax which could raise confusion regarding the behavior of this syntax. This would be surprising to users porting Python 2 codebases that were using this feature.
• This syntax is very similar to tuple literals. This can raise additional confusion because it can be confused with a tuple declaration.

After Separator Proposal

Marking positional-parameters after the / was another idea considered. However, we were unable to find an approach which would modify the arguments after the marker. Otherwise, would force the parameters before the marker to be positional-only as well. For example:

def (x, y, /, z):

If we define that / marks z as positional-only, it would not be possible to specify x and y as keyword arguments. Finding a way to work around this limitation would add confusion given that at the moment keyword arguments cannot be followed by positional arguments. Therefore, / would make both the preceding and following parameters positional-only.