GeezerVison

Controlling the computer mouse in code is a handy task, as it can be helpful for desktop automation, making useful desktop agents, etc. In this tutorial, you will learn how you can control the mouse in Python.

We are going to be using the convenient mouse library. Let's install it:

$ pip3 install mouse

This module helps us take full control of our mouse, such as hooking global events, registering hotkeys, simulating mouse movement and clicks, and much more!

Table of contents:

• Simulating Mouse Clicks
• Getting Mouse Position
• Dragging with the Mouse
• Moving the Mouse
• Making Callbacks on Mouse Events
• Scrolling Up and Down
• Recording Mouse Events and Playing them
• Toy Drawing Project
• Conclusion

Simulating Mouse Clicks

First, let's see how we can simulate mouse clicks:

import mouse

# left click
mouse.click('left')

# right click
mouse.click('right')

# middle click
mouse.click('middle')

Note: It is suggested to run these statements individually in a Python interactive shell such as a Jupyter notebook, IPython, or just a regular Python shell.

The mouse.click() function does what its name suggests; it sends a click with the given button; try it out!

Getting Mouse Position

Second, you can also get the current position of the mouse:

In [22]: mouse.get_position()
Out[22]: (646, 407)

The mouse.get_position() function retrieves the mouse cursor's current position on the screen. When you call this function, it returns a tuple containing two elements: the x-coordinate and the y-coordinate of the cursor's position.

These coordinates are typically represented in pixels. The top-left corner of the screen is considered the origin (0, 0). The x-coordinate increases as you move to the right, and the y-coordinate increases as you move downwards.

For example, the above output means that the mouse cursor is located 646 pixels to the right of the left edge of the main screen and 407 pixels down from the top edge of the screen.

If you have multiple screens, you may get negative values or exceed the resolution of your primary screen. This happens because the operating system treats the entire set of screens as one large display area.

Dragging with the Mouse

You can drag something with the mouse:

# drag from (0, 0) to (100, 100) relatively with a duration of 0.1s
mouse.drag(0, 0, 100, 100, absolute=False, duration=0.1)

Setting absolute equal to False with (0, 0) start positions mean that it drags from the current position to 100 farther (in x and y).

Test this on a file you want to drag on your desktop!

Next, you can also determine whether a button is pressed:

# whether the right button is clicked
In [25]: mouse.is_pressed("right")
Out[25]: False

Moving the Mouse

You can also move the mouse:

# move 100 right & 100 down
mouse.move(100, 100, absolute=False, duration=0.2)

This will move the mouse relatively for 0.2 seconds. Let's break down what each parameter in this function call means:

• (100, 100): These are the x and y coordinates to which you want to move the mouse cursor. In this case, both are set to 100.
  
• absolute=False: This parameter determines the nature of the movement:
  
  • When absolute is set to False, the movement is relative. This means the cursor will move 100 pixels to the right (x-coordinate) and 100 pixels down (y-coordinate) from its current position.
  • If absolute were set to True, the cursor would move to the absolute position (100, 100) on the screen, where (0, 0) is typically the top-left corner of the screen.
• duration=0.2: This parameter specifies the duration of the movement in seconds. It's set to 0.2, meaning the cursor will take 0.2 seconds to complete the movement. This creates a smooth, animated cursor movement rather than an instantaneous jump. If the duration were set to 0, the movement would be immediate.

Making Callbacks on Mouse Events

You can also make callbacks that are called whenever an event occurs, such as a mouse click:

# make a listener when left button is clicked
mouse.on_click(lambda: print("Left Button clicked."))
# make a listener when right button is clicked
mouse.on_right_click(lambda: print("Right Button clicked."))

The above code makes simple callbacks whenever the mouse's buttons are clicked; here, we just used lambda functions for demonstration purposes; you are free to use any function to do whatever you want.

If you want to remove the listeners, you can call unhook_all() to remove all the listeners:

# remove the listeners when you want
mouse.unhook_all()

Scrolling Up and Down

You can also control the mouse wheel, let's scroll down:

# scroll down
mouse.wheel(-1)

Scrolling up:

# scroll up
mouse.wheel(1)

The mouse.wheel() function simulates scrolling the mouse wheel down or up. A negative value like -1 indicates downward scrolling by one notch, while a positive value such as mouse.wheel(5) would indicate upward scrolling by 5 notches.

Recording Mouse Events and Playing them

Finally, you can record all mouse events and then replay them:

# record until you click right
events = mouse.record()

This will record all mouse events until the right button is pressed. Then, it returns a list of events recorded, let's replay them:

# replay these events
mouse.play(events[:-1])

The reason I set events[:-1] instead of all events is that I don't want to play the right button click.

Toy Drawing Project

Before ending the tutorial, let's make a toy project that draws a square and a circle:

import mouse
import math
import time

def draw_square(size):
    # click and hold the left mouse button
    mouse.press()
    mouse.move(size, 0, absolute=False, duration=0.2)
    mouse.move(0, size, absolute=False, duration=0.2)
    mouse.move(-size, 0, absolute=False, duration=0.2)
    mouse.move(0, -size, absolute=False, duration=0.2)
    # release the left mouse button
    mouse.release()


def draw_circle(radius):
    # click and hold the left mouse button
    mouse.press()
    # move the mouse in a circle
    for i in range(0, 360, 5):
        # convert degrees to radians
        angle = math.radians(i)
        # calculate the x and y coordinates
        x = radius * math.cos(angle)
        y = radius * math.sin(angle)
        # move the mouse to the calculated position
        mouse.move(x, y, absolute=False, duration=0.01)
    # release the left mouse button
    mouse.release()

if __name__ == "__main__":
    # Place the mouse at the starting point and then call
    draw_square(200)
    time.sleep(1)
    draw_circle(10)

The output is as expected! Just make sure to open up a drawing program such as Paint, put the mouse there, and execute the code, and it'll draw a square and then a circle!

Conclusion

Alright! That's it for the tutorial. Here are some ideas you can do with this module:

• Making Reinforcement learning agents by playing video games.
• Automating the boring desktop stuff.
• Much more!

You can combine this with controlling your keyboard in Python, and let's see what you can build with these!

Finally, check the complete code here.

Learn also: How to Record your Screen in Python.

Happy Coding ♥