It's recommended that you regularly assess your user list and remove any users that no longer require access to the system. One of the simplest ways of doing this is to remove users that have not logged in in the last [X] amount of time.

You can do this by visiting the Amin > User page, then sorting the user list by 'Last Login'.

Finally, you can opt to select some users and delete them from here.

Note, be careful not to delete any users who have just been added, but who have yet to login for the first time.