There is a Manage Site Collection Users page for the site collection as a whole for Site Owners to delete users from. Deleting users from this page reduces the quota so that new users can be invited. For the site owner, the way to delete accounts is to do the following:
Site Settings, Go to Site Administration, View Site Collection User Information. On this page if you delete an account it deletes the account and removes it from all sub sites.
Using the Manage Users page only removes the account from the Site Groups (roles) for that site.