On the homepage and the browse pages, folders/access levels or categories are displayed with a generic folder icon; we recommend, where possible, to show a relevant thumbnail image instead.

Navigate to Admin > Folders/Categories and click edit next to one of the folders; you will see an 'Add image' field - simply use this to specify a thumbnail for that category. 

If you then navigate back to browse you should be able to see that that category is now shown next to the thumbnail image you selected.

If you are replacing an image and do not see the updated version this is usually a browser cache issue.
In this case, try 'hard refreshing' your browser using the hotkey "CTRL+F5" (Windows) or "CMD+R" (Mac).

Technical Notes for our self-hosted customers: 

What if I always want all categories at every level to be shown with a folder icon instead of bullets?

As of version 3.1478 and above their is a new setting:

always-show-icons-instead-of-bullets-for-categories=true

Simply set this to true to always show icons instead of bullets. Note that this also applies to folders, and you can continue to specify a thumbnail for any category which will override the folder icon. 

Advanced configuration

There are a few further settings related to how categories and their thumbnails are displayed: 

# Thumbnails for category images:
category-image-height=40
category-image-width=40

These control the size of the thumbnails displayed next to categories. Note that increasing these will nearly always require manual updates to your css to cope with the new sizes.

Having text appear below larger homepage images

show-category-image-above-link=true

Set this to true to show the thumbnail above the category name instead of to the left of it. This works best when you increase the thumbnail size using the settings above. Again, this generally necessitates changes to your css to cope.

If set to true then set the following,

# Thumbnails for category images:
category-image-height=180
category-image-width=240