There's no really simple and easy way.
You can go through all Styles in the Object Style Manager dialog one by one and get rid of the ones you don't want. If the style you are trying to delete is used by objects in the project it will prompt you to specify which of the other styles those objects should be assigned to.
Another way to find all of the styles that are actually used is to copy and paste all slides from your current project into a new blank project and then take a look at the Object Style Manager to see which ones OTHER THAN the default styles show up. Only styles that are in use will be copied over.