Basically, the bounce rate is the rate of people going to your site, and leaving after only viewing one page of it. Exit rate on the other hand, will be people who went to your site, browsed around, and left on that page.
A high exit rate on one page could mean that it's simply the end of your site (in which case it's a natural exit), or be a sign that the page in specific needs work on to keep your visitors interested.
A high bounce rate will mean that the page they first come to (likely your index page) is not good enough to make them wish to learn more.
