We define a "Bounce" as a visitor that comes to your site, only views one page and leaves in less than 30 seconds. This is similar to many top-end analytics services, but does differ from Google Analytics' default methodology.
Read about how and why it's different from Google Analytics here.
Bounce rate is affected more by the source of the traffic to your website than by the appearance of your site. Traffic sent from low quality sources (link exchanges, etc.) will have a drastically higher bounce rate than traffic sent from higher quality sources (search, etc.).
When a low quality source sends a spike in traffic to your site, your bounce rate will appear to increase. The increase is accredited to the poor quality traffic and not the site design. Therefore, trying to optimize your site based on bounce rate often will not yield the results you expect.