The block containing username/shopping cart is not being hole-punched properly, and causes private information to be publicly cached and displayed on public pages or other users' pages.
To confirm that this is the problem, you can enable debug mode and append ?LITEMAGE_DEBUG=SHOWHOLES to the end of your site URL. This will show all holes being punched on the page, where you can check if the block containing the username or cart items is being punched properly.
The simplest solution to this problem is to go to the Magento Admin Panel and navigate to System → Configuration → LiteMage Configuration and add header to Customized Block Names for “toplinks” Tag. This will punch one big hole for the entire header block. Please be aware that it is only a possible fix and it may or may not work. If it doesn't work or breaks something, you should reverse this change back.
A more complicated solution involves looking carefully at your template file for customized code, such as:
<?php if($toplinks['displayas'] == 1){echo '<span class="simple">';}
if (Mage::getSingleton('customer/session')->isLoggedIn()==1):
echo '<span class="customer-name"><strong>' . $customer->getCustomer()->getName() . '</strong></
span>';
endif;
Since LiteMage can only punch a hole on a block defined through a layout file, the dynamically generated content here will not be recognized. You can, however, create a new block containing this logic and define that block in the layout file with a unique name. Once you have done this you can then add this new block name to the Customized Block Names for “toplinks” Tag list.