OK, that did not fix it.
Below is code that does fix it when put into the .htaccess but I'm not sure if this will interfere with outher potential cache features.
# Stop caching the cart
RewriteEngine On
<FilesMatch "\.(html|htm|php)$">
FileETag None
<IfModule mod_headers.c>
Header unset ETag
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires "Wed, 12 Jan 1980 05:00:00 GMT"
</IfModule>
</FilesMatch>