APO caches crawler and bot traffic again
Fixed a regression in Automatic Platform Optimization that prevented caching of HTML requests without explicit Accept: text/html headers, commonly from crawlers, bots, and monitors. These requests are now cached properly without requiring workarounds.
We fixed a regression where Automatic Platform Optimization (APO) stopped caching some HTML requests that did not send an explicit Accept: text/html header — commonly crawlers, bots, and uptime monitors. These requests were being served from your origin (cf-cache-status: DYNAMIC) instead of the cache.
APO now caches these requests again. No action is needed. If you added a Transform Rule to set Accept: text/html as a workaround, you can remove it.
For details on how APO decides what to cache, refer to About APO.
Source: original entry ↗