PageSpeed output resources cannot be cached in Google Cloud

[JoyceBabu]

Currently pagespeed adds Cache-Control:max-age=31536000 to optimized resources. Is it possible to add public to it?

Google CDN (alpha) requires 'Cache-Control: public' to be present for resources that must be cached.

[jmarantz]

Hi -- I'm getting some clarification from some people that know more about
the CDN.

On Fri, Mar 11, 2016 at 6:35 AM, Joyce Babu [email protected]
wrote:

Currently pagespeed adds Cache-Control:max-age=31536000 to optimized
resources. Is it possible to add public to it?

Google CDN (alpha) requires 'Cache-Control: public' to be present for
resources that must be cached.

—
Reply to this email directly or view it on GitHub
#1149.

[jmarantz]

Adding "public" to the cache-control value is a requirement for Google CDN, but is not required by the HTTP spec. https://cloud.google.com/compute/docs/load-balancing/http/cdn describes the Google requirements, which are pretty clear.

To meet these without adding an unnecessary qualifier for other caches, we can gate this logic on the presence of response-header
Via: 1.1 google

[jmarantz]

Joyce: is it possible for you to add a workaround for this problem in your nginx config? That might help you get better behavior until we can add this functionality to a PageSpeed release.

[JoyceBabu]

I tried disabling all rewriting rules, and using add_header 'Cache-Control' public;. This resulted in two Cache-Control headers

Cache-Control:max-age=604800
Cache-Control: public

Google CDN did not cache those resources, even though it is technically equivalent to Cache-Control:max-age=604800, public

To meet these without adding an unnecessary qualifier for other caches, we can gate this logic on the presence of response-header

Did you mean request-header?

Joyce: is it possible for you to add a workaround for this problem in your nginx config? That might help you get better behavior until we can add this functionality to a PageSpeed release.

I am unable to modify the Cache-Control headers from nginx config. I tried using POSITION_AUX=true, still I am unable to modify the headers. I remember seeing a comment by Jeff in another issue that nginx location blocks targeting optimized resources doesn't work when LoadFromFile is enabled.

[jmarantz]

Yes I meant the Via request-header :)

there is not an nginx equivalent to apache's "header set" directive?

On Sat, Mar 12, 2016 at 3:37 AM, Joyce Babu [email protected]
wrote:

I tried disabling all rewriting rules, and using add_header
'Cache-Control' public;. This resulted in two Cache-Control headers

Cache-Control:max-age=604800
Cache-Control: public

Google CDN did not cache those resources, even though it is technically
equivalent to Cache-Control:max-age=604800, public

To meet these without adding an unnecessary qualifier for other caches, we
can gate this logic on the presence of response-header

Did you mean request-header?

Joyce: is it possible for you to add a workaround for this problem in your
nginx config? That might help you get better behavior until we can add this
functionality to a PageSpeed release.

I am unable to modify the Cache-Control headers from nginx config. I tried
using POSITION_AUX=true, still I am unable to modify the headers. I
remember seeing a comment by Jeff in another issue that nginx location
blocks don't when LoadFromFile is enabled.

—
Reply to this email directly or view it on GitHub
#1149 (comment)
.

[JoyceBabu]

there is not an nginx equivalent to apache's "header set" directive?

Yes, there is the add_header directive and the more_set_headers directive. But from what I understand, the header filter appears before ngx_pagespeed in the nginx filter chain. Hence, headers are always intercepted and modified by ngx_pagespeed.

Setting the (experimental) environment variable POSITION_AUX=true is supposed to move ngx_pagespeed's position, so that it will be possible to modify the headers from nginx config. I just found out that it was not working because a change introduced for nginx 1.9.11+ had broken that option. After fixing it, now I am able to do more_set_headers "Cache-Control: max-age=31536000, public";

Thank you.

[jmarantz]

Glad you found a workaround, but I want to leave this open till we fix it properly.

[jmarantz]

This auto-closed when I subbmitted the partial fix: apache/incubator-pagespeed-mod@6d81e80

This fix propagates cache-control:public from origin resources to rewritten resources. However, it does not infer that we need cache-control:public when running inside GCE, which it can do with the "Via" request header, if that includes the substring 'google'.

[dhirenpratap]

It gonna help me as ..as google cdn requires public header.

[jmarantz]

You can get this today if you build in trunk, or you can wait for a release that includes this bug fix, but I don't have a date for you yet.

[dhirenpratap]

Can wait for it. Thanks for the hard work.

[justbeez]

Thanks for the effort on this @jmarantz @crowell!

We've been running into the same issues with Cloud CDN on the mod_pagespeed side and trying to come up with workaround based on where this hooks in our chain—and I'm glad to have found this issue over on the ngx_pagespeed tracker.

I see that about a month and a half ago that there wasn't a date for this to roll out yet—does anyone know if there's a window scheduled for an upcoming release? I haven't seen one in quite a while, so I'm mostly just trying to figure out if it's worth building trunk or if my laziness will pay off ;)

Thanks again all!

[crowell]

We're working on getting the release rolled out this week. We'll make an announcement on the mailing lists when it is available.

[olekstumedia]

Any update on this? We really need that new version of ngx_pagespeed module((
GoogleCDN will not work for the all servers that use pagespeed))) This is contradictorily assuming that pagespeed is also related/mainteined by Google company.

[morlovich]

New version with a fix just got released earlier today:
https://groups.google.com/forum/#!topic/ngx-pagespeed-discuss/flIW0eAdVFw

[justbeez]

Just a note for those looking for this on the mod_pagespeed site, this is available in the 1.11.33.3-beta version announced here:
https://groups.google.com/forum/#!topic/mod-pagespeed-announce/UMCmeItofcY

[nathanjosiah]

Any update on this issue? I see that a few versions have been released with a fix for this for both Apache and nginx but I am still unable to change the Cache-Control header for HTML responses. The only resource I can find about the Cache-Control header still being set is on the Configuring Downstream Caches page but this won't work for the Google Cloud CDN because the CDN doesn't allow the required configuration such as the PS-Capability header or the purge URL's.

[jmarantz]

This was fixed a long time ago in apache/incubator-pagespeed-mod@c8dc73f and was released in 1.11, which is marked as stable.

However, if you are talking about HTML responses, then this bug is not related to that at all. Note that caching HTML responses with ngx_pagespeed means you cannot enable any user-agent specific optimizations (e.g. webp transcoding). If that's acceptable, configure ModifyCachingHeaders ( https://modpagespeed.com/doc/configuration#ModifyCachingHeaders) and then NPS will just leave your headers alone.