Redirection lambda function regex missing .map, .json and whatsapp

[YuriScarbaci]

The lambda testing wheter to set the header for prerender.io or not isn't currently testing:

1. user-agent for whatsapp crawler ( should be something like WhatsApp/2.19.81 A )
2. .json files
3. .map files

[YuriScarbaci]

this can be fixed by modifying the lambda function for SetPrerenderHeader

this is the part that needs to be changed:

    if (user_agent && host) {
      var prerender = /googlebot|adsbot\-google|Feedfetcher\-Google|bingbot|yandex|baiduspider|Facebot|facebookexternalhit|twitterbot|WhatsApp|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator/i.test(user_agent[0].value);
      prerender = prerender || /_escaped_fragment_/.test(request.querystring);
      prerender = prerender && ! /\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot|map|json)$/i.test(request.uri);
      if (prerender) {
        headers['x-prerender-token'] = [{ key: 'X-Prerender-Token', value: '${PrerenderToken}'}];
        headers['x-prerender-host'] = [{ key: 'X-Prerender-Host', value: host[0].value}];
        headers['x-prerender-cachebuster'] = [{ key: 'X-Prerender-Cachebuster', value: Date.now().toString()}];
      }
    }

so basically adding |WhatsApp in the first regex

and

|map|json to the third regex

[jinty]

Hi Yuri, Thanks for the info. The change looks good. Would you be able to make a pull request with it? Regards, Brian

…

On Thu, 31 Oct 2019 at 18:49, Yuri Scarbaci ***@***.***> wrote: this can be fixed by modifying the lambda function for SetPrerenderHeader this is the part that needs to be changed: if (user_agent && host) { var prerender = /googlebot|adsbot\-google|Feedfetcher\-Google|bingbot|yandex|baiduspider|Facebot|facebookexternalhit|twitterbot|WhatsApp|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator/i.test(user_agent[0].value); prerender = prerender || /_escaped_fragment_/.test(request.querystring); prerender = prerender && ! /\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot|map|json)$/i.test(request.uri); if (prerender) { headers['x-prerender-token'] = [{ key: 'X-Prerender-Token', value: 'ZOPzswpcTJUhqRXNX2FC'}]; headers['x-prerender-host'] = [{ key: 'X-Prerender-Host', value: host[0].value}]; headers['x-prerender-cachebuster'] = [{ key: 'X-Prerender-Cachebuster', value: Date.now().toString()}]; } } so basically adding |WhatsApp in the first regex and |map|json to the third regex — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#14?email_source=notifications&email_token=AAEHDGNWDLCPMLUFA24MCCDQRMEAHA5CNFSM4JHNAJM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECYPDYQ#issuecomment-548467170>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEHDGIHC5EPJGZFJWAUP63QRMEAHANCNFSM4JHNAJMQ> .

[YuriScarbaci]

I'm not sure enough about the comment in the lambda function saying that the version is to be changed below. Is it enough for me to change what I stated or should I change something more in the cloud formation yaml? On Thu, Oct 31, 2019, 19:48 Brian Sutherland <[email protected]> wrote:

…

Hi Yuri, Thanks for the info. The change looks good. Would you be able to make a pull request with it? Regards, Brian On Thu, 31 Oct 2019 at 18:49, Yuri Scarbaci ***@***.***> wrote: > this can be fixed by modifying the lambda function for SetPrerenderHeader > > this is the part that needs to be changed: > > if (user_agent && host) { > var prerender = /googlebot|adsbot\-google|Feedfetcher\-Google|bingbot|yandex|baiduspider|Facebot|facebookexternalhit|twitterbot|WhatsApp|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator/i.test(user_agent[0].value); > prerender = prerender || /_escaped_fragment_/.test(request.querystring); > prerender = prerender && ! /\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot|map|json)$/i.test(request.uri); > if (prerender) { > headers['x-prerender-token'] = [{ key: 'X-Prerender-Token', value: 'ZOPzswpcTJUhqRXNX2FC'}]; > headers['x-prerender-host'] = [{ key: 'X-Prerender-Host', value: host[0].value}]; > headers['x-prerender-cachebuster'] = [{ key: 'X-Prerender-Cachebuster', value: Date.now().toString()}]; > } > } > > so basically adding |WhatsApp in the first regex > > and > > |map|json to the third regex > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > < #14?email_source=notifications&email_token=AAEHDGNWDLCPMLUFA24MCCDQRMEAHA5CNFSM4JHNAJM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECYPDYQ#issuecomment-548467170 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAEHDGIHC5EPJGZFJWAUP63QRMEAHANCNFSM4JHNAJMQ > > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#14?email_source=notifications&email_token=ACQWZVWXBLXVXKH55WV6LODQRMR6HA5CNFSM4JHNAJM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECY3ONI#issuecomment-548517685>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACQWZVRG3CEHLICE4QEGY6TQRMR6HANCNFSM4JHNAJMQ> .