Last-Modified HTTP header of assets contained in jar is wrong when using sbt >= 1.4.0 #10572

bchazalet · 2020-12-03T17:54:20Z

Play Version

2.7.7

API

Scala

Operating System

macosx but will affect all I believe

JDK

Seen on OpenJDK 15 and Oracle Java 8

Library Dependencies

Sbt 1.4.4

Expected Behavior

Package a play application using universal:packageZipTarball and sbt >= 1.4.0
Deploy the application
Send a GET request for one of the static asset
Inspect the last-modified response header and assert that it aligns with when you packaged the application in step 1

Actual Behavior

Package a play application using universal:packageZipTarball
Deploy the application
Send a GET request for one of the static asset, say
Inspect the last-modified response header: depending on your platform the last modified header will either be set to 1 Jan 1970 or 7 February 2106 (which known to be a common time storage issue).

We've tracked down the issue to this sbt change: sbt/sbt#5344. Basically, sbt now resets file timestamps in jars to 1970-01-01. And that date is in turn used by the play framework to set the last-modified header. You can double check by unpacking the jar yourself, jar fx your-assets-jar.jar and inspecting the dates on the filesystem.

One fix to this is to set an environment variable to the current time when building and packaging the play application, e.g.:

export SOURCE_DATE_EPOCH=`date +%s`

I don't know if this should be fixed by the framework itself, but I think it should at least be documented as it will (and did for us) cause cache issues.

The text was updated successfully, but these errors were encountered:

Fixes sbt#6235 In sbt 1.4.0 (sbt#5344) we started wiping out the timestamps in JAR to make the builds more repeatable. This had an unintended consequence of breaking Play's last-modified response header (playframework/playframework#10572). This adds an opt-out from the default: ```scala Compile / packageBin / packageOptions += Package.keepTimestamps ``` ### before ``` $ ll example total 32 -rw-r--r-- 1 eed3si9n wheel 901 Jan 1 1970 Greeting.class -rw-r--r-- 1 eed3si9n wheel 3079 Jan 1 1970 Hello$.class -rw-r--r-- 1 eed3si9n wheel 738 Jan 1 1970 Hello$delayedInit$body.class -rw-r--r-- 1 eed3si9n wheel 875 Jan 1 1970 Hello.class ``` ### after ``` $ ll target/scala-2.13/hello_2.13-0.1.0-SNAPSHOT/example total 32 -rwxr-xr-x 1 eed3si9n wheel 901 Jan 3 12:20 Greeting.class* -rwxr-xr-x 1 eed3si9n wheel 3079 Jan 3 12:20 Hello$.class* -rwxr-xr-x 1 eed3si9n wheel 738 Jan 3 12:20 Hello$delayedInit$body.class* -rwxr-xr-x 1 eed3si9n wheel 875 Jan 3 12:20 Hello.class* ```

huntc · 2021-01-03T20:55:47Z

To add some context, I think it was me that provided this functionality in Play many moons ago. Assets can of course be provided as Webjars. The date of the webjar is therefore important. My memory on this is vague though.

…mmitDate Fixes sbt#6235 In sbt 1.4.0 (sbt#5344) we started wiping out the timestamps in JAR to make the builds more repeatable. This had an unintended consequence of breaking Play's last-modified response header (playframework/playframework#10572). This adds an opt-out from the default: ```scala ThisBuild / packageOptions += Package.keepTimestamps ``` Before ------ ``` $ ll example total 32 -rw-r--r-- 1 eed3si9n wheel 901 Jan 1 1970 Greeting.class -rw-r--r-- 1 eed3si9n wheel 3079 Jan 1 1970 Hello$.class -rw-r--r-- 1 eed3si9n wheel 738 Jan 1 1970 Hello$delayedInit$body.class -rw-r--r-- 1 eed3si9n wheel 875 Jan 1 1970 Hello.class ``` After ----- ``` $ ll target/scala-2.13/hello_2.13-0.1.0-SNAPSHOT/example total 32 -rwxr-xr-x 1 eed3si9n wheel 901 Jan 3 12:20 Greeting.class* -rwxr-xr-x 1 eed3si9n wheel 3079 Jan 3 12:20 Hello$.class* -rwxr-xr-x 1 eed3si9n wheel 738 Jan 3 12:20 Hello$delayedInit$body.class* -rwxr-xr-x 1 eed3si9n wheel 875 Jan 3 12:20 Hello.class* ```

Fixes sbt#6235 In sbt 1.4.0 (sbt#5344) we started wiping out the timestamps in JAR to make the builds more repeatable. This had an unintended consequence of breaking Play's last-modified response header (playframework/playframework#10572). This adds a global setting called `packageTimestamp`, which is initialized as follows: ```scala packageTimestamp :== Package.defaultTimestamp, ``` Here the `Package.defaultTimestamp` would pick either the value from the `SOURCE_DATE_EPOCH` environment variable or 2010-01-01. To opt out of this default, the user can use: ```scala ThisBuild / packageTimestamp := Package.keepTimestamps // or ThisBuild / packageTimestamp := Package.gitCommitDateTimestamp ``` Before (sbt 1.4.6) ------------------ ``` $ ll example total 32 -rw-r--r-- 1 eed3si9n wheel 901 Jan 1 1970 Greeting.class -rw-r--r-- 1 eed3si9n wheel 3079 Jan 1 1970 Hello$.class -rw-r--r-- 1 eed3si9n wheel 738 Jan 1 1970 Hello$delayedInit$body.class -rw-r--r-- 1 eed3si9n wheel 875 Jan 1 1970 Hello.class ``` After (using Package.gitCommitDateTimestamp) -------------------------------------------- ``` $ unzip -v target/scala-2.13/root_2.13-0.1.0-SNAPSHOT.jar Archive: target/scala-2.13/root_2.13-0.1.0-SNAPSHOT.jar Length Method Size Cmpr Date Time CRC-32 Name -------- ------ ------- ---- ---------- ----- -------- ---- 288 Defl:N 136 53% 01-25-2021 03:09 888682a9 META-INF/MANIFEST.MF 0 Stored 0 0% 01-25-2021 03:09 00000000 example/ 901 Defl:N 601 33% 01-25-2021 03:09 3543f377 example/Greeting.class 3079 Defl:N 1279 59% 01-25-2021 03:09 848b4386 example/Hello$.class 738 Defl:N 464 37% 01-25-2021 03:09 571f4288 example/Hello$delayedInit$body.class 875 Defl:N 594 32% 01-25-2021 03:09 ad295259 example/Hello.class -------- ------- --- ------- 5881 3074 48% 6 files ```

mkurz · 2021-02-22T23:20:52Z

sbt 1.5 makes this configurable:
https://github.com/sbt/sbt/releases/tag/v1.5.0-M2 (see heading "ThisBuild / packageTimestamp setting)
I think we should at least document this.

strelec · 2021-03-21T16:36:16Z

Ok,

if this is the case, then let's stop setting the Last-Modified header on assets - obviously it is not correct and will mess up caching.

This header is AFAIK not mandatory, and we are sending ETag as an alterative.

Since SBT 1.4.0, SBT has edited the last modified dates in packaged JARs, which include web assets (css, js, etc.); Grid's current version sets the date to 1 Jan 2010. Play framework uses these dates to calculate the ETag and Last-Modified headers. Since the headers are now based upon a static date, caching (especially via Cloudfront) is broken, returning outdated assets. This commit changes the default to set the timestamp to match the latest git commit date, so cloudfront should now notice and pick up new assets upon deployment. See also <playframework/playframework#10572> See also <sbt/sbt#6237>

Since SBT 1.4.0, SBT has edited the last modified dates in packaged JARs, which include web assets (css, js, etc.); Grid's current version sets the date to 1 Jan 2010. Play framework uses these dates to calculate the ETag and Last-Modified headers. Since the headers are now based upon a static date, caching (especially via Cloudfront) is broken, returning outdated assets. This commit restores the previous behaviour, so cloudfront should now notice and pick up new assets upon deployment. See also <playframework/playframework#10572> See also <sbt/sbt#6237>

Since SBT 1.4.0, SBT has edited the last modified dates in packaged JARs, which include web assets (css, js, etc.); our current version sets the date to 1 Jan 2010. Play framework uses these dates to calculate the ETag and Last-Modified headers. Since the headers are now based upon a static date, caching (especially via Cloudfront) is broken, returning outdated assets. This commit restores the previous behaviour, so cloudfront should now notice and pick up new assets upon deployment. See also <playframework/playframework#10572> See also <sbt/sbt#6237>

Last-Modified HTTP header of assets contained in jar is wrong See playframework/playframework#10572

RommelTJ · 2024-10-04T15:25:51Z

I'm looking into this. @mkurz, would this be brand new documentation? Or adding to an existing page? Any suggestion on where this documentation should go?

RommelTJ · 2024-10-04T20:18:04Z

I've been looking at updating the documentation for this. I was thinking of changing it here: https://www.playframework.com/documentation/3.0.x/Deploying#The-Native-Packager

However, the suggested workaround from sbt is not working for me. Even after modifying my build.sbt as follows:

val currentTime = System.currentTimeMillis()
ThisBuild / packageTimestamp := Some(currentTime)
Universal / packageBin / packageTimestamp := (ThisBuild / packageTimestamp).value
Compile / packageBin / packageTimestamp := (ThisBuild / packageTimestamp).value
Assets / packageBin / packageTimestamp := (ThisBuild / packageTimestamp).value

I can confirm sbt will use the specified time, at least with sbt 1.8.2 on macOS Seqoia 15.0.1, but the assets still have the default time:

jar tvf lib/dummyproject.dummyproject-1.0-SNAPSHOT-assets.jar | grep -i 'favicon'
   687 Fri Jan 01 00:00:00 PST 2010 public/images/favicon.png

And I get these headers:

HTTP/1.1 200 OK
ETag: "fff49652dc44c91aedff86cf926c5558669a0570"
Accept-Ranges: bytes
Cache-Control: public, max-age=3600
Last-Modified: Fri, 01 Jan 2010 08:00:00 GMT
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Permitted-Cross-Domain-Policies: master-only
Date: Fri, 04 Oct 2024 20:17:15 GMT
Content-Type: image/png
Content-Length: 687

Therefore, I don't want to submit a PR with updated documentation for a workaround that doesn't work. Has anyone gotten this to work?

I also tried setting an environment variable like @bchazalet explained, but I still see the same behavior.

Edit: Nevermind, this works: ThisBuild / packageOptions += Package.FixedTimestamp(Package.keepTimestamps)

Resolves playframework#10572

RommelTJ · 2024-10-04T21:11:32Z

Adding documentation with #12895

bchazalet changed the title ~~last-modified of assets contained in jar is wrong when using sbt >= 1.4.0~~ last-modified HTTP header of assets contained in jar is wrong when using sbt >= 1.4.0 Dec 3, 2020

bchazalet changed the title ~~last-modified HTTP header of assets contained in jar is wrong when using sbt >= 1.4.0~~ Last-Modified HTTP header of assets contained in jar is wrong when using sbt >= 1.4.0 Dec 3, 2020

dontgitit mentioned this issue Jan 1, 2021

Reset timestamps to 1970-01-01 during packaging for repeatable builds sbt/sbt#5344

Merged

eed3si9n mentioned this issue Jan 1, 2021

Make it possible to opt out of reproducible builds (1970-01-01 timestamps) sbt/sbt#6235

Closed

eed3si9n mentioned this issue Jan 3, 2021

packageTimestamp setting sbt/sbt#6237

Merged

andrew-nowak mentioned this issue Jan 14, 2022

Set timestamp in output jars from SBT guardian/grid#3600

Merged

4 tasks

This was referenced Jan 17, 2022

Set timestamp in output jars from SBT guardian/facia-tool#1412

Closed

Set timestamp in output jars from SBT guardian/workflow-frontend#355

Closed

andrew-nowak mentioned this issue Jan 17, 2022

Set timestamp in output jars from SBT guardian/s3-upload#38

Closed

4 tasks

SunPj added a commit to SunPj/play-scala-react-seed that referenced this issue Feb 12, 2024

Fixes "Main page caching doesn't work properly"

82605a6

Last-Modified HTTP header of assets contained in jar is wrong See playframework/playframework#10572

SunPj mentioned this issue Feb 12, 2024

Fixes "Main page caching doesn't work properly" playframework/play-scala-react-seed#192

Merged

fm3 mentioned this issue Aug 20, 2024

keep asset timestamps from filesystem when building jar scalableminds/webknossos#8015

Merged

mkurz added the topic:documentation label Oct 3, 2024

RommelTJ pushed a commit to RommelTJ/playframework that referenced this issue Oct 4, 2024

Asset timestamp handling with sbt

0594138

Resolves playframework#10572

RommelTJ linked a pull request Oct 4, 2024 that will close this issue

Documentation Update: Asset timestamp handling with sbt #12895

Open

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Last-Modified HTTP header of assets contained in jar is wrong when using sbt >= 1.4.0 #10572

Last-Modified HTTP header of assets contained in jar is wrong when using sbt >= 1.4.0 #10572

bchazalet commented Dec 3, 2020 •

edited

Loading

huntc commented Jan 3, 2021

mkurz commented Feb 22, 2021 •

edited

Loading

strelec commented Mar 21, 2021

RommelTJ commented Oct 4, 2024

RommelTJ commented Oct 4, 2024 •

edited

Loading

RommelTJ commented Oct 4, 2024

Last-Modified HTTP header of assets contained in jar is wrong when using sbt >= 1.4.0 #10572

Last-Modified HTTP header of assets contained in jar is wrong when using sbt >= 1.4.0 #10572

Comments

bchazalet commented Dec 3, 2020 • edited Loading

Play Version

API

Operating System

JDK

Library Dependencies

Expected Behavior

Actual Behavior

huntc commented Jan 3, 2021

mkurz commented Feb 22, 2021 • edited Loading

strelec commented Mar 21, 2021

RommelTJ commented Oct 4, 2024

RommelTJ commented Oct 4, 2024 • edited Loading

RommelTJ commented Oct 4, 2024

bchazalet commented Dec 3, 2020 •

edited

Loading

mkurz commented Feb 22, 2021 •

edited

Loading

RommelTJ commented Oct 4, 2024 •

edited

Loading