unexpected EOF on publishing mongodb snapshot

[smarmie]

When publishing the snapshot of the mongodb mirror, aptly panics with unexpected EOF on package /mirror/aptly/pool/b9/13/mongodb-10gen-unstable_2.5.1_amd64.deb

This looks like #440, but I don't know if it's the same cause.

Using aptly 1.0.0 (but also saw this on 0.9.7) on debian 8.7

apt-mirror@apt-mirror:~$ aptly mirror drop mongodb 
Mirror `mongodb` has been removed.
apt-mirror@apt-mirror:~$ aptly db cleanup
Loading mirrors, local repos, snapshots and published repos...
Loading list of all packages...
Deleting unreferenced packages (279)...
Building list of files referenced by packages...
Building list of files in package pool...
Deleting unreferenced files (265)...
Disk space freed: 6.54 GiB...
Compacting database...
apt-mirror@apt-mirror:~$ aptly mirror create mongodb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/InRelease...
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/Release...
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/Release.gpg...
gpgv: Signature made Wed 23 Mar 2016 18:34:48 UTC using RSA key ID 7F0CEB10
gpgv: Good signature from "Richard Kreuter <[email protected]>"

Mirror [mongodb]: http://downloads-distro.mongodb.org/repo/debian-sysvinit/ dist successfully added.
You can run 'aptly mirror update mongodb' to download repository contents.
apt-mirror@apt-mirror:~$ aptly mirror update mongodb
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/InRelease...
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/Release...
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/Release.gpg...
gpgv: Signature made Wed 23 Mar 2016 18:34:48 UTC using RSA key ID 7F0CEB10
gpgv: Good signature from "Richard Kreuter <[email protected]>"
Downloading & parsing package files...
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/10gen/binary-amd64/Packages.gz...
Building download queue...
Download queue: 215 items (5.81 GiB)
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/10gen/binary-amd64/mongodb-org-server_2.6.11_amd64.deb...
.... downloading lots of packages ...
Downloading http://downloads-distro.mongodb.org/repo/debian-sysvinit/dists/dist/10gen/binary-amd64/mongodb-mms-backup-agent_2.4.0.156-1_amd64.deb...

Mirror `mongodb` has been successfully updated.
apt-mirror@apt-mirror:~$ aptly snapshot create mongodb-2017032901 from mirror mongodb 

Snapshot mongodb-2017032901 successfully created.
You can run 'aptly publish snapshot mongodb-2017032901' to publish snapshot as Debian repository.
apt-mirror@apt-mirror:~$ aptly publish snapshot mongodb-2017032901
Loading packages...
Generating metadata files and linking package files...
panic: unable to read .tar archive from /mirror/aptly/pool/b9/13/mongodb-10gen-unstable_2.5.1_amd64.deb: unexpected EOF [recovered]
	panic: unable to read .tar archive from /mirror/aptly/pool/b9/13/mongodb-10gen-unstable_2.5.1_amd64.deb: unexpected EOF

goroutine 1 [running]:
github.com/smira/aptly/cmd.Run.func1(0xc42067bf70)
	/Users/smira/Documents/go/src/github.com/smira/aptly/cmd/run.go:17 +0x167
panic(0xb458c0, 0xc4202af7a0)
	/usr/local/Cellar/go/1.8/libexec/src/runtime/panic.go:489 +0x2cf
github.com/smira/aptly/deb.(*Package).CalculateContents(0xc4200becc0, 0xfcf400, 0xc420ac4460, 0xc4200131b0, 0x1, 0x0)
	/Users/smira/Documents/go/src/github.com/smira/aptly/deb/package.go:428 +0x191
github.com/smira/aptly/deb.(*PackageCollection).loadContents(0xc4204c39e0, 0xc4200becc0, 0xfcf400, 0xc420ac4460, 0x1, 0x0, 0xc42067ac88)
	/Users/smira/Documents/go/src/github.com/smira/aptly/deb/package_collection.go:184 +0x17a
github.com/smira/aptly/deb.(*Package).Contents(0xc4200becc0, 0xfcf400, 0xc420ac4460, 0xb, 0xc4202aff90, 0xb)
	/Users/smira/Documents/go/src/github.com/smira/aptly/deb/package.go:411 +0x7c
github.com/smira/aptly/deb.(*ContentsIndex).Push(0xc4202300f0, 0xc4200becc0, 0xfcf400, 0xc420ac4460, 0xc420137dc8, 0xc4202aff90)
	/Users/smira/Documents/go/src/github.com/smira/aptly/deb/contents.go:30 +0x5a
github.com/smira/aptly/deb.(*PublishedRepo).Publish.func1(0xc4200becc0, 0x0, 0x0)
	/Users/smira/Documents/go/src/github.com/smira/aptly/deb/publish.go:577 +0x798
github.com/smira/aptly/deb.(*PackageList).ForEachIndexed(0xc4203f9080, 0xc42067b6e0, 0xc420381aa0, 0x0)
	/Users/smira/Documents/go/src/github.com/smira/aptly/deb/list.go:171 +0x69
github.com/smira/aptly/deb.(*PublishedRepo).Publish(0xc420137860, 0xfcf400, 0xc420ac4460, 0xfc63c0, 0xc4201af810, 0xc42000e300, 0xfd1020, 0xc420055b60, 0xfd18c0, 0xc420aca150, ...)
	/Users/smira/Documents/go/src/github.com/smira/aptly/deb/publish.go:602 +0xd0e
github.com/smira/aptly/cmd.aptlyPublishSnapshotOrRepo(0xc4201c8b40, 0xc4201caee0, 0x1, 0x2, 0xfc7c80, 0xc42016d4c0)
	/Users/smira/Documents/go/src/github.com/smira/aptly/cmd/publish_snapshot.go:146 +0xeac
github.com/smira/aptly/vendor/github.com/smira/commander.(*Command).Dispatch(0xc4201c8b40, 0xc4201caee0, 0x1, 0x2, 0x1, 0xc4201efe38)
	/Users/smira/Documents/go/src/github.com/smira/aptly/vendor/github.com/smira/commander/commands.go:305 +0x243
github.com/smira/aptly/vendor/github.com/smira/commander.(*Command).Dispatch(0xc4201c8fc0, 0xc4201caed0, 0x2, 0x3, 0xc4201efe01, 0xc4200001a0)
	/Users/smira/Documents/go/src/github.com/smira/aptly/vendor/github.com/smira/commander/commands.go:283 +0x48b
github.com/smira/aptly/vendor/github.com/smira/commander.(*Command).Dispatch(0xc4201c97a0, 0xc4201caec0, 0x3, 0x4, 0xc4201cf6c0, 0xc4201caec0)
	/Users/smira/Documents/go/src/github.com/smira/aptly/vendor/github.com/smira/commander/commands.go:283 +0x48b
github.com/smira/aptly/cmd.Run(0xc4201c97a0, 0xc42000e190, 0x3, 0x3, 0xc420000101, 0x0)
	/Users/smira/Documents/go/src/github.com/smira/aptly/cmd/run.go:41 +0x12a
main.main()
	/Users/smira/Documents/go/src/github.com/smira/aptly/main.go:20 +0xda

[smira]

confirmed, thanks for the report!

[smira]

Looks like really broken archive though:

$ dpkg -c /Users/smira/.aptly/pool/b9/13/mongodb-10gen-unstable_2.5.1_amd64.deb
drwxr-xr-x root/root         0 2013-07-17 23:30 ./
drwxr-xr-x root/root         0 2013-07-17 23:30 ./var/
drwxr-xr-x root/root         0 2013-07-17 23:30 ./var/lib/
drwxr-xr-x root/root         0 2013-07-17 23:30 ./var/lib/mongodb/
drwxr-xr-x root/root         0 2013-07-17 23:30 ./usr/
drwxr-xr-x root/root         0 2013-07-17 23:30 ./usr/sbin/
drwxr-xr-x root/root         0 2013-07-17 23:30 ./usr/bin/
-rwxr-xr-x root/root  19721672 2013-07-17 23:30 ./usr/bin/mongoperf
-rwxr-xr-x root/root  19723912 2013-07-17 23:30 ./usr/bin/mongooplog
-rwxr-xr-x root/root   9662888 2013-07-17 23:30 ./usr/bin/mongo
-rwxr-xr-x root/root  19750056 2013-07-17 23:30 ./usr/bin/mongoimport
-rwxr-xr-x root/root  19718856 2013-07-17 23:30 ./usr/bin/bsondump
-rwxr-xr-x root/root  19720328 2013-07-17 23:30 ./usr/bin/mongotop
-rwxr-xr-x root/root  19787176 2013-07-17 23:30 ./usr/bin/mongorestore
dpkg-deb (subprocess): cannot copy archive member from '/Users/smira/.aptly/pool/b9/13/mongodb-10gen-unstable_2.5.1_amd64.deb' to decompressor pipe: unexpected end of file or stream
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
dpkg-deb: error: subprocess tar returned error exit status 2