Support prune in Bun workspaces

[joshunrau]

Which project is this feature idea for?

Turborepo

Describe the feature you'd like to request

Support turbo prune when using bun.

Describe the solution you'd like

Running turbo prune should work with bun the same as with npm.

Describe alternatives you've considered

Not using bun or not using turbo

[nathanhammond]

We're aware of this limitation. Serializing to the binary Bun lockfile format is a non-trivial task and not imminently forthcoming.

For now, if you want to use turbo prune for a Bun project, you can losslessly use it with either yarn or pnpm. We recommend choosing yarn because the node_modules layout is most-similar to Bun and it is more-likely to be compatible with your project.

cd ./path/to/bun/project
bunx @turbo/workspaces@latest convert ./ yarn --skip-install
turbo prune

If you don't want any of the automated changes to be permanent you can then proceed to revert all changes in the repository.

Note that the output pruned repository must use the package manager you select (yarn or pnpm)—it cannot use bun.

[jordanebelanger]

Note that the output pruned repository must use the package manager you select (yarn or pnpm)—it cannot use bun.

Doesn't bun support the yarn lockfile format? Why the need to switch package manager even after converting a bun.lockb to a yarn lockfile?

P.S would love for this issue to get attention, this is the last blocker for us to move over to bun as our package manager.

[chris-olszewski]

Bun supports serializing the bun.lockb to a yarn.lock to make it human readable, but it doesn't support using parsing a yarn.lock or creating a bun.lockb from a yarn.lock.

There's some chatter around bun getting support for using a package-lock.json and that could open a path forward to turbo prune supporting bun.

[wladpaiva]

Bun supports a human readable lock file now
https://bun.sh/guides/install/yarnlock

Maybe an easy fix would be to have an option for turbo to read specific lock files the same way bun does
turbo prune --yarn

[jukkamarttinen]

This would be the best option, if possible.

[thecrabcode]

Is there any update on this guys? We would also like to use Turborepo alongside bun, but this issue is quite important for us. I'd be happy to help in any way I can

[AirP0WeR]

+1 to this problem

[baptisteArno]

YES please 🙏

[XenoWad01]

I am also very interested in this being supported. My team would love to use bun alongside turbo but this is the main bottleneck preventing us from doing so.

[jukkamarttinen]

This would be lot appreciated for us too.

[mtsewrs]

This is the only thing blocking my company from switching to bun

[XenoWad01]

I am unsure if this actually fixed the problem as I didnt have time to test that the lock file works. But what seemed to have worked for me was doing turbo prune --scope=@bookd/bookd-api --docker; exit 0 which basically ignores the error alltogether. Im not sure if it works properly or not but it does generate an out/ folder properly. It does generate an out folder but I suspect it just contains the non pruned version of the lock file. Here is my docker file for reference:

FROM node:20.11.0 AS init


# RUN npm install -g unzip
RUN npm install -g bun

RUN bun upgrade --canary

RUN npm install -g synp
RUN npm install -g [email protected]


WORKDIR /usr/src/app
COPY . .

RUN turbo prune --scope=@bookd/bookd-api --docker; exit 0

FROM init AS installer

WORKDIR /usr/src/app


COPY --chown=node .gitignore .gitignore
COPY --from=init --chown=node /usr/src/app/out/json/ ./



COPY --from=init --chown=node /usr/src/app/out/full/ ./

RUN rm -rf /usr/src/app/out/
RUN bun install --frozen-lockfile --production
RUN bun run build

ENV PORT=4000

EXPOSE $PORT

WORKDIR /usr/src/app/packages/bookd-api

RUN chown node /usr/src/app/* && chmod 755 /usr/src/app/*

USER node
CMD bun run /usr/src/app/packages/bookd-api/app.ts

(Dont mind the npm stuf it seems bun install -g does not really work inside docker for me because the it doesent seem to add the commands to path inside docker only)

From what I understand the reason for this not being supported is that prune actually generates a new lockfile from the old one with only a certain package's dependencies in it and it cannot currently parse the .lockb format. Bun currently supports installing from lockb (and from package-lock.json if you are on canary as of this moment). Here is the open issue related to it supporting yarn.lock file: oven-sh/bun#6409. As soon as they support installing from a yarn.lock file this should become a non problem because bun install can generate a yarn.lock file with bun install -y or adding [install.lockfile] print = "yarn" to your bunfig.toml file at the root of your project. In theory you could also use https://github.com/imsnif/synp for converting the yarn.lock file to a package-lock.json file. Turbo seems to auto-detect the type of lockfile you have and will whine about you having multiple ones. What could fix this is a argument for turbo or turbo prune to specify the exact lockfile to use (as mentioned here #7456 (comment)). The current workaround proposed by me is that you just delete the lock files you dont need inside docker.

TLDR:

(be on bun canary) => install with bun install -y always or add the [install.lockfile] print = "yarn" to your bunfig.toml file => (inside docker) => install synp => delete .lockb file => convert yarn.lock to package-lock.json with snyp => remove old yarn.lock => you currently have a package-lock.json and if you are on canary you can install from it with bun.

I didnt yet have time to fully test this theory but if someone has time and manages to do so please let me know I would appreciate it alot. Otherwise I will try to come back to you when I properly pull this off ( which will take some time as I am new to docker and devops in general ).

[luksch42]

@XenoWad01 any updates?

[XenoWad01]

@terion-name Yea I am also confused. Here is my output I just ran docker build --pull -t trybun . &> build.log. I used the exact same dockerfile as above. And we got the same turbo version too ... I will look around again tomorrow if I get time.

#0 building with "default" instance using docker driver

#1 [internal] load .dockerignore
#1 transferring context:
#1 transferring context: 152B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 751B done
#2 DONE 0.1s

#3 [internal] load metadata for docker.io/oven/bun:1.1.17
#3 DONE 0.8s

#4 [base 1/1] FROM docker.io/oven/bun:1.1.17@sha256:7b5c05b562efbbec704df426a9eef18f3b935bdc868678895a935bf120bda062
#4 DONE 0.0s

#5 [internal] load build context
#5 transferring context: 901.47kB 0.5s done
#5 DONE 0.5s

#6 [pruned 1/5] RUN bun add -g [email protected]
#6 CACHED

#7 [pruned 2/5] WORKDIR /usr/src/app
#7 CACHED

#8 [pruned 3/5] COPY . .
#8 DONE 1.1s

#9 [pruned 4/5] RUN rm ./bun.lockb
#9 DONE 0.6s

#10 [pruned 5/5] RUN bun run turbo prune --scope=@bookd/bookd-api --docker
#10 0.791 
#10 0.791 Attention:
#10 0.791 Turborepo now collects completely anonymous telemetry regarding usage.
#10 0.791 This information is used to shape the Turborepo roadmap and prioritize features.
#10 0.791 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
#10 0.791 https://turbo.build/repo/docs/telemetry
#10 0.791 
#10 0.948 Generating pruned monorepo for @bookd/bookd-api in /usr/src/app/out
#10 0.978  - Added @bookd/bookd-api
#10 DONE 1.5s

#11 [installer 1/7] WORKDIR /usr/src/app
#11 DONE 0.2s

#12 [installer 2/7] COPY --chown=bun .gitignore .gitignore
#12 DONE 0.2s

#13 [installer 3/7] COPY --from=pruned --chown=bun /usr/src/app/out/json/ ./
#13 DONE 0.2s

#14 [installer 4/7] COPY --from=pruned --chown=bun /usr/src/app/out/full/ ./
#14 DONE 0.3s

#15 [installer 5/7] COPY --from=pruned --chown=bun /usr/src/app/out/yarn.lock ./yarn.lock
#15 DONE 0.2s

#16 [installer 6/7] RUN rm -rf /usr/src/app/out/
#16 DONE 0.6s

#17 [installer 7/7] RUN bun install --frozen-lockfile --production
#17 0.676 bun install v1.1.17 (bb66bba1)
#17 0.688 Resolving dependencies
#17 21.48 Resolved, downloaded and extracted [6547]
#17 21.48 warn: incorrect peer dependency "[email protected]"
#17 21.48 
#17 21.48 warn: incorrect peer dependency "[email protected]"
#17 27.14 Saved yarn.lock
#17 27.16 
#17 27.16 + [email protected]
#17 27.16 
#17 27.16 897 packages installed [26.49s]
#17 27.16 
#17 27.16 Blocked 1 postinstall. Run `bun pm untrusted` for details.
#17 DONE 28.1s

#18 exporting to image
#18 exporting layers

[XenoWad01]

@nzapponi I will also try to look for evidance of bun supporting --frozen-lockfile installing from yarn.lock.

[XenoWad01]

@nzapponi So according to this closed issue it should be able to install from a yarn.lock file: oven-sh/bun#1751. So if I am not missing something this should work.

[XenoWad01]

@terion-name I just checked and It seems I forgot to include that this is in my package.json with the current working setup. "packageManager": "[email protected]", And i just checked if I get the error you got if i instead use "packageManager": "[email protected]",. So that is what the difference is. So basically just add in that line you mentioned above before prunning: RUN json -I -f package.json -e "this.packageManager=\"[email protected]\"".

[XenoWad01]

Turbo basically throws that error based on what you have in package.json under "packageManager". Maybe they should just warn instead of erroring... seeing as you can actually prune bun workspaces if you do these workarounds. This would at least get us rid of the RUN json [...] command and make this solution less hacky until they will actually support bun lockfiles.

[nicholasoxford]

When deploying docker files to Fly.io, using a Turborepo, having the prune command makes life a lot easier. Especially as your monorepo grows in size

[terion-name]

Bun supports yarn.lock for very long time. It is a shame that it is not supported in turbo =(

[apex-woot]

+1
It would be great to have bun support prune in any form to be able to use turbo prune!

[maximblack]

as a workaround, I am using pnpm as a package manager (for pruning), then running the release with bun

# use the official Bun image
# see all versions at https://hub.docker.com/r/oven/bun/tags
FROM oven/bun:1 AS base
WORKDIR /app

FROM node:20-alpine AS pruned
ARG PROJECT=@t4/api
WORKDIR /app

COPY . .
RUN npx turbo prune --scope=${PROJECT} --docker
COPY tsconfig.json ./out/full/

# install dependencies into temp directory
# this will cache them and speed up future builds
FROM node:20-alpine AS deps
ARG CI=true

# Setup pnpm environment
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable
RUN corepack prepare pnpm@latest --activate


RUN mkdir -p /temp/dev
COPY --from=pruned /app/out/json/ /temp/dev/
RUN cd /temp/dev && pnpm install --frozen-lockfile --prefer-frozen-lockfile

# install with --production (exclude devDependencies)
RUN mkdir -p /temp/prod
COPY --from=pruned /app/out/json/ /temp/prod/
RUN cd /temp/prod && pnpm install --prod --frozen-lockfile --prefer-frozen-lockfile

# copy node_modules from temp directory
# then copy all (non-ignored) project files into the image
FROM node:20-alpine AS prerelease
# Setup pnpm environment
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable
RUN corepack prepare pnpm@latest --activate


WORKDIR /app
COPY --from=deps /temp/dev .
COPY --from=pruned /app/out/full/ .

# [optional] tests & build
ENV NODE_ENV=production
RUN cd packages/api && pnpm run zenstack:generate
# RUN bun test
# RUN bun run build
# RUN bun run turbo build --filter=@t4/api

# copy production dependencies and source code into final image
FROM base AS release
WORKDIR /app
COPY --from=deps /temp/prod .
COPY --from=prerelease /app .
# COPY --from=prerelease /app/node_modules/.pnpm/@prisma+client* ./node_modules/.pnpm/
COPY --from=prerelease /app/node_modules/.pnpm/@prisma+client* ./node_modules/.pnpm/@[email protected][email protected]

# run the app
USER bun
EXPOSE 8787/tcp

#HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 CMD [ "wget", "-q0", "http://localhost:8787/health" ]
ENTRYPOINT [ "bun", "run", "packages/api/src/worker.ts" ]
#ENTRYPOINT [ "bash" ]

[JarvisPrestidge]

For anyone looking for make this "work" in the run up to Bun adding their new JSON lock files (when turbo will add support) hopefully in Q3, you can use this approach.

ARG BUN_VERSION=1.1.26

ARG YARN_PKG_MANAGER="this.packageManager=\"[email protected]\""
ARG BUN_PKG_MANAGER="this.packageManager=\"bun@${BUN_VERSION}\""

FROM oven/bun:${BUN_VERSION} as bun
WORKDIR /app

FROM oven/bun:${BUN_VERSION}-alpine as bun-alpine
WORKDIR /app

# ================= TURBO PRUNE ===================

FROM bun as pruned
ARG YARN_PKG_MANAGER
COPY . .
RUN bunx json -I -f package.json -e ${YARN_PKG_MANAGER}
RUN bunx turbo prune --scope="example-service" --docker

# =============== INSTALL & BUILD =================

FROM bun as builder
ARG BUN_PKG_MANAGER
COPY --from=pruned /app/out/full/ .
RUN bunx json -I -f package.json -e ${BUN_PKG_MANAGER}
RUN bun install --production
RUN bunx clean-modules -y "**/*.ts" "**/@types/**"
RUN bunx turbo build --filter="example-service..."

# ================== RELEASE ======================

FROM bun-alpine AS release
USER bun
COPY --from=builder /app .
EXPOSE 3000/tcp
WORKDIR /app/apps/example-service
ENTRYPOINT [ "bun", "dist/main.js" ]

This is for a Nest.js monorepo with internal packages that aren't compiled.

The build script in the example-service simply uses bun build as the "bundler" with "packages": "external" set.

p.s. I needed to run bun install against the "full" output of prune because one of my monorepo packages is a cli binary that i need linked into node_modules/.bin but if you don't have that requirement you can follow the standard, copy over out/json, bun install, copy over out/full then build.