missing main entry

[arlac77]

since version 4.0.0 rollup picks the browser.js instead of node.js

maybe package main entry got lost in 34fedff ?

[gr2m]

Hmm ... the package.json file in the project root is not published, instead this project is build with @pika/pack into a pkg/ folder which then is published. The published package.json has a "main" field for node: https://unpkg.com/browse/[email protected]/package.json

But also a "module" field which points to the browser version. I wasn't aware that folks would use that for Node, too?

[gr2m]

I just made double sure that at least require-ing in node is still working as expected:
https://runkit.com/gr2m/gr2m-universal-user-agent-23

[gr2m]

Any idea how to resolve problem? What would the published package.json look like, ideally?

[arlac77]

There two degrees of freedom

• content
  • browser
  • node
• package format
  • esm
  • require ?

And only there entry points

• browser
• main
• module

Let me dig around
the rollup-plugin-node-resolve plugin
to find other modules with the same pattern (esm module(s) either for browser or node)

How is this handled in the other @octokit modules ?

For my node cli app I want to pick node/esm (through @octokit/rest)

[gr2m]

How is this handled in the other @octokit modules ?

Hmm the other Octokit modules mostly work the same in both Node & browser, with the exception of https://github.com/octokit/auth-app.js/ which is using the browser's WebCrypto API and for Node it's using the jsonwebtoken package

I did not yet migrate all @octokit modules, but soon all of them will be built with https://github.com/pikapkg/pack and will have the separate builds for node (using require) and browsers (using import).

Pika also distributes a dist-src folder, maybe that will work for you? Try to import universal-user-agent/dist-src (See published code here: https://unpkg.com/browse/[email protected]/dist-src/)

[arlac77]

The problem is I am only using

import octokit from "@octokit/rest";

[gr2m]

Ah got it. Hmmm I really don't know, but I hope we find a solution for this. I asked the folks from Pika on twitter, maybe they have an idea what to do: https://twitter.com/gr2m/status/1169357730944958464

[arlac77]

when declaring universal-user-agent as external in rollup i am able to select the node content out of the package (even through @octokit/rest).
But then there is the breaking change between 3.0.0 and 4.0.0 (default export vs named export) which is not reflected in @octokit/rest

[gr2m]

It should be as of v16.28.9: https://github.com/octokit/rest.js/pull/1460

[schof]

@arlac77 I'm trying to use @octokit/rest in node but running into similar problems as you. I'm using webpack and I'm excluding universal-user-agent from the build but AWS Lambda is still trying to load it:

Runtime.ImportModuleError: Error: Cannot find module 'universal-user-agent'

I'm using the latest @octokit/rest but no luck. Any ideas?

[arlac77]

@schof I have no solution only a workaround my list of to be excluded modules (rollup) is getting longer and longer.

// rollup.config.js

const external = [
  ...builtins,
  "universal-user-agent",
  "@octokit/rest",
  "node-fetch"
];

[schof]

@arlac77 I tried the same thing you are doing with similar results. Ultimately I got things working by not excluding anything and monkey patching the universal-user-agent library. Webpack allows you to specify multiple locations for your node file - perhaps roll up will allow you to do something similar?

resolve: {
  // Monkey patch to resolve https://github.com/mobileposse/neptune/issues/296
  modules: [path.resolve(__dirname, 'build/src/monkey-patch'), 'node_modules'],
  extensions: ['.tsx', '.ts', '.js']
}

then i took the index.js contents from the universal-user-agent package and dropped it into build/src/monkey-patch directory:

'use strict'

Object.defineProperty(exports, '__esModule', { value: true })

function _interopDefault(ex) {
  return ex && typeof ex === 'object' && 'default' in ex ? ex['default'] : ex
}

var osName = _interopDefault(require('os-name'))

function getUserAgent() {
  try {
    return `Node.js/${process.version.substr(1)} (${osName()}; ${process.arch})`
  } catch (error) {
    if (/wmic os get Caption/.test(error.message)) {
      return 'Windows <version undetectable>'
    }

    throw error
  }
}

exports.getUserAgent = getUserAgent

Hope that helps. Good luck!

[gr2m]

I’m sorry this is causing so much headache, I hope we find a simpler solution that I can apply to the libraries, universal-user-agent is not the only one that is published this way

[FredKSchott]

Hey yall, sorry for the delay jumping into this! Happy to help where i can. @arlac77 sounds like you summed it up pretty well, we have a 2x2 matrix and only 3 entrypoints to represent them with. On top of that, "module" has a real frontend focus at this point, since web bundlers are primarily the only real environment that they are expected to be used in.

The result is that "Node ESM" is relatively unsupported, given that the official plan is for Node to move to: type: "module", main: "index.mjs". But until then, there really is no good way to use ESM on Node.js.

Like @gr2m I'm still not sure I understand what situation is wanting you to use rollup inside of Node. Are you trying to bundle a Node application into a single JS file? If so, Webpack would probably be a better fit since it's much more Common.js dependent & reliant. https://github.com/zeit/ncc & https://github.com/zeit/pkg may also be good options for you.

If you really do need to use Rollup, have you tried the latest version of https://github.com/rollup/rollup-plugin-node-resolve ? Its most recent syntax allows you to define either mainFields: ['main', 'module'] (to prefer main over module) or mainFields: ['main'] (to only read from main). That could solve your problem as well.

[maplesteve]

@FredKSchott, @gr2m : The @octokit/rest library just doesn't work anymore in a nodeJS environment (Lambda) after the 4.0 release - also when webpack is used. It just crashes with module initialization error: ReferenceError.
Our CI/CD tooling completely stopped working since then and a downgrade of @octokit/rest doesn't seem to work, since the other octokit dependencies also use the v4.

[gr2m]

@maplesteve There must be a way in webpack to workaround this, like there is in rollup as mentioned by @FredKSchott. It's not something that I can fix in Octokit itself, I'm afraid.

But people keep running into this so I'd love to address it in the READMEs.

[gr2m]

@maplesteve @schof @arlac77

Could you please try to directly edit node_modules/universal-user-agent/package.json and add "type": "commonjs" to it, and see if that resolves the issue?

Also, could you please setup a repository with the most minimal build setup that reproduces the problem for you? I would appreciate to have both a setup using webpack and rollup.

Let's fix this 💪

[gr2m]

There is also a new "exports" field that might work:
https://nodejs.org/api/esm.html#esm_package_exports

I've asked on the twitters and Myles shared some helpful links:
https://mobile.twitter.com/gr2m/status/1176199758127362048

[arlac77]

Here a starting point showing rollup selecting browser content by default and the 'external' workaround to exclude 'universal-user-agent' from bundling

arlac77/universal-user-agent-bundler-problems

[FredKSchott]

I’m starting to think this may be a Lambda misconfiguration/issue, If they’re using webpack internally there’s a configuration for this that they could probably set to solve for this: https://webpack.js.org/configuration/resolve/#resolvealiasfields

[maplesteve]

@gr2m
Adding "type": "commonjs" yields the same error:

"errorType":"ReferenceError", 
"stackTrace": ["ReferenceError: navigator is not defined","at getUserAgent
(webpack:///./node_modules/universal-user-agent/dist-web/index.js?:4:5)","at eval
(webpack:///./node_modules/@octokit/endpoint/dist-web/index.js?:364:126)","at
Module../node_modules/@octokit/endpoint/dist-web/index.js

[maplesteve]

@FredKSchott regarding Lambda: "they" in terms of AWS are doing nothing with webpack. Webpack is used to bundle the code which then gets uploaded and used in the Lambda environment. It worked before.

[gr2m]

@maplesteve I found netlify/netlify-lambda#138 which sounds similar to yuor probelem. See their fix at https://github.com/netlify/netlify-lambda/pull/160/files. It looks like webpack is prefering the browser field over module and main which is causing the problem. In the case of universal-user-agent, you might need to set it to just main or ["main", "module"]. Could you give that a try?

I think Shawn summerizes the problem pretty well

this is unfortunately just a result of us using webpack for something it wasn't strictly designed to do

[FredKSchott]

@maplesteve ah gotcha, I’d misunderstood. If you own the webpack config / are running webpack yourself, hopefully that resolveAlias config option fixes thing

[maplesteve]

@gr2m

The webpack changes according to the netlify issue did not help.

I created a repo to reproduce: https://github.com/maplesteve/navigator-error

[gr2m]

Can you please try to set resolveAlias as suggested by Fred? I'd try this

module.exports = {
  resolve: {
    aliasFields: []
  }
};

[maplesteve]

@gr2m tried your suggestion re: aliasFields and also some other values ('browser', 'commonjs'...) but still get the same error. The source still references e.g. @octokit/request/dist-web/index.js where IMO it should pick @octokit/request/dist-node/index.js

[eine]

Also, could you please setup a repository with the most minimal build setup that reproduces the problem for you? I would appreciate to have both a setup using webpack and rollup.

Let's fix this 💪

I think I hit this issue when trying to add webpack to the actions/typescript-action template. This is a minimal main.ts:

//import * as github from '@actions/github';
const github = require('@actions/github');

export async function run() {
  try {
    console.log("HELLO WEBPACK!");

    var slug = ['user','repo']
    if (github.context.payload.repository && github.context.payload.repository.full_name) {
        slug = github.context.payload.repository.full_name.split('/')
    }
  }

  catch (error) {
    throw error;
  }
}

run();

And the webpack.config.js:

const join = require('path').join;
module.exports = {
	entry: './dist/main.js',
	target: 'node',
	output: {
		path: join(__dirname, 'dist'),
		filename: 'main.pack.js'
	},
	optimization: {
		minimize: false
	},
	devtool: 'sourcemap'
};

Either requiring or importing @actions/github adds the following to main.pack.js:

/***/ "./node_modules/universal-user-agent/dist-web/index.js":
/*!*************************************************************!*\
  !*** ./node_modules/universal-user-agent/dist-web/index.js ***!
  \*************************************************************/
/*! exports provided: getUserAgent */
/***/ (function(module, __webpack_exports__, __webpack_require__) {

"use strict";
__webpack_require__.r(__webpack_exports__);
/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, "getUserAgent", function() { return getUserAgent; });
function getUserAgent() {
    return navigator.userAgent;
}

As a result, the action fails:

/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:16222
    return navigator.userAgent;
    ^

ReferenceError: navigator is not defined
    at getUserAgent (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:16222:5)
    at Module../node_modules/@octokit/endpoint/dist-web/index.js (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:821:126)
    at __webpack_require__ (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:20:30)
    at Module../node_modules/@octokit/request/dist-web/index.js (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:1153:75)
    at __webpack_require__ (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:20:30)
    at Object../node_modules/@octokit/graphql/index.js (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:924:21)
    at __webpack_require__ (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:20:30)
    at Object../node_modules/@actions/github/lib/github.js (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:431:19)
    at __webpack_require__ (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:20:30)
    at Object../ts/main.ts (/home/runner/work/_actions/1138-4EB/tip/gha-tip/main.js:16541:29)

EDIT

This ugly workaround fixes it: eine/tip@f603695

[arlac77]

Here another type of workaround:
replacing the whole module with one line of code during bundling (for node runtime only).

rollup.config.js

import virtual from "rollup-plugin-virtual";
...

    plugins: [
      virtual({
        "universal-user-agent":
          "export function getUserAgent() {return `Node.js/${process.version.substr(1)} (${process.platform}); ${process.arch})`}"
      }),
      ...otherPlugins
    ]

[arlac77]

@gr2m there is a recently updated proposal for Conditional Mapping in packages
jkrems/proposal-pkg-exports

[gr2m]

Thanks I'm following it closely, relevant discussion is here: nodejs/modules#401

[eamodio]

Does anyone know of a way to workaround this (other than editing the package.json) with webpack (also without using resolve.mainFields because that affects all modules). I am trying to use octokit/code in 2 of my VS Code extensions and because I use ES modules (mainly for tree-shaking) this breaks the bundling.

I've tried using

alias: {
	'universal-user-agent': 'node_modules/universal-user-agent/dist-node/index.js'
}

But that causes other Module not found: Error: Can't resolve 'universal-user-agent' in other octokit dependencies.

[gr2m]

There is no ESM version of universal-user-agent for Node right now, sorry. The ESM version is for browsers, Node only has a CJS version. Because Node didn't support ESM until very recently.

I've worked with Node's ES Modules working group over the past weeks and there is a clear way forward now. It will take some time for the build tools and Pika to catch up, but Node's latest spec supports all that we need: https://github.com/nodejs/node/blob/master/doc/api/esm.md

The package.json will look like this:

{
  "main": "dist-node-cjs/index.js",
  "module": {
    "browser": "dist-browser/index.js"
    "default": "dist-node/index.js"
  }
}

Once the tooling catches up, I'll update the library. Shouldn't take too long, it's mostly Pika and its Rollup plugins, and Fred from Pika has been involved in the same working group.

[sheepsteak]

I've managed to use Webpack's NormalModuleReplacementPlugin to point Webpack to the correct entry point for my Serverless service:

plugins: [
  new webpack.NormalModuleReplacementPlugin(
      /universal-user-agent\/dist-web\/index\.js/,
      "../dist-src/index.js"
    )
]

I used dist-src as it still uses modules whereas dist-node is CommonJS.

[KamasamaK]

The above workaround with NormalModuleReplacementPlugin did not work for me, but this did:

resolve: {
  // ...
  alias: {
    "universal-user-agent": path.resolve(
      __dirname,
      "node_modules/universal-user-agent/dist-src/index.js"
    )
  }
}

[gr2m]

I'll add a try/catch around the navigator.userAgent, so at least it should no longer throw an error, via #41

Once conditional exports are supported by @pika/pack, I can provide custom exports for different environments, so the resulting user agent will be more helpful than <navigator unknown>, but at least this should unblock people for the time being

Any thoughts?

[gr2m]

I've decided to greatly simplify this package to workaround the build problems y'all are experiencing. I was hoping for conditional exports to be adopted more quickly, but it does not look like it.

See #52

[github-actions]

🎉 This issue has been resolved in version 6.0.0 🎉

The release is available on:

• GitHub release
• npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

[github-actions]

🎉 This issue has been resolved in version 6.0.0 🎉

The release is available on:

• GitHub release
• npm package (@release-6.x dist-tag)

Your semantic-release bot 📦🚀