Access memoryfs files

[sunyang713]

Hi, so I know that this middleware automatically handles the serving of all assets from memory. However, is there a way to let express use this memoryFs for other middleware as well? So that I can, for example, use res.render('myfile.ext') where myfile.ext is also in the webpack-dev memoryFs?

[grrowl]

You can access the filesystem directly through the fileSystem property on the devMiddleware instance:

const compiler = webpack(config),
  devMiddleware = webpackDevMiddleware(compiler)

app.use(devMiddleware)

app.use((req, res) => {
  // rewrite the request to serve the index page
  console.log('serving index.html for', req.url)

  res.set('Content-Type', 'text/html')
  res.send(
    devMiddleware.fileSystem.readFileSync(config.output.path + 'index.html')
  )
})

edit: oops, apparently "You should not access devMiddleware.fileSystem. It's private. Instead rewrite requests."

[SpaceK33z]

@sunyang713, does the link @grrowl posted above answer your question? So you'd need to rewrite the request.

[sunyang713]

Not quite @SpaceK33z, rewriting requests alone doesn't achieve what I need, because I would still need to access the memoryFs. I also posted in the issue that grrowl linked (no response yet). I've pasted below

hi, so just to be clear, is it still okay to manually serve a file from devMiddleware.fileSystem? For my project, this is pretty important - i.e. I need to modify the index.html request handler that devMiddleware automatically implements. How exactly can I "rewrite requests" for devMiddleware.fileSystem files?

[SpaceK33z]

@sunyang713, could you tell me why you need to rewrite the request to index.html? Just understanding your usecase would help me. If you need to set a variable in your index.html, I would suggest to use html-webpack-plugin instead. It allows you to pass custom variables and all other crazy stuff, without hacking the filesystem.

[sunyang713]

Thanks for the response @SpaceK33z . I want to be very clear about my use case, so this response is pretty long. I would be very happy to see if there's a nicer, /cleaner/ way to do this that the community has established, but at the time I hadn't found one, and I haven't found one since.

I do use html-webpack-plugin. I use it to add a hash after my javascript builds. At the same time, I use server-side rendering with react. I need to render some initial content, AFTER the server is running and index.html is generated with the correctly linked files. The solution I am trying to do is using html-webpack-plugin to generate index.ejs; I start with a base template.ejs and a generate a new template. Before I run the server, I start with some base template:

. . .
<%= "\<% if (content) { %\>" %>
<div id="root"><%= "\<%- content %\>" %></div>
<%= "\<% } else { %\>" %>
. . .

At the bottom of the file, the correct index.js?1231243hash file is linked. Then, when I run the server with webpack (in devo; prod vs devo explained later), html-webpack-plugin converts this to:

. . .
<% if (content) { %>
<div id="root"><%- content %></div>
<% } else { %>
. . .

So I've generated a template from a template, and I will then use this to server-render like so:

app.set('view engine', 'ejs')
. . .
// on 200
res.render('index.ejs', appInjections())

where appInjections is a function that renders my react app server-side as a string, and returns an object such as:

{
    "content": "<react id stuff stuff stuff> ... ... . > .> "
}

And res.render will pass this in nicely.

... except it doesn't - here is the problem. In prod, I would build all of my javascript files before running my server. The document lives in the filesystem and is easily accessed using the static-middleware provided by express.

But in devo, I use webpack-dev-middleware which means all files are served from memory. This also means that the generated index.ejs is in memory. res.render("index.ejs") looks for 'index.ejs' in the regular filesystem, not the memoryfs. So for the time being, I've been accessing the memoryfs as a hack. I rewrite the request to / to return:

// the return value of a new function renderApp()
ejs.render(getDocument(), { content })

where getDocument() is a function that looks like this in devo:

function getDocument() {
  const memoryFs = compiler.outputFileSystem // 'compiler' is the webpack compiler
  return memoryFs.readFileSync('index.ejs', 'utf8')
}

In prod it looks like this:

import fs from 'fs'
import path from 'path'

export default function getDocument() {
  return fs.readFileSync('index.ejs', 'utf8')
}

and the response command is changed to:

// on 200
res.send(renderApp())

I would like for getDocument() to look almost identical (more like the prod version), or somehow get rid of the need for external functions with a split between devo/prod, so I could maybe do res.render() for both devo and prod.

[SpaceK33z]

Wow. Okay, thanks for your detailed response. Did you take a look at #118? I'm not sure if it completely fits your usecase, but it has some similarities.

Otherwise maybe it would be an idea to enhance the feature from #118? A PR is welcome if it's not too specific.

[axelpale]

Thanks @sunyang713 for sharing your solution. I stumbled upon this inconvenient issue myself too with a pretty similar tech stack. In my case, I have a single-page app (SPA) with client-side routing and URLs like /login, /invite, /item/123, and the root / of course. To provide a boring use case, if a user bookmarks the /item/123 and browses to it later on, the URL needs to work. The express server should response the request /item/123 with index.html in a manner identical to the requests /, /login, and /invite. Their routing is then made on the client.

However, in prod env this is easy as you showed but not so in dev env where I use the webpack-dev-middleware. The server-side code should have a file access to the webpacked index.html, then to be able to serve it for any valid URL the user throws at the server.

Fortunately in my case the solution was simple. Because my index.html is completely static in the first place, I am able to serve it without messing with webpack's temporary files. I lose the benefits of html uglification and other webpack feats but in dev environment that is ok. For a reference, the server-side catch-all route for the SPA then becomes:

app.get('/*', function (req, res) {
  res.sendFile(path.resolve(__dirname, 'public', 'index.html');
});

I hope this comment to expand the perspective and further describe the issue.

[SpaceK33z]

@axelpale, can't you do something like this:

var webpackMiddleware = require("webpack-dev-middleware");
var historyApiFallback = require("connect-history-api-fallback");
var instance = webpackMiddleware(...);
app.use(instance);
app.use(historyApiFallback());
app.use(instance);

Note that using app.use(instance) twice is on purpose, without this rewriting the request doesn't work.

[axelpale]

@SpaceK33z Interesting! The first app.use(instance) catches /index.html. If not caught, then app.use(historyApiFallback()) will catch /any/thing and rewrite it to /index.html, which is finally caught and served by the second app.use(instance). I get it, cool, thanks!

[eduardoleal]

I tried to do something like @SpaceK33z 's comment but the express server does not respond.

const express = require('express');
const path = require('path');
const app = express();
const historyApiFallback = require('connect-history-api-fallback');

if (process.env.NODE_ENV !== 'production') {
    const webpackMiddleware = require('webpack-dev-middleware');
    const webpack = require('webpack');
    const webpackConfig = require('./webpack.config.js');
    const webpackInstance = webpackMiddleware(webpack(webpackConfig));
    app.use(webpackInstance);
    app.use(historyApiFallback);
    app.use(webpackInstance);
} else {
    app.use(express.static('dist'));
    app.get('*', (req, res) => {
        res.sendFile(path.join(__dirname, 'dist/index.html'));
    })
}

app.listen(process.env.PORT || 3050, () => console.log('Listening...'));

I can't access react routes using browserHistory. It only works if i use hashHistory.

[anthonator]

@eduardoleal were you able to resolve this?

[anthonator]

Oops, you need to pass historyApiFallback in as a function.

@SpaceK33z's example is correct.

[eduardoleal]

Yes, @anthonator!
I'm using the following code:

/* eslint import/no-extraneous-dependencies: ["error", {"devDependencies": true}] */
import 'babel-polyfill';
import express from 'express';
import path from 'path';
import webpackMiddleware from 'webpack-dev-middleware';
import webpack from 'webpack';
import prerender from 'prerender-node';
import webpackConfig from './webpack.config';

const app = express();

if (process.env.NODE_ENV !== 'production') {
  app.use(webpackMiddleware(webpack(webpackConfig)));
} else {
  app.use(prerender);
  app.use(express.static('build'));
  app.get('*', (req, res) => {
    res.sendFile(path.join(__dirname, 'build/index.html'));
  });
}

app.listen(process.env.PORT || 10090, () => {
  console.log('Listening...')
});

[shellscape]

Closing as there seems to be a good resolution here. If anyone would like to add this to the documentation, please open a PR 😄

[robbyemmert]

There really needs to be a more intuitive solution. Adding the instance twice seems really hacky. Is there a good reason that there isn't an equivalent historyApiFallback option in webpack-dev-middleware? This seems like a really common use case that deserves a setting.

[robbyemmert]

Also, could there be performance concerns with adding webpack twice?

[shellscape]

@robbyemmert it looks like you may be new to Github, so please do note that it's a best practice to edit a previous comment to add more info, rather than posting a comment shortly after a previously posted comment.

I'm not sure you're fully understanding the solutions provided in this thread, as neither of the latest examples "add webpack" . Regardless, there's a new feature in writeToDisk you may want to take a look at. You can find more information in the README.