Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Support URI encoded source maps #75

Merged
merged 10 commits into from
Oct 15, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,17 @@ Returns source map converter from given object.

Returns source map converter from given json string.

### fromURI(uri)

Returns source map converter from given uri encoded json string.

### fromBase64(base64)

Returns source map converter from given base64 encoded json string.

### fromComment(comment)

Returns source map converter from given base64 encoded json string prefixed with `//# sourceMappingURL=...`.
Returns source map converter from given base64 or uri encoded json string prefixed with `//# sourceMappingURL=...`.

### fromMapFileComment(comment, mapFileDir)

Expand All @@ -50,11 +54,11 @@ generated file, i.e. the one containing the source map.

### fromSource(source)

Finds last sourcemap comment in file and returns source map converter or returns null if no source map comment was found.
Finds last sourcemap comment in file and returns source map converter or returns `null` if no source map comment was found.

### fromMapFileSource(source, mapFileDir)

Finds last sourcemap comment in file and returns source map converter or returns null if no source map comment was
Finds last sourcemap comment in file and returns source map converter or returns `null` if no source map comment was
found.

The sourcemap will be read from the map file found by parsing `# sourceMappingURL=file` comment. For more info see
Expand All @@ -70,6 +74,10 @@ Converts source map to json string. If `space` is given (optional), this will be
[JSON.stringify](https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/JSON/stringify) when the
JSON string is generated.

### toURI()

Converts source map to uri encoded json string.

### toBase64()

Converts source map to base64 encoded json string.
Expand All @@ -81,6 +89,8 @@ Converts source map to an inline comment that can be appended to the source-file
By default, the comment is formatted like: `//# sourceMappingURL=...`, which you would
normally see in a JS source file.

When `options.encoding == 'uri'`, the data will be uri encoded, otherwise they will be base64 encoded.

When `options.multiline == true`, the comment is formatted like: `/*# sourceMappingURL=... */`, which you would find in a CSS source file.

### addProperty(key, value)
Expand All @@ -107,6 +117,8 @@ Returns `src` with all source map comments pointing to map files removed.

Provides __a fresh__ RegExp each time it is accessed. Can be used to find source map comments.

Breaks down a source map comment into groups: Groups: 1: media type, 2: MIME type, 3: charset, 4: encoding, 5: data.

### mapFileCommentRegex

Provides __a fresh__ RegExp each time it is accessed. Can be used to find source map comments pointing to map files.
Expand Down
38 changes: 30 additions & 8 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ var path = require('path');

Object.defineProperty(exports, 'commentRegex', {
get: function getCommentRegex () {
return /^\s*?\/(?:\/|\*?)[@#]\s+?sourceMappingURL=data:(?:application|text)\/json;(?:charset[:=]\S+?;)?base64,(?:.*?)$/mg;
// Groups: 1: media type, 2: MIME type, 3: charset, 4: encoding, 5: data.
return /^\s*?\/[\/\*][@#]\s+?sourceMappingURL=data:(((?:application|text)\/json)(?:;charset=([^;,]+?)?)?)?(?:;(base64))?,(.*?)$/mg;
}
});


Object.defineProperty(exports, 'mapFileCommentRegex', {
get: function getMapFileCommentRegex () {
// Matches sourceMappingURL in either // or /* comment styles.
Expand Down Expand Up @@ -66,8 +68,9 @@ function Converter (sm, opts) {

if (opts.isFileComment) sm = readFromFileMap(sm, opts.commentFileDir);
if (opts.hasComment) sm = stripComment(sm);
if (opts.isEncoded) sm = decodeBase64(sm);
if (opts.isJSON || opts.isEncoded) sm = JSON.parse(sm);
if (opts.encoding === 'base64') sm = decodeBase64(sm);
phated marked this conversation as resolved.
Show resolved Hide resolved
else if (opts.encoding === 'uri') sm = decodeURIComponent(sm);
if (opts.isJSON || opts.encoding) sm = JSON.parse(sm);

this.sourcemap = sm;
}
Expand All @@ -76,6 +79,7 @@ Converter.prototype.toJSON = function (space) {
return JSON.stringify(this.sourcemap, null, space);
};


if (typeof Buffer !== 'undefined') {
if (typeof Buffer.from === 'function') {
Converter.prototype.toBase64 = encodeBase64WithBufferFrom;
Expand Down Expand Up @@ -104,9 +108,21 @@ function encodeBase64WithBtoa() {
return btoa(unescape(encodeURIComponent(json)));
}

Converter.prototype.toURI = function () {
var json = this.toJSON();
return encodeURIComponent(json);
};

Converter.prototype.toComment = function (options) {
var base64 = this.toBase64();
var data = 'sourceMappingURL=data:application/json;charset=utf-8;base64,' + base64;
var encoding, content, data;
if (options && options.encoding === 'uri') {
phated marked this conversation as resolved.
Show resolved Hide resolved
encoding = '';
content = this.toURI();
} else {
encoding = ';base64';
content = this.toBase64();
}
data = 'sourceMappingURL=data:application/json;charset=utf-8' + encoding + ',' + content;
return options && options.multiline ? '/*# ' + data + ' */' : '//# ' + data;
phated marked this conversation as resolved.
Show resolved Hide resolved
};

Expand Down Expand Up @@ -137,16 +153,22 @@ exports.fromJSON = function (json) {
return new Converter(json, { isJSON: true });
};

exports.fromURI = function (uri) {
return new Converter(uri, { encoding: 'uri' });
};

exports.fromBase64 = function (base64) {
return new Converter(base64, { isEncoded: true });
return new Converter(base64, { encoding: 'base64' });
};

exports.fromComment = function (comment) {
var m, encoding;
comment = comment
.replace(/^\/\*/g, '//')
.replace(/\*\/$/g, '');

return new Converter(comment, { isEncoded: true, hasComment: true });
m = exports.commentRegex.exec(comment);
encoding = m && m[4] || 'uri';
return new Converter(comment, { encoding: encoding, hasComment: true });
};

exports.fromMapFileComment = function (comment, dir) {
Expand Down
126 changes: 111 additions & 15 deletions test/comment-regex.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,27 @@ var test = require('tap').test
, generator = require('inline-source-map')
, convert = require('..')

function comment(prefix, suffix) {
var rx = convert.commentRegex;
return rx.test(prefix + 'sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiIiwic291cmNlcyI6WyJmdW5jdGlvbiBmb28oKSB7XG4gY29uc29sZS5sb2coXCJoZWxsbyBJIGFtIGZvb1wiKTtcbiBjb25zb2xlLmxvZyhcIndobyBhcmUgeW91XCIpO1xufVxuXG5mb28oKTtcbiJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiQUFBQSJ9' + suffix)
function comment(prefix, suffix, rx) {
return rx.exec(prefix + 'sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiIiwic291cmNlcyI6WyJmdW5jdGlvbiBmb28oKSB7XG4gY29uc29sZS5sb2coXCJoZWxsbyBJIGFtIGZvb1wiKTtcbiBjb25zb2xlLmxvZyhcIndobyBhcmUgeW91XCIpO1xufVxuXG5mb28oKTtcbiJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiQUFBQSJ9' + suffix)
phated marked this conversation as resolved.
Show resolved Hide resolved
}
function commentURI(prefix, suffix, rx) {
return rx.exec(prefix + 'sourceMappingURL=data:application/json,%7B%22version%22%3A3%2C%22file%22%3A%22%22%2C%22sources%22%3A%5B%22function%20foo()%20%7B%0A%20console.log(%22hello%20I%20am%20foo%22)%3B%0A%20console.log(%22who%20are%20you%22)%3B%0A%7D%0A%0Afoo()%3B%0A%22%5D%2C%22names%22%3A%5B%5D%2C%22mappings%22%3A%22AAAA%22%7D' + suffix)
}

function commentWithCharSet(prefix, suffix, rx) {
return rx.exec(prefix + 'sourceMappingURL=data:application/json;charset=utf-8;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiIiwic291cmNlcyI6WyJmdW5jdGlvbiBmb28oKSB7XG4gY29uc29sZS5sb2coXCJoZWxsbyBJIGFtIGZvb1wiKTtcbiBjb25zb2xlLmxvZyhcIndobyBhcmUgeW91XCIpO1xufVxuXG5mb28oKTtcbiJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiQUFBQSJ9' + suffix)
}

function commentURIWithCharSet(prefix, suffix, rx) {
return rx.exec(prefix + 'sourceMappingURL=data:application/json;charset=utf-8,%7B%22version%22%3A3%2C%22file%22%3A%22%22%2C%22sources%22%3A%5B%22function%20foo()%20%7B%0A%20console.log(%22hello%20I%20am%20foo%22)%3B%0A%20console.log(%22who%20are%20you%22)%3B%0A%7D%0A%0Afoo()%3B%0A%22%5D%2C%22names%22%3A%5B%5D%2C%22mappings%22%3A%22AAAA%22%7D' + suffix)
}

function commentWithoutMediaType(prefix, suffix, rx) {
return rx.exec(prefix + 'sourceMappingURL=data:;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiIiwic291cmNlcyI6WyJmdW5jdGlvbiBmb28oKSB7XG4gY29uc29sZS5sb2coXCJoZWxsbyBJIGFtIGZvb1wiKTtcbiBjb25zb2xlLmxvZyhcIndobyBhcmUgeW91XCIpO1xufVxuXG5mb28oKTtcbiJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiQUFBQSJ9' + suffix)
}

function commentWithCharSet(prefix, suffix, sep) {
sep = sep || ':';
thlorenz marked this conversation as resolved.
Show resolved Hide resolved
var rx = convert.commentRegex;
return rx.test(prefix + 'sourceMappingURL=data:application/json;charset' + sep +'utf-8;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiIiwic291cmNlcyI6WyJmdW5jdGlvbiBmb28oKSB7XG4gY29uc29sZS5sb2coXCJoZWxsbyBJIGFtIGZvb1wiKTtcbiBjb25zb2xlLmxvZyhcIndobyBhcmUgeW91XCIpO1xufVxuXG5mb28oKTtcbiJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiQUFBQSJ9' + suffix)
function commentURIWithoutMediaType(prefix, suffix, rx) {
return rx.exec(prefix + 'sourceMappingURL=data:,%7B%22version%22%3A3%2C%22file%22%3A%22%22%2C%22sources%22%3A%5B%22function%20foo()%20%7B%0A%20console.log(%22hello%20I%20am%20foo%22)%3B%0A%20console.log(%22who%20are%20you%22)%3B%0A%7D%0A%0Afoo()%3B%0A%22%5D%2C%22names%22%3A%5B%5D%2C%22mappings%22%3A%22AAAA%22%7D' + suffix)
}

// Source Map v2 Tests
Expand All @@ -28,15 +40,21 @@ test('comment regex old spec - @', function (t) {
'\t/*@ ', // multi line style with leading tab
'/*@ ', // multi line style with leading text
].forEach(function (x) {
t.ok(comment(x, ''), 'matches ' + x)
t.ok(commentWithCharSet(x, ''), 'matches ' + x + ' with charset')
t.ok(commentWithCharSet(x, '', '='), 'matches ' + x + ' with charset')
t.ok(comment(x, '', convert.commentRegex), 'matches ' + x)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto here .. existing tests should be touched as little as possible to ensure nothing breaks

t.ok(commentURI(x, '', convert.commentRegex), 'matches ' + x + ' uri')
t.ok(commentWithCharSet(x, '', convert.commentRegex), 'matches ' + x + ' with charset')
t.ok(commentURIWithCharSet(x, '', convert.commentRegex), 'matches ' + x + ' uri with charset')
t.ok(commentWithoutMediaType(x, '', convert.commentRegex), 'matches ' + x + ' without media type')
t.ok(commentURIWithoutMediaType(x, '', convert.commentRegex), 'matches ' + x + ' uri without media type')
});

[
' @// @',
' @/* @',
].forEach(function (x) { t.ok(!comment(x, ''), 'should not match ' + x) })
].forEach(function (x) {
t.ok(!comment(x, '', convert.commentRegex), 'should not match ' + x)
t.ok(!commentURI(x, '', convert.commentRegex), 'should not match ' + x + ' uri')
})

t.end()
})
Expand All @@ -51,15 +69,93 @@ test('comment regex new spec - #', function (t) {
'\t/*# ', // multi line style with leading tab
'/*# ', // multi line style with leading text
].forEach(function (x) {
t.ok(comment(x, ''), 'matches ' + x)
t.ok(commentWithCharSet(x, ''), 'matches ' + x + ' with charset')
t.ok(commentWithCharSet(x, '', '='), 'matches ' + x + ' with charset')
t.ok(comment(x, '', convert.commentRegex), 'matches ' + x)
phated marked this conversation as resolved.
Show resolved Hide resolved
t.ok(commentURI(x, '', convert.commentRegex), 'matches ' + x + ' uri')
t.ok(commentWithCharSet(x, '', convert.commentRegex), 'matches ' + x + ' with charset')
t.ok(commentURIWithCharSet(x, '', convert.commentRegex), 'matches ' + x + ' uri with charset')
t.ok(commentWithoutMediaType(x, '', convert.commentRegex), 'matches ' + x + ' without media type')
t.ok(commentURIWithoutMediaType(x, '', convert.commentRegex), 'matches ' + x + ' uri without media type')
});

[
' #// #',
' #/* #',
].forEach(function (x) {
t.ok(!comment(x, '', convert.commentRegex), 'should not match ' + x)
t.ok(!commentURI(x, '', convert.commentRegex), 'should not match ' + x + ' uri')
})

t.end()
})

test('comment regex groups', function (t) {
[
' //# ', // with leading spaces
'\t//# ', // with leading tab
'//# ', // with leading text
'/*# ', // multi line style
' /*# ', // multi line style with leading spaces
'\t/*# ', // multi line style with leading tab
'/*# ', // multi line style with leading text
].forEach(function (x) {
var m;
m = comment(x, '', convert.commentRegex)
t.ok(m, 'matches ' + x)
phated marked this conversation as resolved.
Show resolved Hide resolved
t.ok(m[0], 'comment')
t.equal(m[1], 'application/json', 'media type')
t.equal(m[2], 'application/json', 'MIME type')
t.equal(m[3], undefined, 'undefined charset')
t.equal(m[4], 'base64', 'base64 encoding')
t.ok(m[5], 'data')
m = commentURI(x, '', convert.commentRegex)
t.ok(m, 'matches ' + x + ' uri')
t.ok(m[0], 'comment uri')
t.equal(m[1], 'application/json', 'media type uri')
t.equal(m[2], 'application/json', 'MIME type uri')
t.equal(m[3], undefined, 'undefined charset uri')
t.equal(m[4], undefined, 'undefined encoding uri')
t.ok(m[5], 'data uri')
m = commentWithCharSet(x, '', convert.commentRegex)
t.ok(m, 'matches ' + x + ' with charset')
t.ok(m[0], 'comment with charset')
t.equal(m[1], 'application/json;charset=utf-8', 'media type with charset')
t.equal(m[2], 'application/json', 'MIME type with charset')
t.equal(m[3], 'utf-8', 'charset with utf-8')
t.equal(m[4], 'base64', 'base64 encoding with charset')
t.ok(m[5], 'data with charset')
m = commentURIWithCharSet(x, '', convert.commentRegex)
t.ok(m, 'matches ' + x + ' uri with charset')
t.ok(m[0], 'comment uri with charset')
t.equal(m[1], 'application/json;charset=utf-8', 'media type uri with charset')
t.equal(m[2], 'application/json', 'MIME type uri with charset')
t.equal(m[3], 'utf-8', 'charset uri with utf-8')
t.equal(m[4], undefined, 'undefined encoding uri with charset')
t.ok(m[5], 'data with charset')
m = commentWithoutMediaType(x, '', convert.commentRegex)
t.ok(m, 'matches ' + x + ' without media type')
t.ok(m[0], 'comment without media type')
t.equal(m[1], undefined, 'undefined media type')
t.equal(m[2], undefined, 'undefined MIME type')
t.equal(m[3], undefined, 'undefined charset without media type')
t.equal(m[4], 'base64', 'base64 encoding without media type')
t.ok(m[5], 'data without media type')
m = commentURIWithoutMediaType(x, '', convert.commentRegex)
t.ok(m, 'matches ' + x + ' uri without media type')
t.ok(m[0], 'comment uri without media type')
t.equal(m[1], undefined, 'undefined media type')
t.equal(m[2], undefined, 'undefined MIME type')
t.equal(m[3], undefined, 'undefined charset uri without media type')
t.equal(m[4], undefined, 'undefined encoding uri without media type')
t.ok(m[5], 'data uri without media type')
});

[
' #// #',
' #/* #',
].forEach(function (x) { t.ok(!comment(x, ''), 'should not match ' + x) })
].forEach(function (x) {
t.ok(!comment(x, '', convert.commentRegex), 'should not match ' + x)
t.ok(!commentURI(x, '', convert.commentRegex), 'should not match ' + x + ' uri')
})

t.end()
})
Expand Down
17 changes: 16 additions & 1 deletion test/convert-source-map.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,45 @@
var test = require('tap').test
, generator = require('inline-source-map')
, convert = require('..')
, decodeBase64 = typeof Buffer.from ?
function decodeBase64(base64) {
phated marked this conversation as resolved.
Show resolved Hide resolved
return Buffer.from(base64, 'base64').toString();
} :
function decodeBase64(base64) {
return new Buffer(base64, 'base64').toString();
}

var gen = generator({charset:"utf-8"})
.addMappings('foo.js', [{ original: { line: 2, column: 3 } , generated: { line: 5, column: 10 } }], { line: 5 })
.addGeneratedMappings('bar.js', 'var a = 2;\nconsole.log(a)', { line: 23, column: 22 })

, base64 = gen.base64Encode()
, uri = encodeURIComponent(decodeBase64(base64))
, comment = gen.inlineMappingUrl()
, comment2 = '//# sourceMappingURL=data:application/json;charset=utf-8,' + uri
, json = gen.toString()
, obj = JSON.parse(json)

test('different formats', function (t) {

t.equal(convert.fromComment(comment).toComment(), comment, 'comment -> comment')
t.equal(convert.fromComment(comment).toComment(), comment, 'comment -> comment (base64)')
t.equal(convert.fromComment(comment2).toComment({ encoding: 'uri' }), comment2, 'comment -> comment (uri)')
t.equal(convert.fromComment(comment).toBase64(), base64, 'comment -> base64')
t.equal(convert.fromComment(comment).toURI(), uri, 'comment -> uri')
t.equal(convert.fromComment(comment).toJSON(), json, 'comment -> json')
t.deepEqual(convert.fromComment(comment).toObject(), obj, 'comment -> object')

t.equal(convert.fromBase64(base64).toBase64(), base64, 'base64 -> base64')
t.equal(convert.fromURI(uri).toURI(), uri, 'uri -> uri')
t.equal(convert.fromBase64(base64).toComment(), comment, 'base64 -> comment')
t.equal(convert.fromBase64(base64).toJSON(), json, 'base64 -> json')
t.equal(convert.fromURI(uri).toJSON(), json, 'uri -> json')
t.deepEqual(convert.fromBase64(base64).toObject(), obj, 'base64 -> object')
t.deepEqual(convert.fromURI(uri).toObject(), obj, 'uri -> object')

t.equal(convert.fromJSON(json).toJSON(), json, 'json -> json')
t.equal(convert.fromJSON(json).toBase64(), base64, 'json -> base64')
t.equal(convert.fromJSON(json).toURI(), uri, 'json -> uri')
t.equal(convert.fromJSON(json).toComment(), comment, 'json -> comment')
t.deepEqual(convert.fromJSON(json).toObject(), obj, 'json -> object')
t.end()
Expand Down