Error encoding JS files with UTF-8 encoding #4

Catmacey · 2013-06-11T11:46:18Z

Hi, I am having problems when trying to minify JS files that contain UTF-8 using JSMin from your combine.jar.

I've created a standalone version of your compressJsWithJSMin() method rather than the whole combine.cfc.

As a simple test I have the string "This is a test Франция" which results in the output "This is a test $@0=F8O"

I've tried setting a Charset string on the output toString() method but it makes no difference.

I realise that this isn't strictly an issue with your combine package but I hope you can forgive me for asking for help here.

I have a simple standalone example that demonstrates the issue;

<cfcontent type="text/html; charset=utf-8" />
<cfprocessingdirective pageEncoding="utf-8"/>
<html>
    <head>
        <title>JSMin UTF-8 test</title>
    </head>
    <body>

<cftry>
    <cfscript>
        testString = "This is a test Франция";
        joInput = createObject("java","java.io.StringReader").init(testString);
        joOutput = createObject("java","java.io.ByteArrayOutputStream").init();
        javaloader  = createObject('component','components.javaloader.JavaLoader').init([application.settings.localdir & "components/jsmin/combine.jar"]);
        jJSMin = javaloader.create("com.magnoliabox.jsmin.JSMin").init(joInput, joOutput);
        jJSMin.jsmin();
    </cfscript>

<cfoutput><pre>
TestString:"#testString#"
Native:"#joOutput.toString()#"
UTF-8:"#joOutput.toString("UTF-8")#"
ISO-8859-1:"#joOutput.toString("ISO-8859-1")#"
US-ASCII:"#joOutput.toString("US-ASCII")#"
</pre></cfoutput>

    <cfset joInput.close() />
    <cfset joOutput.close() />

    <cfcatch>
        <cfdump var="#cfcatch#" />
    </cfcatch>
</cftry>

</body>
</html>

zefer · 2013-06-11T12:26:10Z

Hi @Catmacey.

Firstly, I haven't looked at this project for so long, the code looks totally alien to me, too, as I haven't worked with CFML for 4+ years! So I'm very sorry to say I can't support this project any more. However, I do have a couple of suggestions:

Make sure the file is saved as a UTF-8 file, to ensure your testString isn't affected by encoding issues
Check your test string is output correctly before processing (which your output does, but I couldn't tell from your description if you had checked this)
Try to isolate which process is messing with the encoding, maybe you could bypass the jsmin step to see if that's the cause?

Catmacey · 2013-06-11T12:57:57Z

Hi Joe,

I thought it was a longshot to ask you for help. I could see that the project hadn't had much activity for a while.

I've tried isolating all I can and dumping the output at various stages but ultimatly I think it comes down to JSMin java.

Thanks for the response and also for the tips.
I think I'll put this to one side for the moment and see if I can find another solution.

Best regards

Matt Casey

-----Original Message-----
From: Joe Roberts [mailto:[email protected]]
Sent: 11 June 2013 14:26
To: zefer/Combine
Cc: Catmacey
Subject: Re: [Combine] Error encoding JS files with UTF-8 encoding (#4)

Hi @Catmacey https://github.com/Catmacey .

Firstly, I haven't looked at this project for so long, the code looks totally alien to me, too, as I haven't worked with CFML for 4+ years! So I'm very sorry to say I can't support this project any more. However, I do have a couple of suggestions:

Make sure the file is saved as a UTF-8 file, to ensure your testString isn't affected by encoding issues
Check your test string is output correctly before processing (which your output does, but I couldn't tell from your description if you had checked this)
Try to isolate which process is messing with the encoding, maybe you could bypass the jsmin step to see if that's the cause?

—
Reply to this email directly or view it on GitHub #4 (comment) . https://github.com/notifications/beacon/BtoZ-o_OF9FU1KIu2BCsma_d9hW68rnoNG4K9-B6TVC1xAFOfj9kDcY7spqx1b-5.gif

zefer · 2013-06-11T13:07:11Z

Hi Matt.

The JSmin source is available here https://github.com/zefer/Combine/blob/master/com/magnoliabox/jsmin/JSMin.java - so you could isolate it further by picking the classes it uses, if you wanted. JSMin itself is a really old and unsupported project anyway, so personally, I wouldn't bother.

I do all my asset compilation at build time now. There are lots of modern tools that help you do this, and are far superior to Combine in my opinion.

Catmacey · 2013-06-11T20:29:40Z

Hi Joe,

Yes like you we do minification/etc using other tools but this particular
use was to compress some JS that's built automatically by CF from a large
database query.
In reality it's not that critical as it's fairly easy to control excess
whitespace in the output anyway. It was more of a "because you can" kind
of thing.
But like you... I can't be bothered either... not if the next step is to go
digging around in the Java source. :o)

Thanks again for the help.

Regards

Matt Casey

On 11 June 2013 15:07, Joe Roberts [email protected] wrote:

Hi Matt.

The JSmin source is available here
https://github.com/zefer/Combine/blob/master/com/magnoliabox/jsmin/JSMin.java- so you could isolate it further by picking the classes it uses, if you
wanted. JSMin itself is a really old and unsupported project anyway, so
personally, I wouldn't bother.

I do all my asset compilation at build time now. There are lots of modern
tools that help you do this, and are far superior to Combine in my opinion.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-19260149
.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error encoding JS files with UTF-8 encoding #4

Error encoding JS files with UTF-8 encoding #4

Catmacey commented Jun 11, 2013

zefer commented Jun 11, 2013

Catmacey commented Jun 11, 2013

zefer commented Jun 11, 2013

Catmacey commented Jun 11, 2013

Error encoding JS files with UTF-8 encoding #4

Error encoding JS files with UTF-8 encoding #4

Comments

Catmacey commented Jun 11, 2013

zefer commented Jun 11, 2013

Catmacey commented Jun 11, 2013

zefer commented Jun 11, 2013

Catmacey commented Jun 11, 2013