Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use mapbox-gl-rtl-text to do Arabic text shaping and bidirectional layout (#3708) #3758

Merged
merged 1 commit into from
Jan 18, 2017

Conversation

ChrisLoer
Copy link
Contributor

@ChrisLoer ChrisLoer commented Dec 7, 2016

Fix issue #3708 and issue #3707 by using an Emscripten port of ICU to do Arabic text shaping and bidirectional layout.

screen shot 2016-12-07 at 11 02 35 am

The bulk of the functionality is implemented in https://github.com/mapbox/mapbox-icu-js/blob/master/src/icu.js, https://github.com/mapbox/mapbox-icu-js/blob/master/src/ushape_wrapper.c, and https://github.com/mapbox/mapbox-icu-js/blob/master/src/ubidi_wrapper.c.

Adding the ICU/Emscripten dependency adds considerable size to the mapbox-gl.js package and increases page load times. The current mapbox-icu-js build is the smallest I've been able to get by using supported Emscripten and ICU configuration options. I also run unassert on the Emscripten runtime to filter out unwanted asserts. It is probably possible to make further modifications within the ICU/Emscripten code to strip out unneeded ICU or runtime functionality.

In this PR, all ICU/Emscripten code is pulled directly into the mapbox-gl.js bundle. I also experimented with moving the Emscripten memory initialization data into an asynchronously fetched file in order to decrease JS parse times. Reloading the debug map several times in Chrome on my Macbook, I saw something on the order of 200ms extra page load time, with total scripting time running on the order of 1000ms. I did less testing but saw similar behavior on Firefox and Safari -- in all cases we depend on the browser caching JS compilation results so that the work isn't duplicated across the worker threads. The actual processing of labels once the map is loaded doesn't show up as a noticeable cost.

Baseline ICU in single bundle Asynchronous mem init
mapbox-gl.js size 485 KB 802 KB 655 KB (+73 KB async)
mapbog-gl.js compressed 126 KB 182 KB 176 KB (+12 KB async )
Additional parsing time 150-230ms 145-180ms

When/if we add HarfBuzz support for complex text shaping, we will at least get more mileage out of shared Emscripten components like the JS implementation of the C runtime.

cc @mourner @jfirebaugh @lucaswoj @pveugen

@jfirebaugh
Copy link
Contributor

jfirebaugh commented Dec 7, 2016

Thanks for running the numbers @ChrisLoer!

I'm more concerned about the increase in load time than the increase in bundle size. 150-200ms additional load time, on top of an already significant (order 1 second) load time, on high-end hardware, seems like a steep price to pay.

Can we dive deeper into this aspect of performance and see what our options are for improving it?

@ChrisLoer
Copy link
Contributor Author

ChrisLoer commented Dec 7, 2016

...on high-end hardware...

Well I'm not sure if my 2010-era Macbook is high-end, it certainly doesn't do GL as quickly as my iPhone. 😉 But for parsing javascript I suppose it's probably still pretty good.

One option I considered is deferring the load of mapbox-icu-js until we encounter a label that needs it (so this would only help load times with maps that didn't display any Arabic labels). The approach I experimented with was to make the mapbox-icu-js require just expose a (large!) base64 encoded string, and then I had the worker threads decode and evaluate the base64. After playing with it a while, I gave it up as too ugly.

@ChrisLoer
Copy link
Contributor Author

ChrisLoer commented Dec 7, 2016

I'm more concerned about the increase in load time than the increase in bundle size.

In practice I suspect reducing bundle size will be the most important way to decrease load time, because (1) downloading the bundle is part of overall load time (although it doesn't show up in my numbers), and (2) the load time seems to be driven pretty much entirely by javascript parsing, which I expect is roughly proportional to the size of the bundle (although parsing a big string literal is probably a lot cheaper than parsing the same amount of code).

Looking in the Emscripten runtime, there's a fair amount of code included that we don't use (UTF8/UTF32 support, error reporting and debug tracing code, support for running in javascript environments other than a worker thread, etc.), but just eyeballing it in an editor it looks like a few KB worth of code.

On the ICU side, I haven't found anything obviously unnecessary being included. I used the LLVM -print-callgraph-sccs option to look at the dead code elimination, and what I saw being pulled in looked necessary. The unicode character lookup tables in ushape.cpp and ubidi_props_data.h compress down to about 8KB, so as a first approximation that explains how we got to a 12KB compressed memory initialization file. ICU mixes C and C++ code, but I believe the code I'm pulling in is only dependent on the C runtime -- you can search for C++ runtime calls in the javascript by looking for ___cxa_*.

I'm currently experimenting with the -Oz optimization option, to favor size optimizations over performance optimizations.

@ChrisLoer
Copy link
Contributor Author

Using the -Oz option shaved 12KB off the bundle size, so I committed those changes and updated the table in the original comment. I assume that's good for load time but I don't have a test nearly precise enough to detect the difference reliably.

@ChrisLoer
Copy link
Contributor Author

ChrisLoer commented Dec 8, 2016

There's another approach we can try: instead of transpiling straight to javascript, we can compile most of the code to bytecode, and then run the bytecode in an interpreter. In theory, this should give us faster load times (because we don't have to parse so much javascript) but slower run times. It's also possible to start interpreted and switch to compiled to try to get the best of both worlds.

I tried the simplest version out: distribute our code as bytecode, load it asynchronously, and then run it in the interpreter.

Uncompressed Compressed
mapbox-icu-js (emscripten interpreter and wrapper code) 224 KB 36 KB
mapbox-gl.js (includes minified mapbox-icu-js) 571 KB 151 KB
memory initializer 73 KB 12 KB
ICU bytecode 77 KB 36 KB

Unfortunately, I didn't see a great improvement in load time: evaluating mapbox-icu-js still took around 150-200ms. We may be up against a lower bound for this technique because we have to include the "emterpreter" javascript code even if we're not including any of our code directly. Or I may need better techniques for profiling in Chrome!

Individual calls to applyArabicShaping and processBidirectionalText (the two functions that wrap everything we use from ICU) went from taking under a millisecond using the transpiled javascript with -Oz to a few milliseconds in the interpreted version. Faster is better, but either way it looked like a relatively small part of what the worker threads were doing.

@mourner
Copy link
Member

mourner commented Dec 8, 2016

Fantastic work! Another option for bundling this is to make it an external plugin that people can opt-in to. This is both simpler than lazy loading when encountering Arabic/RTL, and does not add any costs to developers that use other languages. Meanwhile, we could explore slimming down the Emscripten runtime.

@ChrisLoer
Copy link
Contributor Author

Another option for bundling this is to make it an external plugin that people can opt-in to.

If the performance implications are unacceptable, we might have to do this as a stop-gap, but it feels pretty unsatisfying. It's a hard choice to push on developers, who know less than we do about the performance implications, and in many cases may not know any more than we do about what languages their users expect to read. Hopefully it would still be helpful for motivated developers like @mushon.

@mourner, can I enlist you to do a little performance evaluation in your environment, just collecting data using the build in this PR? I'm worried we'll make too many decisions on just my numbers, and I may be mis-reading the timeline, or loading a pathological section of the map (I usually use the city of Mashhad, Iran, FWIW), or have too much other stuff running on my machine...

@kkaefer suggested I look at Cheerp, which transpiles C/C++ code to javascript that uses javascript-style memory instead of emulating a stack and heap. The big attraction for us with that approach is that it has a much more lightweight runtime. The downsides I see (just from a quick read of their docs) are: (1) limited support for C++ would be a big constraint on what code we could pull in, and (2) the license is either GPL or commercial.

It's not a strict requirement for an initial fix, but I'm definitely looking for a solution that we can build on for all of our text challenges (kerning/ligature support, indic text, language-aware line breaking). On the gl-native side there are platform-specific APIs that can do a lot of that for us, but on the gl-js side I still haven't seen anything that looks more promising than the ICU/Harfbuzz combo. I looked a little more at using Canvas to draw the text and then upload it as a WebGL texture, and noticed one more significant downside to that approach -- no support for line-breaking.

@ChrisLoer ChrisLoer self-assigned this Dec 8, 2016
@ChrisLoer ChrisLoer added the GL native → GL JS For feature parity with Mapbox Maps SDK on a native platform label Dec 8, 2016
@ChrisLoer
Copy link
Contributor Author

I spent a couple hours banging my head against getting Cheerp to build ICU. I didn't get very far, but I think I got far enough to give up (in combination with their unfavorable license). The problem that looked too difficult to fix was also very simple: because of its memory model Cheerp can't do memcpy on untyped pointers. The C parts of ICU are filled with code that moves raw bytes around in memory. I could start trying to put types on all of that code, but that seems like a bottomless 🕳️

There is CheerpJ, which we could use to pull in the Java version of ICU, but that's not even released yet, and I'm very skeptical it would end up being faster/lighter.

@ChrisLoer
Copy link
Contributor Author

I stripped the emscripten runtime as much as I felt I could get away with (I only touched the native javascript part of the runtime -- there's a large compiled part and I'd have to dig into the Emscripten source to try to thin that down). Amazingly, the maps still worked after all my hacking. Here are the updated code size stats:

Uncompressed Compressed
mapbox-icu-js (emscripten interpreter and wrapper code) 169 KB 24 KB
mapbox-gl.js (includes minified mapbox-icu-js) 543 KB 142 KB
memory initializer 73 KB 12 KB
ICU bytecode 77 KB 36 KB

But the darn thing still takes 150-200 ms to parse.

@ChrisLoer
Copy link
Contributor Author

Ha! It wasn't really a parsing cost, it's just that at parse time Emscripten creates one giant ArrayBuffer to represent all possible memory. Without understanding what it meant, I picked up a 256 MB "TOTAL_MEMORY" setting from a starter project -- and it was allocating that ArrayBuffer that was taking so long (curiously, I never saw JS Heap usage go over 30 MB in my testing, and I didn't see the process memory usage jump by a large amount either, so I don't know what exactly the browser is doing with that big call). I just tested with a 16 MB TOTAL_MEMORY limit, and the code loads in 40ms (and it seems to be enough memory for ICU to work).

I'll have to determine whether there's a safe (relatively low) memory limit I can put on the ICU code, or re-compile with the ALLOW_MEMORY_GROWTH option, which sounds safest but also is supposed to slow things down.

It looks like I can shave about 28KB (9KB compressed) off the bundle size by hacking down the run time -- so a 173 KB bundle would be 30% larger than the baseline, instead of the current 182 KB (44% larger). That would be nice to have, but I'm not sure it's worth having to kind-of-fork Emscripten.

@ChrisLoer
Copy link
Contributor Author

ChrisLoer commented Dec 9, 2016

Stats for the latest version of this PR:

  • mapbox-gl.js is 803KB/184KB compressed.
  • Evaluating mapbox-icu-js on page load takes on the order of 20ms
  • Profiling a worker thread while panning over a section of Iran showed the arabic shaping and bidirectional text processing calls coming out to about 4% of that worker's CPU time.

@mourner
Copy link
Member

mourner commented Dec 9, 2016

@ChrisLoer great progress! I'll explore performance on this branch further next week for another perspective. One thing to note is that code evaluation is not the biggest bottleneck — in my previous research on time-to-first-render (#3153), I discovered that worker instantiation time linearly depends on the size of the bundle. So if our current TTFR is around 2s, it is likely to get to ~3s with the ICU build.

in many cases may not know any more than we do about what languages their users expect to read

I think they know in most cases because it's a part of the style designing process. Our base styles have english transliterations by default.

@ChrisLoer
Copy link
Contributor Author

worker instantiation time linearly depends on the size of the bundle

To me, that seems like an argument for separating the icu-js code entirely from the main bundle. We could do something like:

  • Worker thread checks each label for ICU dependency
  • If label depends on ICU and ICU isn't loaded, asynchronously fetch ICU
  • Once ICU is loaded, re-draw labels that depend on it

That way, you'd have essentially no effect on maps that didn't need the functionality, and for maps that did need it you'd get the (maybe more graceful?) behavior of loading the base map quickly and then filling in labels. Hopefully the labels could fill in pretty quickly, although you'd introduce extra latency by waiting until the worker was already running to start the network request for the ICU code.

I don't have a good handle on how bad it is to split up the monolithic mapbox-gl.js bundle, though. We already have the dependency on asynchronously loading data from api.mapbox.com... would this be different hosting-wise because of the size, or because it's "code" instead of data?

Thinning the emscripten runtime would still help either way, but it seems like it might not be enough in the "monolithic" case, and it might not be necessary if we're lazy-loading for Arabic labels.

@jfirebaugh
Copy link
Contributor

Let's not do bundle splitting unless we've exhausted all other options. Splitting up the bundle adds significant friction to many use cases:

  • Anyone using Mapbox GL JS with non-Mapbox hosted data has a reasonable expectation that the code isn't going to make requests to api.mapbox.com. That host may not be whitelisted in CSP, blocking any requests.
  • Anyone bundling Mapbox GL JS with tools such as Browserify or Webpack will expect all necessary assets to be included. Anything that complicates the bundling process will add significant support load, as we've seen in the past (Remove support for building mapbox-gl with webpack and other non-browserify bundlers #3235).
  • Anyone hosting Mapbox GL JS on their own CDN will expect all script assets to be in a single .js file, since that's the norm. If we deviate from it, people will forget to upload the auxiliary file, or relative paths will break somehow, and it will be a common support issue.

@1ec5
Copy link
Contributor

1ec5 commented Dec 10, 2016

in many cases may not know any more than we do about what languages their users expect to read

I think they know in most cases because it's a part of the style designing process. Our base styles have english transliterations by default.

For what it’s worth, the lack of bidi and complex text shaping in Mapbox GL was the reason that Mapbox-designed styles wound up using {name_en} instead of {name}, as our raster styles did: mapbox/mapbox-gl-styles#81 (comment). If GL JS relegates bidi and complex text shaping to a plugin, we won’t be able to go back to {name} in these styles because we wouldn’t know at design time whether the host environment would support it.

Chicken, meet egg. Egg, chicken. 🐣

@ChrisLoer
Copy link
Contributor Author

I've updated the PR to run the closure compiler against the Emscripten preamble (the "native" javascript side of their runtime). It provides a very modest improvement by stripping out a few totally unused functions: from 803/182KB to 789/181KB.

Here are the things I was able to strip manually (~14KB worth) but can't easily remove without forking (or submitting change to) Emscripten:

  • Code detecting and responding to environment (node vs. shell vs. browser vs. worker)
  • Stack trace + C++ demangling code for use in abort() call
  • UTF8 conversion code that feeds into printf/stdio support
  • ERRNO table pulled in from C library that's almost completely unused
  • Logic for running a main loop
  • Error messages

@kkaefer
Copy link
Member

kkaefer commented Dec 13, 2016

I've had success with overriding __cxa_demangle in a C++ file. Generally many of the suggestions on http://ptspts.blogspot.de/2013/12/how-to-make-smaller-c-and-c-binaries.html are useful as well in this context.

@ChrisLoer
Copy link
Contributor Author

ChrisLoer commented Dec 13, 2016

@kkaefer Thanks for the pointer! I think the big thing there for me to dig into is whether I'm getting any exception-unwinding code pulled into my output even though none of the live code paths should catch or throw exceptions. merge_all_constants and no_unroll_loops also sound promising, although I'd expect them to already be part of -Oz. FWIW, the compiled ICU code doesn't link against __cxa_demangle, but emscripten has an alternative demangle function it defines for its own use when __cxa_demangle isn't pulled in.

If TTFR really is directly linearly related to uncompressed bundle size, maybe the most promising approach is to see what we can do to make the memory initializer smaller while keeping it in the bundle. The memory initializer is about 177KB minified, but if we take the binary, compress it, and then base 64 encode it, that's only 16KB. I bet we could find/make a javascript inflate that only took up a kilobyte or so. Of course, before the worker could run we'd still have to do the decoding and inflation, which might undermine the bundle size <-> load time relationship @mourner saw in #3153. It would at least give us a way to defer the cost until someone needed to process Arabic/bidirectional text.

@ChrisLoer
Copy link
Contributor Author

I just pushed a commit to stop using printf on error pathways in our wrapper code. This brings the total bundle size down from 789/181KB to 756/171KB. If we want to include some sort of built-in debugging support for error pathways, we could provide a thin wrapper of console.log that wouldn't pull in all the variable substitution stuff printf does.

Using the -g option for emcc allows me to see function names in the compiled output and is starting to give me a better idea of what's in there. There looks to be somewhere around 10-15KB of code supporting C++ functionality that's being called in because the (mostly C) file ushape.cpp uses this POD struct:

struct uShapeVariables {
     UChar tailChar;
     uint32_t uShapeLamalefBegin;
     uint32_t uShapeLamalefEnd;
     uint32_t uShapeTashkeelBegin;
     uint32_t uShapeTashkeelEnd;
     int spacesRelativeToTextBeginEnd;
};

Other than the somewhat extraneous C++ code, the set of compiled function calls looks like it's pretty well limited to what we're actually using for shaping and bidirectional support.

I'll turn my attention towards modifying Emscripten -- it looks more promising than modifying ICU, and hopefully we'll be able to take advantage of the same modifications when we try to pull in HarfBuzz.

@ChrisLoer
Copy link
Contributor Author

I made a kind of hacked-together build that includes mapbox-icu-js as a compressed string, and only decompresses and evaluates it on the worker threads. mapbox-icu-js compresses down to 72KB, 96KB base64 encoded. Including it, along with 46 KB worth of pako for decompression support, brings mapbox-gl.js to 636KB/216KB compressed.

It's hard for me to tell the overall effect on performance -- loading the map takes on the order of 2-4 seconds in all of these cases. Lazy-loading the ICU components takes about 200-300ms on each worker, of which roughly 2/3 is the inflate call. It also seems like it might be taking longer for the lazy-loaded code to warm up. The first calls to applyArabicShaping take almost 100ms, although subsequent calls come down to a few milliseconds.

I tried logging performance.now() in VectorTileSource.done to get an idea of "time to first tile". I'd then load the map several times, discarding the first result (which tended to be slower) and recording the fastest and slowest times. I did this for four configurations, then did it again, and got slower results the second time.... so I'm left without too much faith in the numbers. At any rate here they are:

Configuration TTFT First Pass TTFT Second Pass
No ICU 1.9-2.4s 2.5-3.0s
ICU directly bundled 2.3-2.6s 2.8-3.4s
ICU lazy-loaded 2.2-2.4s 3.0-3.4s
ICU included but not loaded 1.9-2.1s 2.6-3.0s

If I squint at those numbers long enough, I see something like a 15% penalty in load time for the two cases where ICU gets used, and not much of a load time penalty for the last case where ICU is included in the bundle but not decompressed/parsed at runtime (for that test I kept rendering the same view of Iran, but just disabled the runtime loading).

@ChrisLoer
Copy link
Contributor Author

@mourner I just started looking at how some of our other plugins work -- they seem to hook in on the main thread with a call to addControl. We'd have to pass the ICU plugin code to the worker threads, though, and it looks to me like webworkify assumes all code passed to the worker will be a build-time dependency. Did you have an idea of how it would make most sense to load the plugin?

@mourner
Copy link
Member

mourner commented Dec 23, 2016

@ChrisLoer we currently have a mechanism in place for custom sources here: https://github.com/mapbox/mapbox-gl-js/blob/master/js/source/worker.js#L83 — it allows you to pass an url which is then evaluated on the worker side. We might try to generalize it for the ICU case, @anandthakker what do you think?

@ChrisLoer
Copy link
Contributor Author

ChrisLoer commented Jan 6, 2017

The latest commit splits the mapbox-icu-js code out into a plugin that can be loaded at runtime:

map.addWorkerPlugin(
    'https://raw.githubusercontent.com/mapbox/mapbox-icu-js/master/mapbox-icu.js.min');

I'm very uncertain about the design of the "worker plugins", but at least the basic approach works. I'm getting page load times around 1300ms, but I can't directly compare that to the previous numbers since I finally upgraded to a new Macbook and everything is deliciously faster.

With the code split out into a separate plugin, I'm not sure where to put the Arabic tests (maybe build a test harness that lives with mapbox-icu-js?). I tried getting npm run test-suite to run against a locally stored copy of the plugin, and I was able to get it to work but it required a lot of hacking (since this functionality depends on importScripts which is stubbed out in the test context).

@1ec5
Copy link
Contributor

1ec5 commented Jan 14, 2017

This plugin isn't really needed for Hebrew anyways, right? Combining diacritics aren't used in OpenStreetMap, and medial/final variants are encoded separately in Unicode.

If it is necessary for Hebrew, how about mapbox-gl-ar-he, using ISO 639 codes for brevity? (Ignore the fact that gl stands for Galician. 😉)

@ChrisLoer
Copy link
Contributor Author

ChrisLoer commented Jan 14, 2017 via email

@mushon
Copy link

mushon commented Jan 14, 2017 via email

@ChrisLoer ChrisLoer force-pushed the cloer_arabic branch 2 times, most recently from 8158a30 to 547543b Compare January 18, 2017 03:00
@ChrisLoer
Copy link
Contributor Author

Settled on mapbox-gl-rtl-text as the plugin name, rebased, and squashed commits. Should be ready to go.

@1ec5 1ec5 changed the title Use mapbox-icu-js to do Arabic text shaping and bidirectional layout (#3708) Use mapbox-gl-rtl-text to do Arabic text shaping and bidirectional layout (#3708) Jan 18, 2017
- Full support for Unicode Bidirectional Algorithm
- Supports shaping Arabic script
Copy link
Contributor

@lucaswoj lucaswoj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do this! 🌏 🌍 🌎

@ChrisLoer
Copy link
Contributor Author

For anyone who's interested in trying these changes out before our February release, you can now clone the master branch and follow the contributing instructions to see how to make a local build of the mapbox-gl.js bundle. Look at docs/_posts/examples/3400-01-31-mapbox-gl-rtl-text.html for a simple example of how to turn on the plugin (the plugin URL in that example resolves to: https://api.mapbox.com/mapbox-gl-js/plugins/mapbox-gl-rtl-text/v0.1.0/mapbox-gl-rtl-text.js)

@Kashian
Copy link

Kashian commented Jan 26, 2017

@ChrisLoer Thank you Chris for this guide to clone the master branch. When would be the official date to release mapbox-gl.js . You mentioned February, but is it like between 1-5th Feb or 25th to 30th Feb..if it is for end of Feb, then I might have to put hands on cloning and testing it sooner. thanks.

@ChrisLoer
Copy link
Contributor Author

ChrisLoer commented Jan 26, 2017

@Kashian We ended up pushing our next release a little earlier than planned, you can play with the changes in v0.32.1, live as of this morning!

https://www.mapbox.com/mapbox-gl-js/api/#setRTLTextPlugin

@Kashian
Copy link

Kashian commented Jan 27, 2017

@ChrisLoer That is fantastic. You always come with good news. I am going to test it against Persian Map tonight (Melbourne time) and will post the results here. I probably will use a custom Persian font for the map.

@Kashian
Copy link

Kashian commented Jan 27, 2017

@ChrisLoer
Voila! This is a part of Tehran City. Successfully rendered with RTL Plugin for Persian Language.
Then:
I logged into the Mapbox Studio. Edited one of the popluar styles and changed label fonts to Custom Persian TTF font.
On Studio Preview, Persian TTF is shown, but preview on studio still does not support RTL plugin.
When I saved the style and used the style URL inside my custom map html page, I don't see the Persian Font loading and it is still the old font. Cache purged and different browsers tested. How come my custom font in custom style does not show up with this Plugin!

image

@1ec5
Copy link
Contributor

1ec5 commented Jan 27, 2017

Does your custom Persian font contain glyphs for the Arabic Presentation Forms-B character block?

@ChrisLoer
Copy link
Contributor Author

@Kashian Great to see RTL is working for you! You're right to note that Studio preview doesn't currently use the RTL plugin: before we either fold the plugin into the main JS bundle or figure out a clear way to make a "toggle" in Studio, we don't want to mislead Studio users by making RTL text appear to work for everyone in preview.

Did you make any other changes to your style, so you can make sure it was getting updated correctly? I confess I'm not very knowledgable about that process, but my first guess would be that there's some delay on getting the new style into the map. If you can verify that the style is.... wait @1ec5 just beat me to it. Yeah, if your font doesn't have the presentation forms, then we would fall back to a base font.

@Kashian
Copy link

Kashian commented Jan 27, 2017

@ChrisLoer Thanks. I just went out and came home and checked it again ! wonderfully this time it works with my TTF font. So obviously problem is resolved now and yes, @1ec5 , The TTF font had the Arab Presentation Forms-B. For the reference, I paste the result back again here. Default original font (used for park name) and the font I replaced for name of roads. Both works fine. Something must have been updated in past few hours. I will check more on shape of words on the corners and sharp turns and even use 2-3 more fonts to see what comes up. Thanks.
image

@mushon
Copy link

mushon commented Jan 31, 2017

@ChrisLoer first of all, you're a superhero, and you have our full support in keeping up with the good work you've been doing so far and seeing the process through until RTL languages are fully represented by default in every Mapbox map.
For now, I think @Kashian and I would not be the only ones waiting for some solution for activating RTL on the editor, as it is not really possible for us to design custom maps when the labels we use (either from Mapbox layers or from our own datasets) are unreadable.
I think a toggle on the debug menu makes sense (hey, look… there's even a mockup for that), what say you?
image

@ChrisLoer
Copy link
Contributor Author

@mushon I like the mockup! 😄 I understand that it's going to be difficult to design an RTL map in Studio when you're "blind" to label appearance, and I totally agree that RTL support isn't done until it shows up through the whole stack.

Unfortunately, I can't give you an estimate yet on when we'll have the support into Studio -- we're still figuring things out as we broaden our script support (RTL was just our first and highest priority). Doing something like you suggested in your mockup is definitely on our radar as a possible stop-gap. I can tell you that I won't consider myself done with the script support project until we have full support in Studio as well.

@mushon
Copy link

mushon commented Mar 19, 2017

@ChrisLoer, I understand you don't want the Mapbox Studio to misrepresent what can be done with Mapbox and since RTL is not fully supported in mobile it is not rolled into the studio, but in the meanwhile, we can't really design our maps as we have that exact problem - Mapbox Studio does not represent how our design will actually look like. Is there a way to maybe include this as a Chrome Extension or something, this stop-gap is even more needed now that we actually expect Mapbox to serve RTL languages. (thanks!)

@ChrisLoer
Copy link
Contributor Author

@mushon @ericrwolfe is working on an iOS app that will do live preview of studio edits -- it's not public yet, but it might be the closest stop-gap we have on the way.

@mushon
Copy link

mushon commented Mar 20, 2017

thank @ChrisLoer, I am not sure how would an iOS app be used in the map style design process. However, a quick blogpost allowing to hack the studio with a chrome extension could push us very quickly forward. Especially on acute mission critical features like running the RTL plugin in Mapbox Studio.

@ChrisLoer ChrisLoer mentioned this pull request Jun 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GL native → GL JS For feature parity with Mapbox Maps SDK on a native platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants