-
Notifications
You must be signed in to change notification settings - Fork 24
Writing i18n tests
The Internationalization Working Group has created a large number of tests, accessed via a test framework, and provides results for those tests. To find tests and results, see the test index page. We port tests to the Web Platform Tests repository, but some of our tests are exploratory or test features not described in W3C specifications.
Basically, the test format we use conforms to that of the WPT documentation, and you should be familiar with those guidelines. You should also look at existing tests in the test suite to get some ideas about how to create similar tests. There are also some differences between our approach to test development and that of WPT, because our audience is different. Here we provide some essential guidelines for i18n tests.
Here is an example of a typical, short test file:
<!DOCTYPE html>
<html lang="en" >
<head>
<meta charset="utf-8">
<title>arabic text-shadow</title>
<meta name="assert" content="[exploratory] text-shadow on cursive Arabic text doesn't break the joins">
<link rel="help" href="https://drafts.csswg.org/css-text-decor-3/#text-shadow-property">
<link rel="author" title="r12a" href="mailto:[email protected]">
<style>
#test { text-shadow: -6px 4px 2px pink; }
/* the CSS below is not part of the test */
.test {
margin: .5em 5%;
font-size: 8em;
}
</style>
</head>
<body>
<div>Test passes if there are no pink lines between joined characters.</div>
<div class="test" id="test" lang="ar">نشاط التدويل</div>
</body>
</html>
Every file should have lang="en" in the HTML tag (unless it is testing language), and should have <meta charset="utf-8"/> (unless testing character encodings).
No line in the file should end with superfluous white space. If you are testing spaces at the end of a line, use  to represent the space. (This rule exists because WPT tests require it.)
Every file should contain the following, worded as concisely as possible (without being obscure!):
- a title element
- a
meta name="assert"element - a
link rel="help"element - a
link rel="author"element - a
divat the start of the body that gives instructions on how to tell if the test passes/fails - the test itself
All test files should test a single, specific thing. If you want to test multiple aspects of an issue, create multiple test files.
Let's look at these in a little more detail.
Generally, tests should display only the "Test passes if..." sentence and the test data itself. Occasionally, some additional controls may be added.
Other information describing the test, how it works, or so on, should be included in comments inside the source (and then only where useful), rather than appearing to the viewer. This makes it easier for a person scoring the tests to do so quickly, seeing only essential information.
In the i18n test framework, the assertion and the title are displayed alongside the actual test file, so there is no need to repeat them on the surface of the test file itself.
The title element is used in the test suite to help people spot a particular test at a glance. Ideally, every title should be unique to its test, and give an idea of what the test is about and how it differs from others. Conventionally, i18n tests use all lowercase.
In the i18n test results pages, if you include a colon in the title, only the text following the colon is displayed. For example, <title>CSS3 Text, text transform: capitalize, Dutch IJ</title> will be displayed as capitalize, Dutch IJ in the left column of a results table.
You must always include an assertion. The assertion tells you what the test sets out to check, and is one of the most important parts of the test. Without an assertion, reviewers and people looking at the results page cannot be sure what the test is about.
Writing an assertion is also good for you as a test writer, because it ensures that you have a clear idea what specifically you are setting out to test, and helps you ensure that you are testing a single thing. The assertion should be as concise as possible.
If this is an exploratory test, rather than one that tests a specific part of a specification, you should start the assertion with "[exploratory] ". Be sure to spell it correctly, as the test framework looks for this.
Use the href attribute to point to the section in the spec that describes the feature you are testing. This is picked up by the test framework.
If this is an exploratory test (so there is no actual spec text), try to point to a section of a spec that describes something similar.
This is useful if we need to change the test later, since it allows us to notify or consult the original author, where needed.
If you modify a test, you should add another such meta element.
When testing CSS features, it's usually helpful to add the relevant declarations at the start of the style element, and indicate which style declarations are not actually relevant to the test itself (see the example above).
When a test is displayed, people will want to know whether it passes or fails. It is essential that you indicate, as concisely as possible, what conditions indicate a pass.
The instructions will be displayed at the top of the page, and should always contain "Test passes if ...". You may precede or follow that with brief instructions, such as "Double-click on the middle word."
If you want to add further details, or provide controls for the use of the person looking at the test, include these in a separate div immediately below. Further details may be statements such as "Skip this test is vertical text is not supported."
The markup for a test is typically included in a div with an id and class called test.
The tests we develop for i18n are different from WPT because they are used educationally and target authors, they are not just for machine testing. This creates differences such as the following:
- We are most interested in tests that show whether the basic functionality works, and less interested in edge-cases.
- We prefer tests that allow you to see the feature in action, especially if it involves writing systems, rather than just producing a green box via a script. Therefore many of our tests use actual text in a given writing system, and we sometimes create the same test using a variety of writing systems for the text.
- Many of the tests we run cannot be automated.
- We prefer overlaying tests and references, where feasible, because it makes it easier for humans to grok the differences. This doesn't work as well for WPT variants of the test, since there tends to be some subpixel leaking of colour, so we need to change those tests before porting them to WPT. Hopefully, it's just a question of taking out the CSS that does the overlay.
- We need to be able to group and arrange test results in a way that ‘tells a story’ or illustrates how a feature works.
- We are less interested in nightly results than in results for production versions of browsers – the latter are the main focus for content authors. If we do record results for nightlies, we also record them for the latest released version of a browser.
This section provides some bits of code that may be useful.
Some special fonts are available for tests, including monospaced fonts. If you need to use them, check with the W3C staff.
It is often useful to use a webfont, either to avoid spatial variation across devices, or sometimes just to ensure that the person running a test can see all the characters. We typically use webfonts derived from Google's Noto fonts. Here is an example of some code you would put at the start of your style element for that:
@font-face {
font-family: 'webfont';
src: url('../../../fonts/noto/NotoSerifDevanagari-regular.woff2') format('woff');
font-weight: normal;
font-style: normal;
}
.test, .ref { font-family: webfont, serif; ... }
In some cases, it is useful to change the font used in the test, especially for exploratory tests. Here is some code you can use for that.
<p>Change the font to <input id="fontFamily" type="text" onChange="setFont(this.value)"/></p>
If you think it's useful to provide a list of fonts, you could use the following code:
<div class="control">Change the font:
<select onchange="document.getElementById('test').style.fontFamily = this.value; return false;">
<option value="0">Select...</option>
<option value="Al Bayan">Al Bayan</option>
<option value="Baghdad">Baghdad</option>
<option value="Decotype Naskh">Decotype Naskh</option>
<option value="Geeza Pro">Geeza Pro</option>
<option value="KufiStandardGK">KufiStandardGK</option>
<option value="Nadeem">Nadeem</option>
<option value="Damascus">Damascus</option>
...
</select>
</div>
To allow the person viewing the test to increase/decrease the font size, you could use the following code. Note that you should only do this where there is some need (typically in exploratory tests only).
<p>Set font size <input id="fontSize" min="10" max="120" step="1" value="36" oninput="document.getElementById('sizeOfFont').textContent = this.value; tests = document.querySelectorAll('.test'); for (let i=0;i<tests.length;i++) { tests[i].style.fontSize = this.value+'px';}" style="width:40%; margin-left: 5%;" type="range"/> <span id="sizeOfFont" style="font-size:80%;">36</span>px</p>
This is an example of a test that writes the text of the test over the reference text (relevant lines are marked with an initial *). The latter is coloured red, so that any differences show through. Particularly where there are several things to check (such as in text transform tests), this makes it much easier for people to see the differences, however, as noted above, these tests will need to be changed before porting to the WPT repository.
<!DOCTYPE html>
<html lang="en" >
<head>
<meta charset="utf-8"/>
<title>text transform capitalize, Dutch IJ</title>
<meta name="assert" content="[exploratory] The brower tailors text-transform: capitalize to put both I and J in titlecase at the start of a word when the language is Dutch.">
<link rel='author' title='r12a' href='mailto:[email protected]'>
<link rel='help' href='http://dev.w3.org/csswg/css-text-3/#text-transform'>
<link rel="match" href="reference/text-transform-tailoring-001-ref.html">
<style type='text/css'>
@font-face {
font-family: 'webfont';
src: url('../../fonts/sil/GentiumPlus-regular.woff2') format('woff');
font-weight: normal;
font-style: normal;
}
.test, .ref { font-size: 200%; line-height: 2.5em; font-family: 'Gentium Plus', 'Noto Serif', 'Noto Sans', webfont, sans-serif; }
.test span, .ref span { margin-right: 0; white-space: nowrap; }
span { display: inline-block; min-width: 2em; }
*.ref span { color: red; } .ref { z-index: -100; }
*.ref, .test { position:absolute; top:0; }
/* the CSS above is not part of the test */
.test { text-transform: capitalize; }
</style>
</head>
<body>
<p class="instructions">Test passes _if there is no red_.</p>
*<div style="position:relative">
<div class="test" lang="nl">ijsland</div>
<div class="ref">IJsland</div>
*</div>
</body>
</html>
In some cases (eg. when exploring line breaking behaviour) it can be useful to allow the person reading the page to expand or shrink the width of the box containing the test data. Here is some code you could use for that.
<div id="advice"> Use the slider to adjust the width of the box.<br/>
<input id="fontSizeSlider" min="10" max="90" step=".25" value="90" oninput="document.getElementById('sizeIndicator').textContent = Math.floor(window.innerWidth/100*this.value); tests = document.querySelectorAll('.test'); for (let i=0;i<tests.length;i++) { tests[i].style.width = this.value+'%';}" style="width:80%; margin-left: 5%;" type="range"> <span id="sizeIndicator" style="font-size:80%;">90</span>px</div>
<script>document.getElementById('sizeIndicator').textContent = Math.floor(window.innerWidth/100*90)</script>
That code will allow the user to simultaneously alter the width of all divs with the class name test.
This is done by the W3C staff person. They will bundle up your tests, or add them to other bundles, add them to the test framework, and create a results page.
Anyone can submit results for a test, or set of tests. Follow the online instructions while looking at a test, and then send the results to the W3C staff person.