-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
First cut at typescript-lite generator implementation. #20
Changes from 10 commits
68b8537
0f3987c
365024d
4a28666
d76b78f
dfc04f7
d0c1c28
e9faaaa
52af142
4e03895
cbd365b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,9 @@ | ||
language: scala | ||
scala: 2.10.5 | ||
jdk: oraclejdk8 | ||
node_js: | ||
- "4" | ||
|
||
script: | ||
- sbt ++$TRAVIS_SCALA_VERSION fulltest | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
namespace org.coursera.arrays | ||
|
||
record WithAnonymousUnionArray { | ||
`unionsArray`: array[union[int, string]] | ||
`unionsMap`: map[string, union[string, int]] | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,8 @@ | ||
namespace org.coursera.enums | ||
|
||
/** | ||
* An enum dedicated to the finest of the food groups. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. :-) |
||
*/ | ||
enum Fruits { | ||
/** | ||
* An Apple. | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
.gradle | ||
build/ | ||
|
||
generator/build | ||
testsuite/testsuiteTests/generated | ||
|
||
# Ignore Gradle GUI config | ||
gradle-app.setting | ||
|
||
# Avoid ignoring Gradle wrapper jar file (.jar files are usually ignored) | ||
!gradle-wrapper.jar |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,259 @@ | ||
Typescript-Lite Courier Data Binding Generator | ||
============================================== | ||
|
||
Experimental! | ||
|
||
Lightweight Courier bindings for Typescript. | ||
|
||
The guiding philosophy here was to achieve type safety with as small a runtime footprint as possible -- use the fact | ||
that JSON is native to javascript to lightly define types and interfaces that describe your courier specs. | ||
|
||
These bindings will achieve their goal if: | ||
|
||
* You can just cast the result of a JSON HTTP response (or other JSON source) into the base expected | ||
type and stay typesafe without having to cast anything more afterwards. | ||
* You can hand-write JSON in your test-cases and API requests, and those test-cases | ||
fail to compile when the record contract breaks. | ||
* You can enjoy the sweet auto-complete and compile-time warnings you've come to enjoy from typescript. | ||
|
||
These bindings require Typescript 1.8+ | ||
|
||
Features missing in action | ||
-------------------------- | ||
|
||
* **Coercer support**. The strong runtime component of Coercers don't gel easily with the philosophy of casting records | ||
lightly into target typescript interfaces, so there wasn't an easy fit. | ||
* Maybe these get included when Typescript-lite becomes just plain ol typescript. | ||
* **Keyed maps**. Javascript objects naturally only support string-keyed maps. Since we are avoiding runtime | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What is the behavior when a keyed map is encountered? Will it just fail to compile? Do you get a warning? What is the generated type? (Apologies in advance if you already have a test case enforcing a particular behavior that I haven't gotten to yet.) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I believe that the generated type will be a How are typed maps sent down the wire? Are they just sent down as json objects or has the wire protocol been updated to include type information? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK yeah typed maps are converted into string -> value maps. And given what I understand of the wire protocol that should be fine. |
||
overhead as much as possible we did not want to introduce a new type, nor the means to serialize said type. | ||
* Integrating `Immutable.js` would make a lot of sense when we want to expand into this direction. | ||
* **Non-JSON serialization**. Although courier supports more compact binary formats (PSON, Avro, BSON), Typescript-lite | ||
bindings currently only supports valid JSON objects. | ||
* **Default values**. As with the other points, default values require a runtime. | ||
* **Flat-typed definitions**. This was just an oversight. Gotta add these. | ||
|
||
Running the generator from the command line | ||
------------------------------------------- | ||
|
||
Build a fat jar from source and use that. See the below section *Building a fat jar*. | ||
|
||
How code is generated | ||
--------------------- | ||
|
||
**Records:** | ||
|
||
* Records are generated as an interface within the typescript `namespace` specified by your record. | ||
* If you don't use a namespace, the interface will be injected at the top-level | ||
|
||
e.g. the result of [Fortune.courier](https://github.com/coursera/courier/blob/master/reference-suite/src/main/courier/org/example/Fortune.courier): | ||
|
||
```typescript | ||
// ./my-tslite-bindings/org.example.Fortune.ts | ||
import { FortuneTelling } from "./org.example.FortuneTelling"; | ||
import { DateTime } from "./org.example.common.DateTime"; | ||
|
||
/** | ||
* A fortune. | ||
*/ | ||
export interface Fortune { | ||
|
||
/** | ||
* The fortune telling. | ||
*/ | ||
telling : FortuneTelling; | ||
|
||
createdAt : DateTime; | ||
} | ||
``` | ||
|
||
**Enums:** | ||
|
||
* Enums are represented as [string literal types](https://basarat.gitbooks.io/typescript/content/docs/types/stringLiteralType.html). | ||
* Convenience constants matching the string literals are provided despite having a runtime cost | ||
* TODO(erem) We should consider changing this. Not sure if the benefit outweighs the wire cost. | ||
* Unlike other bindings, Typescript-lite does not include an `UNKNOWN$` option. In case of wire inconsistency | ||
you will have to just fall through to `undefined`. | ||
|
||
e.g. the result of [MagicEightBallAnswer.courier](https://github.com/coursera/courier/blob/master/reference-suite/src/main/courier/org/example/MagicEightBallAnswer.courier): | ||
|
||
```typescript | ||
// ./my-tslite-bindings/org.example.MagicEightBallAnswer.ts | ||
/** | ||
* Magic eight ball answers. | ||
*/ | ||
export type MagicEightBallAnswer = "IT_IS_CERTAIN" | "ASK_AGAIN_LATER" | "OUTLOOK_NOT_SO_GOOD" ; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @eboto Sorry, I'm not familiar with the idioms of typescript. Can you provide a rationale for not including There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure! We use That pretty much leaves it on the client to compare to all the KNOWN enums, and let the unknown one fall off the end. e.g. if you have const fruit: Fruit = /* get the fruit */
if (fruit == Fruit.APPLE) {
// something
} else if (fruit == Fruit.ORANGE) {
// something
} else {
// handle unknown
} Now thinking about it though, I could provide a utility function like Did that make sense? What do you think? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks pretty reasonable. Given that they're represented as strings, it should be reasonable to use them in a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good point! Don't know why I reached for this gross if-elses first. I'm going to give the switch usage a workup in the testsuite then update the docs. According to this typescript issue, we will get compiler warnings future versions for strings that are not in the string literal type...so that will make it even nicer (as long as it doesn't stop you from falling through to EDIT: Also I think I agree with you about the constants down the wire. I'm going to keep them. =) |
||
export module MagicEightBallAnswer { | ||
|
||
export const IT_IS_CERTAIN: MagicEightBallAnswer = "IT_IS_CERTAIN"; | ||
|
||
export const ASK_AGAIN_LATER: MagicEightBallAnswer = "ASK_AGAIN_LATER"; | ||
|
||
export const OUTLOOK_NOT_SO_GOOD: MagicEightBallAnswer = "OUTLOOK_NOT_SO_GOOD"; | ||
} | ||
``` | ||
|
||
**Arrays:** | ||
|
||
* Arrays are represented as typescript arrays. | ||
|
||
**Maps:** | ||
|
||
* Maps are represented as javascript objects, as interfaced by `courier.Map<ValueT>` | ||
* Only string-keyed maps are currently supported. | ||
|
||
**Unions:** | ||
|
||
* Unions are represented as an intersection type between all the members of the union. | ||
* A Run-time `unpack` accessor is provided to test each aspect of the union | ||
* Unlike other serializers, no `UNKNOWN$` member is generated for the union. If all provided accessors end up yielding | ||
undefined, then conclude that it was an unknown union member (see second example) | ||
|
||
e.g. The result of [FortuneTelling.pdsc](https://github.com/coursera/courier/blob/master/reference-suite/src/main/pegasus/org/example/FortuneTelling.pdsc): | ||
```typescript | ||
// file: my-tslite-bindings/org.example.FortuneTelling.ts | ||
import { MagicEightBall } from "./org.example.MagicEightBall"; | ||
import { FortuneCookie } from "./org.example.FortuneCookie"; | ||
|
||
export type FortuneTelling = FortuneTelling.FortuneCookieMember | FortuneTelling.MagicEightBallMember | FortuneTelling.StringMember; | ||
export module FortuneTelling { | ||
export interface FortuneTellingMember { | ||
[key: string]: FortuneCookie | MagicEightBall | string; | ||
} | ||
|
||
export interface FortuneCookieMember extends FortuneTellingMember { | ||
"org.example.FortuneCookie": FortuneCookie; | ||
} | ||
|
||
export interface MagicEightBallMember extends FortuneTellingMember { | ||
"org.example.MagicEightBall": MagicEightBall; | ||
} | ||
|
||
export interface StringMember extends FortuneTellingMember { | ||
"string": string; | ||
} | ||
|
||
export function unpack(union: FortuneTelling) { | ||
return { | ||
fortuneCookie: union["org.example.FortuneCookie"] as FortuneCookie, | ||
magicEightBall: union["org.example.MagicEightBall"] as MagicEightBall, | ||
string_: union["string"] as string | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like you're doing some escaping of some of the types. We've used There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I believe that |
||
}; | ||
} | ||
} | ||
``` | ||
|
||
Here's how you would use one: | ||
```typescript | ||
import { FortuneTelling } from "./my-tslite-bindings/org.example.FortuneTelling"; | ||
import { MacigEightBall } from "./my-tslite-bindings/org.example.MagicEightBall"; | ||
import { FortuneCookie } from "./my-tslite-bindings/org.example.FortuneCookie"; | ||
|
||
const telling: FortuneTelling = /* Get the union from somewhere...probably the wire */; | ||
|
||
const { fortuneCookie, magicEightBall, string_ } = FortuneTelling.unpack(telling); | ||
|
||
if (fortuneCookie) { | ||
// do something with fortuneCookie | ||
} else if (magicEightBall) { | ||
// do something with magicEightBall | ||
} else if (string_) { | ||
// do something with str | ||
} else { | ||
throw 'a fit because no one will tell your fortune'; | ||
} | ||
``` | ||
|
||
Projections and Optionality | ||
--------------------------- | ||
|
||
It is common to send and receive partial data through REST. This | ||
is very commonly used when a subset of fields of a resources are "projected". | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We've been tussling internally with how to maintain type safety while supporting projections. While for our JS clients it's not important, for our mobile clients it is. Can you elaborate on why you went down this route, and what the advantages and disadvantages have been in your experience? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'll freely admit that I haven't thought much about projections because they aren't part of my company's use case right now. That was just a part of the documentation that I overlooked. I should revisit it. I think Intersection types may be a good approach in typescript Imagine the following courier type:
If we wanted to support projections, we could generate the following types module Message {
interface Id {
id: string;
}
interface Subject {
subject: string;
}
interface Body {
body: string;
}
}
type Message = Message.Id & Message.Subject & Message.Body; In your application code when you request some projection, instead of using the Message type you could just safely cast the message down to its component projections! For example: function getMessageIdAndBody(id: string): Promise<Message.Id & Message.Body> {
return http.get('/messages/' + id + '?projection=(id,body)').then((resp) => {
return resp.data as (Message.Id & Message.Body);
})
} Any attempt to access Anyways. I haven't actually tried any of that out, but I'm pretty certain it would work. So projections work pretty smoothly in typescript. =) If it sounds good, I can put this intent into the docs and replace the current description of projections. |
||
|
||
Since even fields that are marked as required in a Pegasus schema may be absent when data is | ||
projected, Courier's [Optionality](https://github.com/coursera/courier/blob/master/typescript/generator/src/main/java/org.coursera.courier.tslite.TSProperties.java#L31) | ||
settings defaults to REQUIRED_FIELDS_MAY_BE_ABSENT. This allows a single | ||
generated typescript struct to be used for bindings to unprojected and projected data. | ||
|
||
If this behaviour is not desired, one may set `Optionality` to `STRICT`. | ||
|
||
See the [Optionality property docs](https://github.com/coursera/courier/blob/master/typescript/generator/src/main/java/org.coursera.courier.tslite.TSProperties.java#L8) | ||
for details on how to set the `Optionality` property. | ||
|
||
Custom Types | ||
------------ | ||
|
||
Custom Types allow any Typescript type to be bound to any pegasus primitive type. | ||
|
||
For example, say a schema has been defined to represent a "date time" as a unix timestamp long: | ||
|
||
``` | ||
namespace org.example | ||
|
||
typeref DateTime = long | ||
``` | ||
|
||
This results in a typescript file: | ||
```typescript | ||
// ./my-tslite-bindings/org.coursera.customtypes.DateTime.ts | ||
export type DateTime = number; | ||
``` | ||
|
||
JSON Serialization / Deserialization | ||
------------------------------------ | ||
|
||
JSON serialization is trivial in typescript. Just take use `JSON.stringify` on any courier type that compiles. | ||
|
||
```typescript | ||
import { Message } from "./my-tslite-bindings/org.coursera.records.Message"; | ||
|
||
const message: Message = {"body": "Hello Pegasus!"}; | ||
const messageJson = JSON.stringify(message); | ||
``` | ||
|
||
And of course you can read results as well. | ||
|
||
```typescript | ||
import { Message } from "./my-tslite-bindings/org.coursera.records.Message"; | ||
|
||
const messageStr: string = /* Get the message string somehow */ | ||
const message = JSON.parse(messageStr) as Message; | ||
``` | ||
|
||
Runtime library | ||
--------------- | ||
|
||
All generated Typescript-lite bindings depend on a `CourierRuntime.ts` class. This class provides the very minimal | ||
functionality and type definitions necessary for generated bindings to work. | ||
|
||
Building from source | ||
-------------------- | ||
|
||
See the main CONTRIBUTING document for details. | ||
|
||
Building a Fat Jar | ||
------------------ | ||
|
||
```sh | ||
$ sbt | ||
> project typescript-lite-generator | ||
> assembly | ||
``` | ||
|
||
This will build a standalone "fat jar". This is particularly convenient for use as a standalone | ||
commandline application. | ||
|
||
Testing | ||
------- | ||
|
||
1. No testing has been done yet except verifying that the generated files from reference-suite pass `tsc`. | ||
That's next on the list =) | ||
|
||
TODO | ||
---- | ||
|
||
* [ ] Add support for flat type definitions | ||
* [ ] Figure out the best way to distribute the 'fat jar'. | ||
* [ ] Consider getting rid of the string literal constants in generated enums. They may not give much for the wire cost. | ||
* [ ] Automate distribution of the Fat Jar | ||
* [ ] Publish Fat Jar to remote repos? Typically fat jars should not be published to maven/ivy | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Definitely happy to work with you on this. I've been looking at Bintray, but haven't investigated further. Do you have any experience here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hm unfortunately I don't have much experience with it. My first thoughts are just to be messy and toss the damned thing up onto S3, or even onto IPFS which would be amazing. |
||
repos, but maybe it should be hosted for easy download elsewhere? |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
import sbt.inc.Analysis | ||
|
||
name := "courier-typescript-lite-generator-test" | ||
|
||
packagedArtifacts := Map.empty // do not publish | ||
|
||
libraryDependencies ++= Seq( | ||
ExternalDependencies.JodaTime.jodaTime) | ||
|
||
autoScalaLibrary := false | ||
|
||
crossPaths := false | ||
|
||
// Test Generator | ||
forkedVmCourierGeneratorSettings | ||
|
||
forkedVmCourierMainClass := "org.coursera.courier.TypeScriptLiteGenerator" | ||
|
||
forkedVmCourierClasspath := (dependencyClasspath in Runtime in typescriptLiteGenerator).value.files | ||
|
||
forkedVmSourceDirectory := (sourceDirectory in referenceSuite).value / "main" / "courier" | ||
|
||
forkedVmCourierDest := file("typescript-lite") / "testsuite" / "src" / "tslite-bindings" | ||
|
||
forkedVmAdditionalArgs := Seq("STRICT") | ||
|
||
(compile in Compile) := { | ||
(forkedVmCourierGenerator in Compile).value | ||
|
||
Analysis.Empty | ||
} | ||
|
||
lazy val npmTest = taskKey[Unit]("Executes NPM test") | ||
|
||
npmTest in Test := { | ||
(compile in Compile).value | ||
|
||
val result = """./typescript-lite/testsuite/full-build.sh"""! | ||
|
||
if (result != 0) { | ||
throw new RuntimeException("NPM Build Failed") | ||
} | ||
} | ||
|
||
test in Test := (npmTest in Test).value |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
src/test/mainGeneratedPegasus | ||
src/main/mainGeneratedPegasus |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are these surrounded by ticks?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Completely unnecessary...just didn't think to change it from the file I copied to make this test-case.
I'll update that in a followup pr...