Introduce JSDoc-based Type System with `checkJs` flag #5318

mkslanc · 2023-09-17T14:26:47Z

Issue #, if available:

Description of changes:
This PR introduces a comprehensive type system for our plain JavaScript project, leveraging the power of JSDocs and TypeScript's checkJs feature.

Details:

All primary JavaScript files have been adorned with JSDoc type annotations to ensure type safety without converting the project to TypeScript.
Configured tsconfig.json to enable checkJs, allowing TypeScript to check our plain JavaScript files for type errors.
Refactored some parts of the codebase to address type inconsistencies and to ensure more robust typing.

Benefits:

Improved code clarity with explicit type annotations.
Early detection of type-related errors, leading to more robust code.
Provides a foundation for potential migration to TypeScript in the future, should the need arise.
Enhances the developer experience by providing type hints and autocompletion in supported editors.

P.S.: I left //@ts-expect-error with TODO in different places in codebase, where I think some bugs could occur
P.P.S.: not whole classes/ functions covered with types right now, somewhere I left any types, because it already become quite big change. Could be addressed in separate PR.

Updated JSDoc's for ace-code base
Pre-build ace.d.ts generator for ace-code and ace-builds
Fixed modes declarations
Resolved TODO's

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…h tsc

# Conflicts: # src/selection.js

# Conflicts: # src/autocomplete.js

# Conflicts: # package.json # src/lib/app_config.js # src/lib/lang.js

whazor · 2023-09-18T08:37:02Z

Nice work!! I agree on the any types, it would be best for the first PR to only add the start point, and we could slowly improve types over time.

kungfooman · 2023-09-18T08:55:52Z

src/keyboard/textinput.js


    text.style.opacity = "0";
    parentNode.insertBefore(text, parentNode.firstChild);

    var copied = false;
    var pasted = false;
+    /**
+     * 
+     * @type {false | {[key: string]: any}}}


Any opinion about this style vs Record<string, any>?

kungfooman · 2023-09-18T08:57:52Z

src/mode/behaviour/cstyle.js

    return types.indexOf(token.type || token) > -1;
 };

-CstyleBehaviour.recordAutoInsert = function(editor, session, bracket) {
+CstyleBehaviour["recordAutoInsert"] = function(editor, session, bracket) {


I would rather aim to make it work as it is, without trying to please TS language server too much by rewriting everything into ["..."]. You probably just need to define a proper typedef.

Generally I agree with you, but lets leave it for further improvements (especially whole mode/* part)

src/mode/text.js

kungfooman · 2023-09-18T09:08:18Z

Very nice work! 👍

I just happen to develop a tool right now which parses JavaScript projects with JSDoc via Babel and then adding automatic type assertions at runtime. I use it so far only for the PlayCanvas game engine and it helped me to squeeze out a bunch of bugs already and I'm looking to support more projects. However, it's under heavy construction right now because I prepare for an official release to make it work for all kinds of different projects:

https://github.com/kungfooman/RuntimeTypeInspector.js

(You can watch the first video to see how it works)

Once I have more time, I will look into the integration here. Very happy to see better JSDoc across all kinds of projects 💪 💯 🥇

mkslanc · 2023-09-18T16:26:48Z

@nightwing, @whazor, @akoreman, @InspiredGuy I'm hoping to gather thoughts and suggestions on a few decisions we're considering regarding new type system, because it would influence everyone's experience:

What is the preferred method for storing interface types such as Point/Position? Should we place them in ace.d.ts or utilize @typedef within the ace source files?
Do we intend to generate the corresponding .d.ts files after each PR merge, or is it solely for internal development to improve type assistance?
If we are generating .d.ts files, should the main declaration file depend on this generated declaration instead of the source code?
What would be the most effective method for creating declarations for ace-builds?
Regarding code style, should we use @typedef at the beginning of a file for type aliasing, such as:

/**
 * @typedef {import("../ace").Ace.Point} Point
 */

For each type sourced from ace.d.ts? Or, should we maintain it as import("../ace").Ace.Point for every parameter since we cannot create a type alias for the namespace, to the best of my knowledge?

kungfooman · 2023-09-19T07:40:11Z

For each type sourced from ace.d.ts? Or, should we maintain it as import("../ace").Ace.Point for every parameter since we cannot create a type alias for the namespace, to the best of my knowledge?

Typedef type aliasing has pretty much two issues:

They export automatically without a way to declare them as private, hence you have these issues: Support for opting out of jsdoc @typedef exports microsoft/TypeScript#46011 (comment)
Now you require two mouse clicks instead of one to get to the "real" type

If you don't care about two mouse clicks and just wanna put a ts-ignore over such export * from '...', you can use typedefs.

marinsokol5 · 2023-09-22T10:38:33Z

Hey, nice work!! 🚀

I think checkJs flag is a great addition to the codebase as indeed it gives us that nice extra verification step that stuff are as we type them to be.

I am kind of lost though with what we want to achieve with the generated types:

We generate .d.ts files from JSDoc and we put them into types folder but we don't publish this folder, so we still fully rely on ace.d.ts instead. So why do we generate them?
Like do we expect consumers to use an import from types like import {CommandBarTooltip} from "ace-code/types/ext/command_bar";
Or do we expect them to set "allowJs": true, in their tsconfig.json and then import from source by import {CommandBarTooltip} from "ace-code/src/ext/command_bar";

Do you know of any other popular open source project relying on JSDoc?
I am also curious on if the rely on types exported from their js files or do they maintain .d.ts files.

mkslanc · 2023-09-22T13:06:29Z

We generate .d.ts files from JSDoc and we put them into types folder but we don't publish this folder, so we still fully rely on ace.d.ts instead. So why do we generate them?

It could provide better intellisense for internal development purposes, but right now I also don't see a lot of benefits on generating types except the case when it would help to convert project to plain typescript.

Like do we expect consumers to use an import from types like import {CommandBarTooltip} from "ace-code/types/ext/command_bar";

Or do we expect them to set "allowJs": true, in their tsconfig.json and then import from source by import {CommandBarTooltip} from "ace-code/src/ext/command_bar";

Personally, I anticipate consumers won't have to change their usage much, which is why I retained the type aliases in ace.d.ts. The primary idea is to provide better types for external users and make it easier for new contributors to understand the codebase.

Do you know of any other popular open source project relying on JSDoc? I am also curious on if the rely on types exported from their js files or do they maintain .d.ts files.

I don't know of many projects using this approach, as it's often a temporary solution before migrating to full TypeScript. However, some examples include:

P.S.: many cases of uses mixins (EventEmitter and OptionsProvider mostly) in aces classes make us to provide

declare module "ace-code/src/editor" {
    export const Editor: {
        new(renderer: import("./src/virtual_renderer").IVirtualRenderer, session?: import("./src/edit_session").IEditSession, options?: Object): import("./src/editor").IEditor;
    };
}

or

import {Ace} from "../ace";
import {IEditSession} from "../src/edit_session";
import {IVirtualRenderer} from "../src/virtual_renderer";
export const Editor: {
    new(renderer: IVirtualRenderer, session?: IEditSession, options?: Object): Ace.Editor;
};

for any public ace module (actually everything in src dir)
and those who uses direct imports from ace-code codebase like import {Editor} from "ace-code/src/editor"; . At least I couldn't find better solution right now. If this is deemed an acceptable solution, I can write a generator based on the TS Language Service.

mkslanc · 2023-11-02T11:15:37Z

Also, according to different structure of ace-code and ace-builds it's a bit challenging to solve everything and not break something in process :)

marinsokol5 · 2023-11-02T11:23:57Z

@mkslanc Looks like you got quite far in the process indeed!

Can we keep it without Step 3 though?
So can we introduce JSDoc + checkJS mechanism without the .d.ts updates right now, so no significant changes to ace.d.ts nor auto generating it yet from JSDoc.
And then we have the separate discussion for how do we plan to maintain our types (do we auto-generate them only or do we have separate .d.ts or a mix of both).

That way we improve the situation right now and merge the majority of this PR but lower the risk of introducing consumer changes.

mkslanc · 2023-11-02T12:37:32Z

@marinsokol5 I think it shouldn't be hard to separate those parts - I will leave only JSDoc's changes + tsconfig in that PR and create separate PR with generator. But maybe we will need to separate "typing" in "package.json" for internal and external (published) usage. I need to do few tests to be sure of that

# Conflicts: # ace.d.ts # src/autocomplete.js

mkslanc · 2023-11-02T15:46:48Z

@marinsokol5:

For ace-code, we will use the newly corrected ace.d.ts for both external and internal usage. (It worked surprisingly smoothly for me.)
For ace-builds: Renamed old ace.d.ts to ace-internal.d.ts to prevent breaking changes.
The generator will be in a separate PR, and it seems we could use it only for ace-builds to generate one large declaration file, similar to what Monaco does.

Please test it thoroughly. I don't want to introduce any regressions to the codebase. :)

P.S.: from my side I tested it in different packages, where we used ace-builds and ace-code (ace-linters, ace-layout, ace-samples) and it worked.

marinsokol5 · 2023-11-03T10:14:56Z

@mkslanc

Is it possible to keep ace.d.ts as it is before this PR? I understand that types will be a bit outdated and we won't leverage our JSDoc exports, but it would be just temporary duplication. We can open that PR right after we merge this one, and then we figure out the auto-generation and what to do ace-code vs ace-build. Just for easier reviewing and a more specific focus of the review.

mkslanc · 2023-11-03T11:16:36Z

@marinsokol5 It's possible, but I will need to change all import("../ace").Ace. to import("../ace-internal").Ace. If this is acceptable, I will make changes

marinsokol5 · 2023-11-03T13:10:44Z

@mkslanc Is it possible to not have ace-internal.d.ts at this moment in time, just ace.d.ts?

mkslanc · 2023-11-03T13:15:54Z

@marinsokol5 Unfortunately, no. Types in JSDoc's are highly dependent on ace.d.ts declarations and especially on declare module "./src/virtual_renderer" with interfaces like that - export interface VirtualRenderer extends Ace.EventEmitter<Ace.VirtualRendererEvents>, Ace.OptionsProvider<Ace.VirtualRendererOptions>. You could try to remove any declare module statement (most representable would be removing editor/edit_session/virtual_renderer) and see drastic increase of errors in type checking.

mkslanc · 2023-11-03T13:23:20Z

@marinsokol5 Also we could set "checkJs" to false, but it would nullify all advantages of type-checking

# Conflicts: # ace.d.ts # src/autocomplete.js

nightwing · 2023-11-27T23:28:47Z

@mkslanc sadly we need to generate the .d.ts files otherwise this example is failing ajaxorg/ace-samples@master...ts, and we may even need the generated file to pass strict check which is specified in the failing test. but not sure about that.

# Conflicts: # ace.d.ts # src/autocomplete.js

codecov · 2023-12-04T14:50:32Z

Codecov Report

Attention: 33 lines in your changes are missing coverage. Please review.

Comparison is base (e8674fc) 87.58% compared to head (07b0241) 87.59%.

Files	Patch %	Lines
src/keyboard/gutter_handler.js	78.12%	7 Missing ⚠️
src/lib/event.js	68.18%	7 Missing ⚠️
src/editor.js	37.50%	5 Missing ⚠️
src/autocomplete.js	80.95%	4 Missing ⚠️
src/ext/inline_autocomplete.js	88.88%	2 Missing ⚠️
src/virtual_renderer.js	50.00%	2 Missing ⚠️
src/ext/static_highlight.js	83.33%	1 Missing ⚠️
src/keyboard/textinput.js	83.33%	1 Missing ⚠️
src/lib/report_error.js	0.00%	1 Missing ⚠️
src/line_widgets.js	75.00%	1 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5318      +/-   ##
==========================================
+ Coverage   87.58%   87.59%   +0.01%     
==========================================
  Files         583      583              
  Lines       46369    46389      +20     
  Branches     7016     7018       +2     
==========================================
+ Hits        40610    40636      +26     
+ Misses       5759     5753       -6

Flag	Coverage Δ
unittests	`87.59% <85.65%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mkslanc · 2023-12-04T14:52:03Z

@mkslanc sadly we need to generate the .d.ts files otherwise this example is failing ajaxorg/[email protected], and we may even need the generated file to pass strict check which is specified in the failing test. but not sure about that.

@nightwing Separate internal and external declaration files, should make it for now. I think generator should be separate PR, especially considering the fact that we need to run it in strict mode

nightwing · 2023-12-04T15:08:40Z

src/selection.js

@@ -77,7 +69,7 @@ class Selection {

    /**
     * Returns an object containing the `row` and `column` current position of the cursor.
-     * @returns {Object}
+     * @returns {import("../ace-internal").Ace.Point}


Could you convert this to a typedef, and do the same for other instances where same import() is used several times in a file?

Yes, sure, but users would need to click twice to see the real type declaration. If this is ok, I would process with changes across codebase

mkslanc added 15 commits August 25, 2023 18:02

correct many descriptions; automatic type declarations generation wit…

7e7f329

…h tsc

continue type casting

1d09f95

convert to es6 class

9110b45

continue providing types

ab962b9

continue providing types

84d383b

Merge branch 'selection-es6' into better-types

55f04df

# Conflicts: # src/selection.js

Merge branch 'master' into better-types

d9dc97e

# Conflicts: # src/autocomplete.js

continue providing types

f60e632

continue providing types

d8411db

Merge branch 'master' into better-types

25ba191

# Conflicts: # package.json # src/lib/app_config.js # src/lib/lang.js

resolve merge conflicts with new version

6cf736d

revert demo changes

5d05997

fix eslint errors

02c9a3c

fix bugs

9eb7d72

return demo

d9e9a19

kungfooman reviewed Sep 18, 2023

View reviewed changes

src/mode/text.js Outdated Show resolved Hide resolved

continue providing types

cd5ce9a

marinsokol5 mentioned this pull request Sep 20, 2023

add changeAnnotation event to types of EditSession #5319

Closed

describe available events for different classes

47027a1

mkslanc and others added 2 commits September 22, 2023 17:55

improvements in types

ba40d17

better way to merge interfaces

caaa69e

nightwing marked this pull request as ready for review September 26, 2023 00:33

fix left errors in generation

da92d09

mkslanc added 4 commits November 2, 2023 19:11

change declaration structure for ace-code

9a5b4d0

Merge branch 'master' into better-types

29ba8ab

# Conflicts: # ace.d.ts # src/autocomplete.js

add types to new declarations

f36b98e

remove declaration_generator; make ace-builds to use old declaration

16db9ca

marinsokol5 removed the needs-rebase label Nov 3, 2023

mkslanc mentioned this pull request Nov 17, 2023

Change generator to work with new JSDoc-based Type System ajaxorg/ace-api-docs#6

Merged

mkslanc added 2 commits November 17, 2023 18:59

Merge branch 'master' into better-types

a88c30d

# Conflicts: # ace.d.ts # src/autocomplete.js

update to last version

8581365

mkslanc added 3 commits December 4, 2023 18:36

return base ace.d.ts; use ace-internal for inner typings

cec9e2a

Merge branch 'master' into better-types

bd7c3ed

# Conflicts: # ace.d.ts # src/autocomplete.js

resolve merge conflicts

cc47699

nightwing reviewed Dec 4, 2023

View reviewed changes

nightwing force-pushed the better-types branch 2 times, most recently from 79a25f8 to c6398d6 Compare December 5, 2023 23:19

cleanup

07b0241

nightwing force-pushed the better-types branch from c6398d6 to 07b0241 Compare December 5, 2023 23:40

nightwing merged commit a7412b7 into ajaxorg:master Dec 6, 2023
2 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce JSDoc-based Type System with `checkJs` flag #5318

Introduce JSDoc-based Type System with `checkJs` flag #5318

mkslanc commented Sep 17, 2023 •

edited

Loading

whazor commented Sep 18, 2023

kungfooman Sep 18, 2023

kungfooman Sep 18, 2023

mkslanc Sep 19, 2023

kungfooman commented Sep 18, 2023

mkslanc commented Sep 18, 2023

kungfooman commented Sep 19, 2023

marinsokol5 commented Sep 22, 2023

mkslanc commented Sep 22, 2023

mkslanc commented Nov 2, 2023

marinsokol5 commented Nov 2, 2023 •

edited

Loading

mkslanc commented Nov 2, 2023

mkslanc commented Nov 2, 2023

marinsokol5 commented Nov 3, 2023 •

edited

Loading

mkslanc commented Nov 3, 2023

marinsokol5 commented Nov 3, 2023

mkslanc commented Nov 3, 2023 •

edited

Loading

mkslanc commented Nov 3, 2023

nightwing commented Nov 27, 2023

codecov bot commented Dec 4, 2023 •

edited

Loading

mkslanc commented Dec 4, 2023

nightwing Dec 4, 2023

mkslanc Dec 4, 2023

Introduce JSDoc-based Type System with checkJs flag #5318

Introduce JSDoc-based Type System with checkJs flag #5318

Conversation

mkslanc commented Sep 17, 2023 • edited Loading

Details:

Benefits:

whazor commented Sep 18, 2023

kungfooman Sep 18, 2023

Choose a reason for hiding this comment

kungfooman Sep 18, 2023

Choose a reason for hiding this comment

mkslanc Sep 19, 2023

Choose a reason for hiding this comment

kungfooman commented Sep 18, 2023

mkslanc commented Sep 18, 2023

kungfooman commented Sep 19, 2023

marinsokol5 commented Sep 22, 2023

mkslanc commented Sep 22, 2023

mkslanc commented Nov 2, 2023

marinsokol5 commented Nov 2, 2023 • edited Loading

mkslanc commented Nov 2, 2023

mkslanc commented Nov 2, 2023

marinsokol5 commented Nov 3, 2023 • edited Loading

mkslanc commented Nov 3, 2023

marinsokol5 commented Nov 3, 2023

mkslanc commented Nov 3, 2023 • edited Loading

mkslanc commented Nov 3, 2023

nightwing commented Nov 27, 2023

codecov bot commented Dec 4, 2023 • edited Loading

Codecov Report

mkslanc commented Dec 4, 2023

nightwing Dec 4, 2023

Choose a reason for hiding this comment

mkslanc Dec 4, 2023

Choose a reason for hiding this comment

Introduce JSDoc-based Type System with `checkJs` flag #5318

Introduce JSDoc-based Type System with `checkJs` flag #5318

mkslanc commented Sep 17, 2023 •

edited

Loading

marinsokol5 commented Nov 2, 2023 •

edited

Loading

marinsokol5 commented Nov 3, 2023 •

edited

Loading

mkslanc commented Nov 3, 2023 •

edited

Loading

codecov bot commented Dec 4, 2023 •

edited

Loading