diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index f81f43f7..40966eae 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-04-05T10:33:53","documenter_version":"1.3.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-04-05T10:43:21","documenter_version":"1.3.0"}} \ No newline at end of file diff --git a/dev/index.html b/dev/index.html index 2a243d91..6118a2f3 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,3 +1,3 @@ Home · StaticLint.jl

StaticLint

Dev Project Status: Active - The project has reached a stable, usable state and is being actively developed. codecov.io

Static Code Analysis for Julia

Installation and Usage

using Pkg
-Pkg.add("StaticLint")
using StaticLint

Documentation: Dev

Description

This package supports LanguageServer.jl functionality broadly by:

  1. linking the file tree of a project
  2. marking scopes/namespaces within the syntax tree (ST)
  3. marking variable bindings (functions, instances, etc.)
  4. linking identifiers (i.e. variable names) to the relevant bindings
  5. marking possible errors within the ST

Identifying and marking errors (5.) is, in general, dependent on steps 1-4. These are achieved through a single pass over the ST of a project. A pass over a single EXPR is achieved through calling a State object on the ST. This State requires an AbstractServer that determines how files within a project are loaded and makes packages available for loading.

Passes

For a given experssion x this pass will:

  • Handle import statements (resolve_import). This either explicitly imports variables into the current state (for statements such as import/using SomeModule: binding1, binding2) or makes the exported bindings of a modules available more generally (e.g. using SomeOtherModule). The availability of includable packages is handled by the getsymbolserver function called on the state.server.
  • Determine whether x introduces a new variable. mark_bindings! performs this and may mark bindings for child nodes of x (e.g. when called on an expression that defines a Function this will mark the arguments of the signature as introducing bindings.)
  • Adds any binding associated with x to the variable list of the current scope (add_binding).
  • Handles global variables (mark_globals).
  • Special handling for macros introducing new bindings as necessary, at the moment limited to deprecate, enum, goto, label, and nospecialize.
  • Adds new scopes for the interior of x as needed (scopes).
  • Resolves references for identifiers (i.e. a variable name), macro name, keywords in function signatures and dotted names (e.g. A.B.c). A name is first checked against bindings introduced within a scope then against exported variables of modules loaded into the scope. If this fails to resolve the name this is repeated for the parent scope. References that fail to resolve at this point, and are within a delayed scope (i.e. within a function) are added to a list to be resolved later.
  • If x is a call to include(path_expr) attempt to resolve path_expr to a loadable file from state.server and pass across the files ST (followinclude).
  • Traverse across child nodes of x (traverse) in execution order. This means, for example, that in the expression a = b we traverse b then a (ignoring the operator).

Server

As mentioned, an AbstractServer is required to hold files within a project and provide access to user installed packages. An implementation must support the following functions:

StaticLint.hasfile(server, path)::Bool : Does the server have a file matching the name path.

StaticLint.getfile(server, path)::AbstractFile : Retrieves the file path - assumes the server has the file.

StaticLint.setfile(server, path, file)::AbstractFile : Stores file in the server under the name path, returning the file.

StaticLint.canloadfile(server, path)::Bool : Can the server load the file denoted by path, likely from an external source.

StaticLint.loadfile(server, path)::AbstractFile : Load the file at path from an external source (i.e. the hard drive).

StaticLint.getsymbolserver(server)::Dict{String,SymbolServer.ModuleStore} : Retrieve the server's depot of loadable packages.

An AbstractFile must support the following:

StaticLint.getpath(file) : Retrieve the path of a file.

StaticLint.getroot(file) : Retrieve the root of a file. The root is the main/first file in a file structure. For example the StaticLint.jl file is the root of all files (including itself) in src/.

StaticLint.setroot(file, root) : Set the root of a file.

StaticLint.getcst(file) : Retrieve the cst of a file.

StaticLint.setcst(file, cst::CSTParser.EXPR) : Set the cst of a file.

StaticLint.getserver(file) : Retrieve the server holding of a file.

StaticLint.setserver(file, server::AbstractServer) : Set the server of a file.

StaticLint.semantic_pass(file, target = nothing(optional)) : Run a full pass on the ST of a project (i.e. ST of all linked files). It is expected that file is the root of the project.

+Pkg.add("StaticLint")
using StaticLint

Documentation: Dev

Description

This package supports LanguageServer.jl functionality broadly by:

  1. linking the file tree of a project
  2. marking scopes/namespaces within the syntax tree (ST)
  3. marking variable bindings (functions, instances, etc.)
  4. linking identifiers (i.e. variable names) to the relevant bindings
  5. marking possible errors within the ST

Identifying and marking errors (5.) is, in general, dependent on steps 1-4. These are achieved through a single pass over the ST of a project. A pass over a single EXPR is achieved through calling a State object on the ST. This State requires an AbstractServer that determines how files within a project are loaded and makes packages available for loading.

Passes

For a given experssion x this pass will:

Server

As mentioned, an AbstractServer is required to hold files within a project and provide access to user installed packages. An implementation must support the following functions:

StaticLint.hasfile(server, path)::Bool : Does the server have a file matching the name path.

StaticLint.getfile(server, path)::AbstractFile : Retrieves the file path - assumes the server has the file.

StaticLint.setfile(server, path, file)::AbstractFile : Stores file in the server under the name path, returning the file.

StaticLint.canloadfile(server, path)::Bool : Can the server load the file denoted by path, likely from an external source.

StaticLint.loadfile(server, path)::AbstractFile : Load the file at path from an external source (i.e. the hard drive).

StaticLint.getsymbolserver(server)::Dict{String,SymbolServer.ModuleStore} : Retrieve the server's depot of loadable packages.

An AbstractFile must support the following:

StaticLint.getpath(file) : Retrieve the path of a file.

StaticLint.getroot(file) : Retrieve the root of a file. The root is the main/first file in a file structure. For example the StaticLint.jl file is the root of all files (including itself) in src/.

StaticLint.setroot(file, root) : Set the root of a file.

StaticLint.getcst(file) : Retrieve the cst of a file.

StaticLint.setcst(file, cst::CSTParser.EXPR) : Set the cst of a file.

StaticLint.getserver(file) : Retrieve the server holding of a file.

StaticLint.setserver(file, server::AbstractServer) : Set the server of a file.

StaticLint.semantic_pass(file, target = nothing(optional)) : Run a full pass on the ST of a project (i.e. ST of all linked files). It is expected that file is the root of the project.

diff --git a/dev/objects.inv b/dev/objects.inv index 91dbe5d3..3369388e 100644 Binary files a/dev/objects.inv and b/dev/objects.inv differ diff --git a/dev/syntax/index.html b/dev/syntax/index.html index 7c518a39..f08fb5bb 100644 --- a/dev/syntax/index.html +++ b/dev/syntax/index.html @@ -1,3 +1,3 @@ -Syntax Reference · StaticLint.jl

Syntax Reference

Main

StaticLint.followinclude — Method
followinclude(x, state)

Checks whether the arguments of a call to include can be resolved to a path. If successful it checks whether a file with that path is loaded on the server or a file exists on the disc that can be loaded. If this is successful it traverses the code associated with the loaded file.

source
StaticLint.get_path — Method
get_path(x::EXPR)

Usually called on the argument to include calls, and attempts to determine the path of the file to be included. Has limited support for joinpath calls.

source
StaticLint.semantic_pass — Function
semantic_pass(file, modified_expr=nothing)

Performs a semantic pass across a project from the entry point file. A first pass traverses the top-level scope after which secondary passes handle delayed scopes (e.g. functions). These secondary passes can be, optionally, very light and only seek to resovle references (e.g. link symbols to bindings). This can be done by supplying a list of expressions on which the full secondary pass should be made (modified_expr), all others will receive the light-touch version.

source
StaticLint.traverse — Method
traverse(x, state)

Iterates across the child nodes of an EXPR in execution order (rather than storage order) calling state on each node.

source
StaticLint.Binding — Type

Bindings indicate that an EXPR may introduce a new name into the current scope/namespace. Struct fields:

  • name: the EXPR that defines the unqualifed name of the binding.
  • val: what the binding points to, either a Binding (indicating ..), EXPR (this is generally the expression that defines the value) or SymStore.
  • type: the type of the binding, either a Binding, EXPR, or SymStore.
  • refs: a list containing all references that have been made to the binding.
source
StaticLint.add_binding — Function
add_binding(x, state, scope=state.scope)

Add the binding of x to the current scope. Special handling is required for:

  • macros: to prefix the @
  • functions: These are added to the top-level scope unless this syntax is used to define a closure within a function. If a function with the same name already exists in the scope then it is not replaced. This enables the refs list of the Binding of that 'root method' to hold a method table, the name of the new function will resolve to the binding of the root method (to get a list of actual methods -[get_method(ref) for ref in binding.refs if get_method(ref) !== nothing]). For example
[1] f() = 1
-[2] f(x) = 2

[1] is the root method and the name of [2] resolves to the binding of [1]. Functions declared with qualified names require special handling, there are comments in the source.

Some simple type inference is run.

source
StaticLint.mark_bindings! — Method
mark_bindings!(x::EXPR, state)

Checks whether the expression x should introduce new names and marks them as needed. Generally this marks expressions that would introdce names to the current scope (i.e. that x sits in) but in cases marks expressions that will add names to lower scopes. This is done when it is not knowable that a child node of x will introduce a new name without the context of where it sits in x -for example the arguments of the signature of a function definition.

source
StaticLint.lint_file — Function
lint_file(rootpath, server)

Read a file from disc, parse and run a semantic pass over it. The file should be the root of a project, e.g. for this package that file is src/StaticLint.jl. Other files in the project will be loaded automatically (calls to include with complicated arguments are not handled, see followinclude for details). A FileServer will be returned containing the Files of the package.

source
StaticLint.lint_string — Function
lint_string(s, server; gethints = false)

Parse a string and run a semantic pass over it. This will mark scopes, bindings, references, and lint hints. An annotated EXPR is returned or, if gethints = true, it is paired with a collected list of errors/hints.

source
StaticLint.interpret_eval — Method

interpret_eval(x::EXPR, state)

Naive attempt to interpret x as though it has been eval'ed. Lifts any bindings made within the scope of x to the toplevel and replaces (some) interpolated binding names with the value where possible.

source
StaticLint.resolve_getfield — Method
resolve_getfield(x::EXPR, parent::Union{EXPR,Scope,ModuleStore,Binding}, state::State)::Bool

Given an expression of the form parent.x try to resolve x. The method called with parent::EXPR resolves the reference for parent, other methods then check whether the Binding/Scope/ModuleStore to which parent points has a field matching x.

source
StaticLint.valofid — Method
valofid(x)

Returns the string value of an expression for which isidentifier is true, i.e. handles NONSTDIDENTIFIERs.

source
StaticLint.scopes — Method
scopes(x::EXPR, state)

Called when traversing the syntax tree and handles the association of scopes with expressions. On the first pass this will add scopes as necessary, on following passes it empties it.

source
StaticLint.getenv — Method
getenv(file::File, server::FileServer)

Get the relevant ExternalEnv for a given file.

source

Linting

StaticLint.check_kw_default — Method
check_kw_default(x::EXPR, server)

Check that the default value matches the type for keyword arguments. Following types are checked: String, Symbol, Int, Char, Bool, Float32, Float64, UInt8, UInt16, UInt32, UInt64, UInt128.

source
StaticLint.collect_hints — Function

collect_hints(x::EXPR, env, missingrefs = :all, isquoted = false, errs = Tuple{Int,EXPR}[], pos = 0)

Collect hints and errors from an expression. missingrefs = (:none, :id, :all) determines whether unresolved identifiers are marked, the :all option will mark identifiers used in getfield calls."

source
+Syntax Reference · StaticLint.jl

Syntax Reference

Main

StaticLint.followinclude — Method
followinclude(x, state)

Checks whether the arguments of a call to include can be resolved to a path. If successful it checks whether a file with that path is loaded on the server or a file exists on the disc that can be loaded. If this is successful it traverses the code associated with the loaded file.

source
StaticLint.get_path — Method
get_path(x::EXPR)

Usually called on the argument to include calls, and attempts to determine the path of the file to be included. Has limited support for joinpath calls.

source
StaticLint.semantic_pass — Function
semantic_pass(file, modified_expr=nothing)

Performs a semantic pass across a project from the entry point file. A first pass traverses the top-level scope after which secondary passes handle delayed scopes (e.g. functions). These secondary passes can be, optionally, very light and only seek to resovle references (e.g. link symbols to bindings). This can be done by supplying a list of expressions on which the full secondary pass should be made (modified_expr), all others will receive the light-touch version.

source
StaticLint.traverse — Method
traverse(x, state)

Iterates across the child nodes of an EXPR in execution order (rather than storage order) calling state on each node.

source
StaticLint.Binding — Type

Bindings indicate that an EXPR may introduce a new name into the current scope/namespace. Struct fields:

  • name: the EXPR that defines the unqualifed name of the binding.
  • val: what the binding points to, either a Binding (indicating ..), EXPR (this is generally the expression that defines the value) or SymStore.
  • type: the type of the binding, either a Binding, EXPR, or SymStore.
  • refs: a list containing all references that have been made to the binding.
source
StaticLint.add_binding — Function
add_binding(x, state, scope=state.scope)

Add the binding of x to the current scope. Special handling is required for:

  • macros: to prefix the @
  • functions: These are added to the top-level scope unless this syntax is used to define a closure within a function. If a function with the same name already exists in the scope then it is not replaced. This enables the refs list of the Binding of that 'root method' to hold a method table, the name of the new function will resolve to the binding of the root method (to get a list of actual methods -[get_method(ref) for ref in binding.refs if get_method(ref) !== nothing]). For example
[1] f() = 1
+[2] f(x) = 2

[1] is the root method and the name of [2] resolves to the binding of [1]. Functions declared with qualified names require special handling, there are comments in the source.

Some simple type inference is run.

source
StaticLint.mark_bindings! — Method
mark_bindings!(x::EXPR, state)

Checks whether the expression x should introduce new names and marks them as needed. Generally this marks expressions that would introdce names to the current scope (i.e. that x sits in) but in cases marks expressions that will add names to lower scopes. This is done when it is not knowable that a child node of x will introduce a new name without the context of where it sits in x -for example the arguments of the signature of a function definition.

source
StaticLint.lint_file — Function
lint_file(rootpath, server)

Read a file from disc, parse and run a semantic pass over it. The file should be the root of a project, e.g. for this package that file is src/StaticLint.jl. Other files in the project will be loaded automatically (calls to include with complicated arguments are not handled, see followinclude for details). A FileServer will be returned containing the Files of the package.

source
StaticLint.lint_string — Function
lint_string(s, server; gethints = false)

Parse a string and run a semantic pass over it. This will mark scopes, bindings, references, and lint hints. An annotated EXPR is returned or, if gethints = true, it is paired with a collected list of errors/hints.

source
StaticLint.interpret_eval — Method

interpret_eval(x::EXPR, state)

Naive attempt to interpret x as though it has been eval'ed. Lifts any bindings made within the scope of x to the toplevel and replaces (some) interpolated binding names with the value where possible.

source
StaticLint.resolve_getfield — Method
resolve_getfield(x::EXPR, parent::Union{EXPR,Scope,ModuleStore,Binding}, state::State)::Bool

Given an expression of the form parent.x try to resolve x. The method called with parent::EXPR resolves the reference for parent, other methods then check whether the Binding/Scope/ModuleStore to which parent points has a field matching x.

source
StaticLint.valofid — Method
valofid(x)

Returns the string value of an expression for which isidentifier is true, i.e. handles NONSTDIDENTIFIERs.

source
StaticLint.scopes — Method
scopes(x::EXPR, state)

Called when traversing the syntax tree and handles the association of scopes with expressions. On the first pass this will add scopes as necessary, on following passes it empties it.

source
StaticLint.getenv — Method
getenv(file::File, server::FileServer)

Get the relevant ExternalEnv for a given file.

source

Linting

StaticLint.check_kw_default — Method
check_kw_default(x::EXPR, server)

Check that the default value matches the type for keyword arguments. Following types are checked: String, Symbol, Int, Char, Bool, Float32, Float64, UInt8, UInt16, UInt32, UInt64, UInt128.

source
StaticLint.collect_hints — Function

collect_hints(x::EXPR, env, missingrefs = :all, isquoted = false, errs = Tuple{Int,EXPR}[], pos = 0)

Collect hints and errors from an expression. missingrefs = (:none, :id, :all) determines whether unresolved identifiers are marked, the :all option will mark identifiers used in getfield calls."

source