Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infer# integration #1361

Closed
wants to merge 35 commits into from
Closed
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
661ef65
integrate with infersharp
xi-liu-ds Nov 3, 2020
cd1e6fc
more changes to compatible with infersharp
xi-liu-ds Nov 5, 2020
3fa4160
remove useless adds
xi-liu-ds Nov 5, 2020
a5c023b
resolve incompatibilities
xi-liu-ds Nov 7, 2020
d9b00ca
Merge remote-tracking branch 'upstream/master' into integration
xi-liu-ds Nov 7, 2020
0e4489a
null dereference support for C#
xi-liu-ds Nov 11, 2020
9374648
Merge branch 'master' of github.com:xi-liu-ds/infer into integration
xi-liu-ds Nov 12, 2020
b5a8d6d
generate specs files
xi-liu-ds Nov 14, 2020
545c78b
Merge branch 'master' of https://github.com/xi-liu-ds/infer into inte…
xi-liu-ds Nov 17, 2020
cc87294
add resource leak support
xi-liu-ds Dec 1, 2020
4f4acf6
Merge pull request #1 from xi-liu-ds/integration
xi-liu-ds Dec 1, 2020
4e86aa8
Merge branch 'master' of https://github.com/xi-liu-ds/infer
xi-liu-ds Dec 1, 2020
280a5fb
remove fallbacknode
xi-liu-ds Dec 1, 2020
ebf8e88
add get proc desc function for .net
xi-liu-ds Dec 2, 2020
f0b6e33
fix float on nan
xi-liu-ds Dec 2, 2020
0ec94eb
Merge remote-tracking branch 'upstream/master'
xi-liu-ds Dec 14, 2020
71914ae
seperate infer analyze json logics
xi-liu-ds Dec 14, 2020
a0e0c28
fix deprecated warnings
xi-liu-ds Dec 15, 2020
01b1037
Merge branch 'master' into master
xi-liu-ds Dec 16, 2020
a67918c
resolve comments
xi-liu-ds Dec 24, 2020
7047a71
Merge branch 'master' of https://github.com/xi-liu-ds/infer
xi-liu-ds Dec 24, 2020
8848fdb
Merge remote-tracking branch 'upstream/master'
xi-liu-ds Dec 24, 2020
6986656
fix build failure
xi-liu-ds Dec 24, 2020
f3b95de
add unit tests
xi-liu-ds Dec 29, 2020
db3ada8
add unit tests make file
xi-liu-ds Dec 29, 2020
4a001b4
Merge pull request #2 from xi-liu-ds/xiaoyu/integration/unittests
xi-liu-ds Dec 29, 2020
b2f300a
Merge remote-tracking branch 'upstream/master'
xi-liu-ds Jan 6, 2021
77270e2
format cleanup
xi-liu-ds Jan 6, 2021
4270215
Merge branch 'master' of https://github.com/xi-liu-ds/infer
xi-liu-ds Jan 6, 2021
de588c5
remove BUILD_DOTNET_ANALYZERS condition
xi-liu-ds Jan 6, 2021
6ca305a
focus on resource leaks and null deref
xi-liu-ds Jan 19, 2021
0145560
remove unit test jsons files
xi-liu-ds Jan 22, 2021
d0945ba
xz unit tests
xi-liu-ds Jan 22, 2021
56d910e
Merge pull request #3 from xi-liu-ds/xi-liu-ds/xzjsons
xi-liu-ds Jan 28, 2021
c35f6c1
Merge branch 'master' into master
xi-liu-ds Feb 2, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:

- name: Install Required Apt Packages for Ubuntu
run: |
sudo apt install libmpfr-dev libsqlite3-dev ninja-build
sudo apt install libmpfr-dev sqlite3 libsqlite3-dev ninja-build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is sqlite3 itself used anywhere yet?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good overall I think. I haven't read much of AnalyzeJson yet.

Are you able to run the auto-formatter with make fmt? You'll need this version of ocamlformat: https://github.com/ocaml-ppx/ocamlformat/tree/nebuchadnezzar
I think this command does the trick: opam pin add ocamlformat.0.14.1-13-g000da99 -k git https://github.com/ocaml-ppx/ocamlformat.git#nebuchadnezzar

I got build error in this ocamlformat branch. I will try again after they fixed the pipeline.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is sqlite3 itself used anywhere yet?

Removed. Good catch!

sudo apt clean
if: runner.os == 'Linux'

Expand Down
125 changes: 125 additions & 0 deletions infer/src/IR/CSharpClassName.ml
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
(*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*)

open! IStd
module F = Format
module L = Logging

(** invariant: if [namespace = Some str] then [not (String.equal str "")]. [classname] appears first
so that the comparator fails earlier *)
type t = {classname: string; namespace: string option} [@@deriving compare, equal, yojson_of]

module Map = Caml.Map.Make (struct
type nonrec t = t [@@deriving compare]
end)

module Set = Caml.Set.Make (struct
type nonrec t = t [@@deriving compare]
end)

let make ~namespace ~classname =
match namespace with Some "" -> {namespace= None; classname} | _ -> {namespace; classname}


let from_string str =
match String.rsplit2 str ~on:'.' with
| None ->
{classname= str; namespace= None}
| Some ("", _) ->
L.die InternalError "Empty namespace path in CSharp qualified classname.@."
| Some (pkg, classname) ->
{classname; namespace= Some pkg}


let to_string = function
| {classname; namespace= None} ->
classname
| {classname; namespace= Some pkg} ->
String.concat ~sep:"." [pkg; classname]


let pp fmt = function
| {classname; namespace= None} ->
F.pp_print_string fmt classname
| {classname; namespace= Some pkg} ->
F.fprintf fmt "%s.%s" pkg classname


let namespace {namespace} = namespace

let classname {classname} = classname

let is_int s =
try
ignore (int_of_string s) ;
true
with Failure _ -> false


let get_outer_class_name {namespace; classname} =
String.rsplit2 classname ~on:'$' |> Option.map ~f:(fun (outer, _) -> {namespace; classname= outer})


(*
Anonymous classes have two forms:
- classic anonymous classes: suffixes in form of $<int>.
- classes corresponding to lambda-expressions: they are manifested as $Lambda$.
- two forms above nested inside each other.
Also non-anonymous (user-defined) name can be nested as well (Class$NestedClass).
In general case anonymous class name looks something like
Class$NestedClass$1$17$5$Lambda$_1_2, and we need to return Class$NestedClass *)
let get_user_defined_class_if_anonymous_inner {namespace; classname} =
let is_anonymous_name = function
| "Lambda" ->
true
| name when is_int name ->
true
| _ ->
false
in
let pieces = String.split classname ~on:'$' in
let first_anonymous_name = List.findi pieces ~f:(fun _index name -> is_anonymous_name name) in
Option.bind first_anonymous_name ~f:(fun (index, _name) ->
(* Everything before this index did not have anonymous prefixes. Deem it user defined. *)
match List.take pieces index with
| [] ->
(* This is a weird situation - our class _starts_ with an anonymous name.
This should not happen normally, but we can not rule this out completely since there is no physical limitations on
the bytecode names.
In this case, we return [None] because formally this is not an anonymous _inner_ class, but anonymous outermost class instead.
TODO: redesign this API so this case is modelled directly
*)
None
| list ->
(* Assemble back all pieces together *)
Some {namespace; classname= String.concat ~sep:"$" list} )


let is_anonymous_inner_class_name t = get_user_defined_class_if_anonymous_inner t |> is_some

let is_external_via_config t =
let namespace = namespace t in
Option.exists ~f:Config.csharp_namespace_is_external namespace


let pp_with_verbosity ~verbose fmt t =
if verbose then pp fmt t else F.pp_print_string fmt (classname t)


module Normalizer = HashNormalizer.Make (struct
type nonrec t = t [@@deriving equal]

let hash = Hashtbl.hash

let normalize t =
let classname = HashNormalizer.StringNormalizer.normalize t.classname in
let namespace =
IOption.map_changed t.namespace ~equal:phys_equal ~f:HashNormalizer.StringNormalizer.normalize
in
if phys_equal classname t.classname && phys_equal namespace t.namespace then t
else {classname; namespace}
end)
52 changes: 52 additions & 0 deletions infer/src/IR/CSharpClassName.mli
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
(*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*)

open! IStd

type t [@@deriving compare, equal, yojson_of]

module Map : Caml.Map.S with type key = t

module Set : Caml.Set.S with type elt = t

val make : namespace:string option -> classname:string -> t

val from_string : string -> t

val to_string : t -> string
(** [to_string (from_string "X.Y.Z") = "X.Y.Z"] *)

val pp : Format.formatter -> t -> unit
(** [pp] includes namespace if any *)

val pp_with_verbosity : verbose:bool -> Format.formatter -> t -> unit
(** if [verbose] then print namespace if present, otherwise only print class *)

val namespace : t -> string option

val classname : t -> string

val is_external_via_config : t -> bool
(** Considered external based on config flags. *)

val get_outer_class_name : t -> t option
(** If this is an inner class, return the closest outer, e.g. A$B for A$B$C. None if the class is
outermost *)

val is_anonymous_inner_class_name : t -> bool
(** True if it is either "classic" anonymous csharp class: *)

val get_user_defined_class_if_anonymous_inner : t -> t option
(** If the current class is anonymous ([is_anonymous_inner_class_name] is true), return the
corresponding user defined (not anonymous) class this anonymous class belongs to.

In general case, BOTH anonymous classes and user-defined classes can be nested:
SomeClass$NestedClass$1$17$5. In this example, we should return SomeClass$NestedClass.

If this is not an anonymous class, returns [None]. *)

module Normalizer : HashNormalizer.S with type t = t
5 changes: 4 additions & 1 deletion infer/src/IR/CallFlags.ml
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,19 @@ type t =
; cf_injected_destructor: bool
; cf_interface: bool
; cf_is_objc_block: bool
; cf_noreturn: bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was deleted by 9979489 because it is dead code. It looks like the flag has been reintroduced by mistake in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was deleted by 9979489 because it is dead code. It looks like the flag has been reintroduced by mistake in this PR.

Removed. Thanks

; cf_virtual: bool }
[@@deriving compare]

let pp f
({cf_assign_last_arg; cf_injected_destructor; cf_interface; cf_is_objc_block; cf_virtual}[@warning
({cf_assign_last_arg; cf_injected_destructor; cf_interface; cf_is_objc_block; cf_noreturn; cf_virtual}[@warning
"+9"])
=
if cf_assign_last_arg then F.pp_print_string f " assign_last" ;
if cf_injected_destructor then F.pp_print_string f " injected" ;
if cf_interface then F.pp_print_string f " interface" ;
if cf_is_objc_block then F.pp_print_string f " objc_block" ;
if cf_noreturn then F.pp_print_string f " noreturn" ;
if cf_virtual then F.pp_print_string f " virtual" ;
()

Expand All @@ -37,4 +39,5 @@ let default =
; cf_injected_destructor= false
; cf_interface= false
; cf_is_objc_block= false
; cf_noreturn= false
; cf_virtual= false }
1 change: 1 addition & 0 deletions infer/src/IR/CallFlags.mli
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ type t =
(** true if this is an implicit C++ destructor call injected by the clang frontend *)
; cf_interface: bool
; cf_is_objc_block: bool
; cf_noreturn: bool
; cf_virtual: bool }
[@@deriving compare]

Expand Down
2 changes: 2 additions & 0 deletions infer/src/IR/CapturedVar.ml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ module F = Format

type t = {name: Mangled.t; typ: Typ.t; capture_mode: Pvar.capture_mode} [@@deriving compare]

let make ~name ~typ ~capture_mode = {name; typ; capture_mode}

let pp fmt {name; typ; capture_mode} =
F.fprintf fmt "(%a,@,%a,@,%s)" Mangled.pp name (Typ.pp_full Pp.text) typ
(Pvar.string_of_capture_mode capture_mode)
2 changes: 2 additions & 0 deletions infer/src/IR/CapturedVar.mli
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ open! IStd
type t = {name: Mangled.t; typ: Typ.t; capture_mode: Pvar.capture_mode} [@@deriving compare]

val pp : Format.formatter -> t -> unit

val make : name:Mangled.t -> typ:Typ.t -> capture_mode: Pvar.capture_mode -> t
9 changes: 8 additions & 1 deletion infer/src/IR/Cfg.ml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ let iter_over_sorted_procs cfg ~f =
|> List.sort ~compare:compare_proc_desc_by_proc_name
|> List.iter ~f


let get_all_defined_proc_names cfg =
let procs = ref [] in
let f pname pdesc = if Procdesc.is_defined pdesc then procs := pname :: !procs in
Expand All @@ -40,6 +39,14 @@ let create_proc_desc cfg (proc_attributes : ProcAttributes.t) =
Procname.Hash.add cfg pname pdesc ;
pdesc

(** Get a procdesc or create if it is a new one*)
let get_proc_desc cfg (proc_attributes : ProcAttributes.t) =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks a bit dangerous to add to the Cfg module and I'm not sure I understand why it's needed: what happens if the proc name is already present? The proc desc will get created but not recorded in the CFG, so in the end it's discarded?

This is only used once, could you maybe rewrite the call site to if not (Cfg.mem proc_attributes.proc_name) then Cfg.create_proc_desc ...?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks a bit dangerous to add to the Cfg module and I'm not sure I understand why it's needed: what happens if the proc name is already present? The proc desc will get created but not recorded in the CFG, so in the end it's discarded?

This is only used once, could you maybe rewrite the call site to if not (Cfg.mem proc_attributes.proc_name) then Cfg.create_proc_desc ...?

Good suggestion. I have rewritten this call.

let pdesc = Procdesc.from_proc_attributes proc_attributes in
let pname = proc_attributes.proc_name in
if not (Procname.Hash.mem cfg pname) then
Procname.Hash.add cfg pname pdesc ;
pdesc


let iter_sorted cfg ~f = iter_over_sorted_procs cfg ~f

Expand Down
3 changes: 3 additions & 0 deletions infer/src/IR/Cfg.mli
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ val create : unit -> t
val create_proc_desc : t -> ProcAttributes.t -> Procdesc.t
(** Create a new procdesc and add it to the cfg *)

val get_proc_desc : t -> ProcAttributes.t -> Procdesc.t
(** Get a procdesc and add it to the cfg if it is new*)

val iter_sorted : t -> f:(Procdesc.t -> unit) -> unit
(** Iterate over all the proc descs in the cfg in ascending order *)

Expand Down
1 change: 0 additions & 1 deletion infer/src/IR/Fieldname.ml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ let to_simplified_string fld =
String.concat ~sep:"." [class_only; fld.field_name] )
else fld.field_name


let to_full_string fld =
(if is_java fld then dot_join else cc_join) (Typ.Name.name fld.class_name) fld.field_name

Expand Down
2 changes: 2 additions & 0 deletions infer/src/IR/Ident.ml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ let knormal = KNormal

let kprimed = KPrimed

let knone = KNone

let equal_kind = [%compare.equal: kind]

(* timestamp for a path identifier *)
Expand Down
4 changes: 4 additions & 0 deletions infer/src/IR/Ident.mli
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ val knormal : kind

val kfootprint : kind

val knone : kind

val name_spec : name
(** Name used for spec variables *)

Expand All @@ -79,6 +81,8 @@ val name_to_string : name -> string
val get_name : t -> name
(** Name of the identifier. *)

val create_with_stamp : kind -> name -> int -> t

val create : kind -> int -> t
(** Create an identifier with default name for the given kind *)

Expand Down
21 changes: 21 additions & 0 deletions infer/src/IR/Procdesc.ml
Original file line number Diff line number Diff line change
Expand Up @@ -452,6 +452,18 @@ module Node = struct
let succs = get_succs node in
let preds = get_preds node in
NodeKey.compute node ~simple_key ~succs ~preds

let print_node node =
let print_node_list nl =
List.iter ~f:(fun n -> Printf.printf "%d " (get_id n)) nl
in
Printf.printf "id: %d preds: " (get_id node) ; print_node_list node.preds ;
Printf.printf "succs: " ; print_node_list node.succs;
Printf.printf "exn: "; print_node_list node.exn;
Location.pp_line Format.std_formatter node.loc ;
print_endline "" ;
print_endline (get_description Pp.text node) ;
()
end

(* =============== END of module Node =============== *)
Expand Down Expand Up @@ -641,6 +653,11 @@ let set_start_node pdesc node = pdesc.start_node <- node
let append_locals pdesc new_locals = pdesc.attributes.locals <- pdesc.attributes.locals @ new_locals

(** Set the successor nodes and exception nodes, and build predecessor links *)
let set_succs_exn_base (node : Node.t) succs exn =
node.succs <- succs ;
node.exn <- exn ;
List.iter ~f:(fun (n : Node.t) -> n.preds <- node :: n.preds) succs

let set_succs (node : Node.t) ~normal:succs_opt ~exn:exn_opt =
let remove_pred pred_node (from_node : Node.t) =
from_node.preds <- List.filter from_node.preds ~f:(fun pred -> not (Node.equal pred pred_node))
Expand Down Expand Up @@ -882,3 +899,7 @@ let load =
|> SqliteUtils.check_result_code db ~log:"load bind proc_uid" ;
SqliteUtils.result_single_column_option ~finalize:false ~log:"Procdesc.load" db stmt
|> Option.bind ~f:SQLite.deserialize )

let print_pdesc_nodes pdesc =
List.iter ~f:Node.print_node pdesc.nodes ;
print_endline ""
6 changes: 6 additions & 0 deletions infer/src/IR/Procdesc.mli
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,8 @@ module Node : sig
val pp_stmt : Format.formatter -> stmt_nodekind -> unit

val compute_key : t -> NodeKey.t

val print_node : t -> unit
end

(** Map with node id keys. *)
Expand Down Expand Up @@ -315,6 +317,8 @@ val fold_nodes : t -> init:'accum -> f:('accum -> Node.t -> 'accum) -> 'accum
val fold_slope_range : Node.t -> Node.t -> init:'accum -> f:('accum -> Node.t -> 'accum) -> 'accum
(** fold between two nodes or until we reach a branching structure *)

val set_succs_exn_base : Node.t -> Node.t list -> Node.t list -> unit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use set_succs instead?

If this is needed could you add a doc string explaining that it should be used carefully because it won't update the node's successors' pred links? (not sure why that is ok except when the current node is known to have no succs, but then this is the same as set_succs just below) Named arguments like set_succs would be nice too since the types don't help to disambiguate the arguments here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use set_succs instead?

If this is needed could you add a doc string explaining that it should be used carefully because it won't update the node's successors' pred links? (not sure why that is ok except when the current node is known to have no succs, but then this is the same as set_succs just below) Named arguments like set_succs would be nice too since the types don't help to disambiguate the arguments here.

It was an outdated function. I have replaced it with set_succs


val set_succs : Node.t -> normal:Node.t list option -> exn:Node.t list option -> unit
(** Set the successor nodes and exception nodes, if given, and update predecessor links *)

Expand Down Expand Up @@ -351,3 +355,5 @@ val shallow_copy_code_from_pdesc : orig_pdesc:t -> dest_pdesc:t -> unit
module SQLite : SqliteUtils.Data with type t = t option

val load : Procname.t -> t option

val print_pdesc_nodes : t -> unit
Loading