[WIP] BitCircuitPair: bit circuits for pair of ciruict (eg for expectations)

[khosravipasha]

Got some code working for bit circuit for pair of circuits. Took some debugging but now the numbers for expectation match with other implementations.

Todo:

• Helper functions
• CUDA version
• Add units tests for cpu version
• Tests for CUDA version
• See if can optimize and parallelize more on both the CPU/GPU versions.
• Memory optimization? (comment 2)
• Good Benchmark to see how much faster (from small to big cases)
  having pr-constraint by itself is useful to have.
• Easy and Efficient Support for batching (by reusing memory etc)
  • See commnet 5, seems much faster

Comments:

1. Did not add Pair leaves for constant nodes (TRUE,FALSE) probably won't need them. But added for literals pairs that don't agree like (-3 and 3) but probably don't need those either as its always not satisfied.

2. Right now had to also store bit circuit for both the PC and LC as well, maybe can optimize that part. This was to get the paramters easier. Without having to repeat them.

3.In small cases its not faster really. In the biggest case I tried its about 1.5-2x times faster than the other two implementations I tried. Probably still room for optimization, did not use many of the tricks yet like @views.

4. Runtime + Batching: (RTX)

training set has 36 variables and 935 samples.
Probablistic Circuit with 27493 nodes and 16272 parameters.
Regression Circuit with 1076 nodes and 659 parameters.
pc nodes 27493, lc nodes 1076, data (935, 36)  
  6.778533 seconds (6.28 M allocations: 2.853 GiB, 4.33% gc time)
  4.903041 seconds (5.64 M allocations: 10.759 GiB, 19.88% gc time)
  3.762878 seconds (3.06 M allocations: 9.933 GiB, 12.06% gc time)
no_agree(EPS=5.0e-5) 0
  0.666297 seconds (2.69 M CPU allocations: 141.079 MiB, 8.76% gc time) (297 GPU allocations: 766.406 MiB, 0.63% gc time of which 93.96% spent allocating)
no_agree(EPS=5.0e-5) 0
  0.044030 seconds (112.21 k CPU allocations: 6.541 MiB) (259 GPU allocations: 45.304 MiB, 0.37% gc time)
PC   bit nodes  (4, 12465)), edges (3, 16272)
Pair bit nodes  (6, 99686)), edges (5, 248450)
no_agree(EPS=5.0e-5) 0

a. Recursive Exp
b. Bit circuit cpu
c. Bit circuit cpu & reuse bit circuit + memories
d. Bit circuit gpu
e. Bit circuit gpu & reuse bitcuit + memories