Harry 2 #3689

ifesdjeen · 2024-11-14T19:09:32Z

Make it easier to reuse generators, make Harry more extensible and accord-proof, refactor Harry's major subsystems:

SELECT is now a regular Harry operation, not a separate "validate" step
Flushes (or any custom operation) are now a part of a model, Harry will run them for you
Simplified validation of subset/wildcard queries
All Operations and are now SchemaSpec-agnostic, and can use any generators (like QT or whatever)
You read it right, Harry now can use any generator and still be able to not to keep generated values in memory
There are no concepts of “descriptor selectors”, or “bijection” exposed anymore. You just provide a Generator, Harry figures out the rest.
All Cassandra data types are now supported
No upper limit on the number of columns (also, clustering columns)
Basic (for now) support for Transactions
It is now possible to visit multiple partition in the same Visit (essentially, for transactions)
Much much simpler to generate synthetic workloads now (basically, you just write a Generator
Add support for all single-cell types.

Most of it was just a refactoring of Harry, to allow the following (still pending):

Easier to add subset SELECTION (important for SAI testing)
Easier add support for functions
Easier to add support for collections and UDTs
Easier to add support for LET bindings

test/distributed/org/apache/cassandra/fuzz/sai/MultiNodeSAITest.java

test/harry/main/org/apache/cassandra/harry/gen/InvertibleGenerator.java

test/distributed/org/apache/cassandra/fuzz/sai/SingleNodeSAITestBase.java

test/distributed/org/apache/cassandra/fuzz/sai/StaticsTortureTest.java

test/distributed/org/apache/cassandra/distributed/test/accord/AccordHostReplacementTest.java

test/distributed/org/apache/cassandra/distributed/test/log/FuzzTestBase.java

test/distributed/org/apache/cassandra/distributed/test/log/MetadataChangeSimulationTest.java

test/distributed/org/apache/cassandra/distributed/test/log/ModelState.java

test/distributed/org/apache/cassandra/distributed/test/log/OperationalEquivalenceTest.java

test/harry/main/org/apache/cassandra/harry/ColumnSpec.java

test/harry/main/org/apache/cassandra/harry/SchemaSpec.java

test/harry/main/org/apache/cassandra/harry/checker/TestHelper.java

src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java

test/distributed/org/apache/cassandra/distributed/test/accord/AccordHostReplacementTest.java

test/distributed/org/apache/cassandra/distributed/upgrade/ClusterMetadataUpgradeHarryTest.java

test/distributed/org/apache/cassandra/fuzz/ring/LossyNetworkTest.java

test/harry/main/org/apache/cassandra/harry/gen/SchemaGenerators.java

test/harry/main/org/apache/cassandra/harry/gen/ValueGenerators.java

test/harry/main/org/apache/cassandra/harry/gen/rng/PureRng.java

test/harry/main/org/apache/cassandra/harry/model/QuiescentChecker.java

test/harry/main/org/apache/cassandra/harry/execution/QueryBuildingVisitExecutor.java

test/distributed/org/apache/cassandra/fuzz/topology/HarryTopologyMixupTest.java

test/harry/main/org/apache/cassandra/harry/SchemaSpec.java

test/harry/main/org/apache/cassandra/harry/execution/CQLVisitExecutor.java

test/harry/main/org/apache/cassandra/harry/execution/QueryBuildingVisitExecutor.java

test/harry/main/org/apache/cassandra/harry/gen/ValueGenerators.java

dcapwell · 2024-11-20T19:18:46Z

test/unit/org/apache/cassandra/cql3/ast/Select.java

@@ -60,25 +60,26 @@ public class Select implements Statement
    public final Optional<OrderBy> orderBy;
    public final Optional<Value> limit;
    public final boolean allowFiltering;
-
+    public final boolean insertNewLine;


please remove this from the state, we can add toCQL versions that allow overriding this

patch to fix here: dcapwell@1c8c349

Given these are not harry apis, and APIs I am maintaining, this is a blocking comment.

I want to soften what I mean by "blocking comment" here... Alex and I spoke in slack and I stated that most of my comments are suggestions and are non-blocking comments, if we disagree his view wins. I said I would be explicit about what is blocking (as the default isn't) but then this comes off very harsh...

My concerns here is putting the string formatting state into the object rather than providing it to the toCQL directly... I am open to other proposals

Removed insertNewLine, replaced newlines with space in Harry compile code.

I do not want you thinking you can not make improvements to the ast package... I was trying to express formatting decisions should be given to the logic that does the formatting rather than part of the object

test/harry/main/org/apache/cassandra/harry/gen/SchemaGenerators.java

test/harry/main/org/apache/cassandra/harry/SchemaSpec.java

…cord-proof, refactor Harry's major subsystems * SELECT is now a regular Harry operation, not a separate "validate" step * Flushes (or any custom operation) are now a part of a model, Harry will run them for you * Simplified validation of subset/wildcard queries * All Operations and are now SchemaSpec-agnostic, and can use any generators (like QT or whatever) * You read it right, Harry now can use any generator and still be able to not to keep generated values in memory * There are no concepts of “descriptor selectors”, or “bijection” exposed anymore. You just provide a Generator, Harry figures out the rest. * All Cassandra data types are now supported * No upper limit on the number of columns (also, clustering columns) * Basic (for now) support for Transactions * It is now possible to visit multiple partition in the same Visit (essentially, for transactions) * Much much simpler to generate synthetic workloads now (basically, you just write a Generator * Add support for all single-cell types. Most of it was just a refactoring of Harry, to allow the following (still pending): * Easier to add subset SELECTION (important for SAI testing) * Easier add support for functions * Easier to add support for collections and UDTs * Easier to add support for LET bindings

ifesdjeen requested review from maedhroz and dcapwell November 14, 2024 19:09