Allow DROP NOT NULL on column of compressed table

[mkindahl]

Remove the restriction on ALTER TABLE ... ALTER COLUMN SET/DROP NOT NULL on compressed tables.

Dropping or setting the NOT NULL constraint on a column of a compressed table is safe since it does not require re-writing any data. Setting NOT NULL will require a full table scan to verify that there are no NULL in the table.

Fixes #5776

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.11%. Comparing base (59f50f2) to head (601c367).
Report is 601 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7455      +/-   ##
==========================================
+ Coverage   80.06%   82.11%   +2.04%     
==========================================
  Files         190      230      +40     
  Lines       37181    43095    +5914     
  Branches     9450    10830    +1380     
==========================================
+ Hits        29770    35386    +5616     
- Misses       2997     3391     +394     
+ Partials     4414     4318      -96     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[mkindahl]

It is actually fine to use SET NOT NULL as well. It will generate a full table scan to investigate if there are any NULL present, but otherwise it should not require any dramatic changes.

[antekresic]

Setting NOT NULL on a column which has compressed NULL values doesn't work properly. Maybe we should just limit this change to dropping NOT NULL since setting requires decompression which isn't done in the current change.

Also, lets add a test for this, trying to set NOT NULL on a column which has NULL values and the hypertable is full compressed (not partial).

[mkindahl]

It is actually fine to use SET NOT NULL as well. It will generate a full table scan to investigate if there are any NULL present, but otherwise it should not require any dramatic changes.

This turned out to not be the case. If executed on a compressed chunk, it works for hypercore-compressed tables but not for heap-compressed tables. It seems like hypercore-compressed tables check for the presence of NULL using table_beginscan, which works for hypercore-compressed tables but not for heap-compressed tables.

[mkindahl]

Setting NOT NULL on a column which has compressed NULL values doesn't work properly. Maybe we should just limit this change to dropping NOT NULL since setting requires decompression which isn't done in the current change.

Yeah, I discovered that it does not work for compressed tables using the heap access method, but this works fine for chunks with the hypercore access method (since it looks like a normal hypertable). Added checks so that we only allow this for compressed chunks that are using hypercore table access method.

Also, lets add a test for this, trying to set NOT NULL on a column which has NULL values and the hypertable is full compressed (not partial).

There is an existing test in compression_errors, but that does not compress the table, so the question is if we should check the flag on the hypertable (and ignore the status of the chunks), or if we should allow this with the compression flag set and chunk fully decompressed.

[feikesteenbergen]

Could we get the simplest thing merged the DROP NOT NULL? the rest is also useful, but easier to workaround for users

[mkindahl]

Could we get the simplest thing merged the DROP NOT NULL? the rest is also useful, but easier to workaround for users

Ok, split out the code to deal with SET NOT NULL into #7463 and left the other pull request here. Still awaiting reviews.