fix: Arrow library usage and memory usage cleanup

[zeroshade]

Description

Updates to Arrow v11 and leverages the compute functions + significantly cleaning up the usage of the Arrow library to be cleaner and fixing the memory leaks that existed in the incorrect usage of the Arrow library. This includes properly calling Release on record batches and arrays and adding CheckedAllocators to some tests to ensure there are no leaks.

This also unifies the Arrow memory allocator used so that consumers can provide their own Arrow memory.Allocator if they want.

Checklist

• Code compiles correctly
• Run make fmt to fix inconsistent formats
• Run make lint to get lint errors and fix all of them
• Created tests which fail without the change (if possible)
• All tests passing
• Extended the README / documentation, if necessary

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[zeroshade]

I have read the CLA Document and I hereby sign the CLA

[zeroshade]

While i needed to upgrade to Arrow v11 to do the cleanup and fixes, #764 should get merged first as he went further than I did and also updated other dependencies and removed the replace directives from the go.mod. After that is merged, my change can get merged (or just updated to not do any changes to the vendor dir and only be the changes to the gosnowflake driver itself).

I'm going to second @kenshaw's request to just eliminate the vendoring altogether and just follow the standard Go module handling to make upgrades like this easier in the future.

[yevgenypats]

@zeroshade nice! I was just looking at what the status of arrow in this driver and saw this PR.

Anything we can help here or we can use this branch?

[zeroshade]

@yevgenypats if you have any contacts at snowflake, try to push them to review and pay attention to this PR and the linked #764 PR that's the best help I can get right now.

[sfc-gh-igarish]

@zeroshade Thanks for the PR. This PR has many files changes mainly for Arrow library, we will take a look before merging it.

[yevgenypats]

@yevgenypats if you have any contacts at snowflake, try to push them to review and pay attention to this PR and the linked #764 PR that's the best help I can get right now.

Unfortunately don't have any contact.

@sfc-gh-igarish - this just touches the vendor directory as far as I can see so there are not a lot of changes.

[yevgenypats]

@yevgenypats if you have any contacts at snowflake, try to push them to review and pay attention to this PR and the linked #764 PR that's the best help I can get right now.

@zeroshade Just an FYI on the status of this driver, we are using this driver in production for a while now for our snowflake destination but I had quite a bit of issues due to a lot of limitations on what it can or cannot do. we ended up just using load to snowflake via CSV and using this driver just to run the load from csv/json command (I think this is also the way that most users load things to snowflake from a remote storage with files in csv/parquet/json format and why some of the drivers seems to be in "alpha").

[sfc-gh-igarish]

I approved #764 first. As I read #764 PR should merge first. Right?

[sfc-gh-igarish]

LGTM. As mentioned in the PR, merge it after #764 merge.

[sfc-gh-igarish]

I approved this also.

[sfc-gh-igarish]

Please resolve the conflicts then I can help to merge this PR. I already merged 764.

[zeroshade]

Thanks much @sfc-gh-igarish! I'll resolve the conflicts first thing tomorrow for myself and tag you in a comment when I do.

[zeroshade]

@sfc-gh-igarish rebased and conflicts fixed! thanks!

[sfc-gh-igarish]

Created a linked PR and start running test.