[Shopify] Add Metafields to Products and Variants

[tinestaric]

This is a PR without a dedicated issue as it's part of delivery for development agreed directly with @AndreiPanko

Here's the overview of the changes:

Obsolete fields:
• Obsoleted Value Type as it‘s no longer used in Shopify API -> Replaced with Type field
• Obsoleted Owner Resource field. I‘m not sure why this was a free text that was then always populated through a case statement. It‘s replaced with an Owner Type field that is an Enum.

New enums and interfaces:
• „Shpfy Metafield Type“ and „Shpfy IMetafield Type“ – Used for handling the metafield type. The interface defines if the type has a special AssistEdit, logic to execute the assist edit, and validations when free input is allowed for a value. I have a couple of questions here, see below. Rating, Rich Text, and List Types are not included, Rating and Rich Text options are added but commented out so it‘s clear that it was intentional. It also has String and Integer types which are there to support backward compatibility, if old values are still in Shopify.
• „Shpfy Metafield Owner Type“ and „Shpfy IMetafield Owner Type“ – Used for building the full owner resource ID (gid://shopify//) and for retrieving metafield IDs of a single resource (used during export, to identify which metafields should be sent for update based on UpdatedAt)
• Metafield Dimenion/Volume/Weight Type – three enums to support a selection of Units when assist editing these metafield types

Pages:
• „Shpfy Metafields“ – used for viewing and editing metafields for a resource. It‘s only editable if Shop is syncing Products to Shopify and Can update Shopify products. On Insert of a new record, it‘s immediately sent to Shopify and is only inserted in the DB when we get back the Shopify ID for the metafield. Types that have AssistEdit functionality defined cannot be edited directly on the line. OnValidate of Value a check is executed to ensure value is correct for a type.
• „Shpfy Metafield Assist Edit“ – used for assist edit of complex metafield types. It has several groups defined, and only the appropriate one is displayed to the user.
• Action for display metafields for a resource has been added to the Products and Variants pages.

GraphQL query changes:
• ProductById and VariantById have been modified so that they retrieve the first 50 metafields (previously the first 10). Previously there was a filter to only retrieve Metafields with the „Microsoft.Dynamics365.BusinessCentral“ namespace. I‘ve removed that to get all the metafields of a resource.
• ProductMetafieldIds and VariantMetafieldIds were added. These are used during the export of metafields to determine which BC metafields should be sent for an update
• MetafieldsSet – This is used to create/update metafields for a resource

Changes to API codeunits:
• Product API – on UpdateShopifyProductFields a metafields part of JSON is now passed to Metafields API where it creates or updates Metafields in BC based on information retrieved from Shopify
• Variant API – on UpdateShopifyVariantFields the metafields part of the JSON is passed to Metafields API, same as above.
• „Shpfy Product Export“ – on UpdateProductData execute Metafields API to update/create metafields for the product and all variants that belong to the product
• „Metafield API“ – Implements logic to parse the „metafields“JSON that is retrieved with products/variants during import and creates/updates metafield records in BC.
Adds logic to export metafields for a specific resource. Only sends an update for metafields that have a lower UpdatedAt timestamp than BC record‘s last updated at timestamp. It batches the Metafields to 25 at a time, as that‘s the max MetafieldsSet mutation can handle as per the documentation.

Added logic to delete metafields on delete of a product/variant

A couple of questions or concerns:
• New metafields that get created in Shopify do not have a Metafield Definition attached to them. Should we care about this right now? If yes, should we also store metafield definitions in BC, or just create new definitions if any are missing in Shopify?
• I‘ve obsoleted the two fields and one enum, should I be wrapping the obsoletions in some preprocessing symbols? Which one?
• I‘m doing some regex validations when checking if the value of metafield is correct. I‘m not sure if the one for mixed reference and file reference is correct. I‘d appreciate it if someone could take a look at that.
• If value validation fails, it throws a pretty „simple“ error message right now, should I make this more expressive?
• Currently, the „update metafields“ has no error handling in place if the update query fails. Technically I‘m trying to catch everything on data input, but there can always be unexpected situations. How should I handle this?
• The code for sending the metafield to Shopify immediately after it‘s inserted through UI is currently on the Page‘s OnInsertRecord trigger. I‘m not sure about it, should I move it to the table trigger? The hesitation is because there already is some logic on the table from the partial implementation of metafields on Customers.
• I‘ve placed interface implementations in a separate folder under Codeunits folder, that‘s not the usual practice on MS apps, so I wanted to double-check it.
• Money metafield type needs to have the same currency as the shop, which leads to this completely separate code path after validating the Value. I‘m not sure if there‘s a better approach to handle this.
• I saw that most procedures in the app have documentation triggers that are not really beneficial. I‘ve only added them to the public or internal procedures. I‘ve also skipped on most object documentation triggers.

Fixes #26819
Fixes AB#443908

[pri-kise]

@tinestaric this is not directly related to your changes, but I was surprised that you didn't changed any permission sets or added InherenPermissionSets to internal Codeunits until I saw the following permission set.

https://github.com/microsoft/ALAppExtensions/blob/main/Apps/W1/Shopify/app/src/PermissionSets/ShpfyObjects.PermissionSet.al

@JesperSchulz is there any good reason to keep this permission Set with the * permissions?

Do the new implementation codeunits needs to have the Access scope Public?

I don't have experience with Shopify but with APIs in general.
After taking a short look in the API and if I understand the documentation for the file reference type correctly then you can reference there to any file on shopify. Not only to images.
Therefore I'd say that the regex should allow other patterns, too. e.g. for gid://shopify/GenericFile/1072273801 (check the response of get files)
https://shopify.dev/docs/api/admin-graphql/2024-01/queries/files#examples-Get_a_list_of_files

[pri-kise]

Currently, the „update metafields“ has no error handling in place if the update query fails. Technically I‘m trying to catch everything on data input, but there can always be unexpected situations. How should I handle this?

@tinestaric if I understand the documentation correctly then it's possible to define Validations for Metafields in Shopify:
https://shopify.dev/docs/apps/custom-data/metafields/definitions/validation
Currently we don't know in BC which additional validations are set up in Shopify. This could lead to errors that you wouldn't catch on data input.

[tinestaric]

@pri-kise
Thank you! I've adjusted the RegEx for files.
I've also added error handling on the page where the metafields are sent directly to Shopify on insert. I haven't seen any place in the app that would actually handle the user errors, everything simply gets logged to the log entry table, and someone has to take it from there, so currently metafields have the same approach.

[tinestaric]

I see that I'm getting errors from AppSourceCop due to field length extension. I assumed this is not an issue since the table is marked as Internal, and as such we're in full control of who this field extension would break?

Is this not the case, and I should only temporary obsolete fields and extend the field under a preprocessing symbol?

[JesperSchulz]

I see that I'm getting errors from AppSourceCop due to field length extension. I assumed this is not an issue since the table is marked as Internal, and as such we're in full control of who this field extension would break?

Is this not the case, and I should only temporary obsolete fields and extend the field under a preprocessing symbol?

If you're getting a false positive (which it looks like you're doing), just use a pragma to disable the error.

[onbuyuka]

I think we should create a new folder Metafields instead of all objects in Base folder

@tinestaric please take a look

[tinestaric]

@onbuyuka Moved all metafield specific objects under a new Metafield folder.
I'm not sure what the sequence of processing PRs is going to be, but Translations and Metafields share the same new "Owner Type" (in translations it's called Resource Type) Enum/Interface, that should be merged into one.

[JesperSchulz]

Had to re-ID:
Codeunit 30168 -> 30350
Codeunit 30159 -> 30351
Codeunit 30314 -> 30352
Codeunit 30213 -> 30353
Codeunit 30313 -> 30354

I cannot update app.json, hence the build will probably fail.

[JesperSchulz]

ID clashes.

[JesperSchulz]

Fix further ID-clashes.

[JesperSchulz]

@tinestaric, could you extend the range in the app.json to 30360? I had to change some IDs to prevent clashes.

[tinestaric]

@JesperSchulz Done