-
-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added support for plugins within LogBatchPurger + added archiving in BigObject plugin #288
Added support for plugins within LogBatchPurger + added archiving in BigObject plugin #288
Conversation
…gerPlugin that contains 2 interfaces + helper methods, switched to using multiple fields on LoggerPlugin__mdt to indicate Apex classes & Flows to run for a plugin, removed SObject-specific fields on LoggerPlugin__mdt, added new fields Log__c.LogPurgeAction__c and LoggerSettings__c.DefaultLogPurgeAction__c
…new custom setting LoggerSettings__c.DefaultLogPurgeAction__c
…in overhaul changes via LoggerPlugin class, removed old class LoggerSObjectHanderPlugin
…ng the ability to create plugins for LogBatchPurger, using the interface LoggerPlugin.Batchable - The BigObject plugin (and other plugins) can then leverage this to run additional logic before Log__c, LogEntry__c & LogEntryTag__c records are hard-deleted - For the BigObject plugin, it will be able to archive data into LogEntryArchive__b before LogBatchPurger deletes the data within the custom objects - Also finished some test improvements for triggerable plugins within LoggerSObjectHandler
…aming convention change ('CustomSaveMethodBigObject' instead of 'AdditionalSaveMethodsBigObject'), and renamed plugin CMDT file from 'BigObjectArchiving' to 'LogEntryArchiving', updated labels on some deprecated fields
…related handler methods
… a new class, LogEntryArchivePlugin, and implemented interface LoggerPlugin.Batchable so that Log__c/LogEntry__c/LogEntryTag__c records can be archived via LogBatchPurger before they're deleted - The plugin class handles talking with Logger, and builder class handles converting LogEntryEvent__e or LogEntry__c records to LogEntryArchive__b records
… LogEntryArchive__b as part of #117
…gin_Tests, fixed tests in LogBatchPurger_Tests
…tMock` & updated approaches to use a Map instead of List for CMDT records
…0 days, fixed LogEntryArchiveBuilder using the wrong value for LoggedById__c, updated index on LogEntryArchive__b again (still a WIP), retrieved & formatted metadata for LogEntryArchive__b, updated Admin.profile
@jamessimone there are a few failing tests in this PR that I'm not too worried about for this PR - 1 is an issue for Flows that I'll fix in the |
Awesome. Incredible write-up — looking forward to reviewing! |
…r & LoggerPlugin - Added LoggerParameter.matchOnPrefix() to return CMDT records with a specified prefix in the DeveloperName field - Added inner Comparable class in LoggerPlugin to handle custom sorting, since there are some limitations with SOQL queries for CMDT - Classes like LogBatchPurger, LoggerSObjectHandler, and LoggerSettingsController no longer need to track their own mock CMDT records, the LoggerPlugin & LoggerParameter classes now fully handle mocks for their corresponding CMDT objects - LoggerParameter & LoggerPlugin now require mock CMDT records to have DeveloperName populated - they'll throw errors if the field is null - Also added System namespace to some calls for Test.isRunningTest() to handle crazy orgs that have a custom Test class deployed
…stantSettings to both scratch org definition files
nebula-logger/core/main/configuration/classes/LoggerParameter.cls
Outdated
Show resolved
Hide resolved
nebula-logger/core/main/plugin-framework/classes/LoggerPlugin.cls
Outdated
Show resolved
Hide resolved
nebula-logger/core/tests/log-management/classes/LogBatchPurger_Tests.cls
Show resolved
Hide resolved
nebula-logger/core/tests/log-management/classes/LoggerSObjectHandler_Tests.cls
Outdated
Show resolved
Hide resolved
… LoggerSObjectHandler - I'll reimplement new tests when I also implement Flow plugins for LogBatchPurger
9ed34d6
to
b3f1907
Compare
232a85a
to
90fa34b
Compare
…lse to reduce issues in orgs upgrading, re-addedthe deprecated class LoggerSObjectHandlerPlugin to the managed package's deprecated folder
90fa34b
to
54b1c0f
Compare
b109fcf
to
8345106
Compare
8345106
to
5bc2e2e
Compare
Codecov Report
@@ Coverage Diff @@
## feature/plugin-framework-overhaul #288 +/- ##
====================================================================
Coverage ? 93.67%
====================================================================
Files ? 40
Lines ? 4176
Branches ? 86
====================================================================
Hits ? 3912
Misses ? 259
Partials ? 5
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report at Codecov.
|
…on Rules plugin (#298) - Replaced `LoggerSObjectHandlerPlugin` abstract class with new class `LoggerPlugin` that contains 2 interfaces + helper methods, switched to using multiple fields on `LoggerPlugin__mdt` to indicate Apex classes & Flows to run for a plugin, removed SObject-specific fields on `LoggerPlugin__mdt`, added new fields `Log__c.LogPurgeAction__c` and `LoggerSettings__c.DefaultLogPurgeAction__c` - Added support for plugins within LogBatchPurger + added archiving in BigObject plugin (#288) - Added SObject-specific 'execution order' fields on `LoggerPlugin__mdt` - Added picklist values for API versions in calendar year 2022 - Reintroduced the ability to disable trigger handler classes by adding new `LoggerParameter__mdt` records to control via configuration - Renamed CMDT records that control each trigger handler - Added new abstract methods in LoggerSObjectHandler for retrieving each handler's related records in `LoggerParameter__mdt` & `LoggerPlugin__mdt` - Added the ability to configure the list of save methods shown in `loggerSettings` LWC via `LoggerParameter__mdt`, using the prefix 'CustomSaveMethod' - Standardized test-visible method naming conventions to start with `setMock` & updated approaches to use a Map instead of List for CMDT records - Fixed issues in LoggerSObjectHandler for invalid Apex & Flow plugins, started new test class LoggerSObjectHandler_Tests - Added `Log__c` list view to show logs that will be purged in the next 10 days, fixed LogEntryArchiveBuilder using the wrong value for LoggedById**c, updated index on LogEntryArchive**b again (still a WIP), retrieved & formatted metadata for LogEntryArchive\*\*b, updated Admin.profile - Closed #226 by adding new "Log Retentions Rule" plugin, based on my other project, ApexValidationRules - @jamessimone closed #117, and I closed #228 by adding new "Big Object Archiving" plugin - Lowercased the plugin directories - Added plugins folder to the script for generating apex doc files, generated updated docs for Apex - Bumped package version number for the managed package - Added 5 minute wait between unlocked package version creation & package installation, added deprecated `LoggerPlugin__mdt` validation rule & field back into the managed-package folder, cleaned up some docs markdown files - Added tests in LoggerSettingsController_Tests & LogHandler_Tests for new custom setting LoggerSettings**c.DefaultLogPurgeAction**c - Added 'deprecated' to the label of several deprecated fields on `LoggerPlugin__mdt` & removed the related handler methods - More improvements for CMDT records used in the classes LoggerParameter & LoggerPlugin - Added LoggerParameter.matchOnPrefix() to return CMDT records with a specified prefix in the DeveloperName field - Added inner Comparable class in LoggerPlugin to handle custom sorting, since there are some limitations with SOQL queries for CMDT - Classes like LogBatchPurger, LoggerSObjectHandler, and LoggerSettingsController no longer need to track their own mock CMDT records, the LoggerPlugin & LoggerParameter classes now fully handle mocks for their corresponding CMDT objects - LoggerParameter & LoggerPlugin now require mock CMDT records to have DeveloperName populated - they'll throw errors if the field is null - Also added System namespace to some calls for Test.isRunningTest() to handle crazy orgs that have a custom Test class deployed - Added the ability to disable all triggers during tests via new @testvisible method LoggerSObjectHandler.shouldExecute(Boolean) - Moved LoggerTestUtils to the configuration folder, added public methods for mocking all CMDT records - Consolidated plugin-framework folders back into the configuration folders - Moved demo experience cloud metadata under config, instead of under nebula-logger, updated build.yml to deploy instead of push metadata - Updated test classes to use new overloads for LoggerTestUtils.setMock() - Added new class LoggerDataStore to centralize all database, event bus & queueable operations - Closed #193 by adding classes `LoggerMockDataCreator` for generic mock & data creation (part of configuration layer), `LoggerTestConfigurator` forNebula Logger-specific mocking & setup (part of the log-management layer), and `LoggerMockDataStore` for mocking database & event bus operations throughout the codebase - Started work on #292 by adding businessStatus, complianceGroup & securityClassification nodes to several custom fields (still a WIP) - Rewrote tests in `LogEntryEventBuilder_Tests` to use new methods in `LoggerMockDataCreator` for creating mock instances of database result classes - Scope creep: added fallback fields for setting `RecordName__c` in LogEntryHandler on objects that don't have a display name field, also added some caching - Moved permissionSets and customPermissions folders from configuration to log-management - This should make the configuration folder fully standalone (i.e., it could be deployed by itself) - The perm sets were already referencing all of the objects within log-management, so it seems like it's better to keep the perm sets organized with - Changed labels to just "Entry #" on both `LogEntryEvent__e` and `LogEntryArchive__b` - Fixed an issue in `LogEntryEventBuilder` where some stack trace lines would be duplicated - Scope creeped a bit by adding new fields `DatabaseResultCollectionSize__c` and `RecordCollectionSize__c` - I had planned to add these as part of #222, but decided to add them now so I could duplicate the fields to the `LogEntryArchive__b` big object as well. This also helped to identify (and fix) some issues with missing asserts in LogEntryEventHandler_Tests. - Replaced settings field `IsPlatformEventStorageEnabled__c` with new field `DefaultPlatformEventStorageLocation__**__c`, providing a way for plugins (big object plugin in particular) to provide additional storage location. The big object plugin provides a custom option BIG_OBJECT, in addition to the default value CUSTOM_OBJECTS that will now be used out of the box - Added stack trace fields to the Slack message created via SlackLoggerPlugin - Updated most test classes to use `@IsTest(isParallel=true)`. A few tests (especially integration tests) that do DML on User can't use parallel testing, but most test classes are now using it - Partially implemented #240 by adding HTTP Request & HTTP Response fields to `LogEntryEvent__e`, `LogEntry__c` and `LogEntryArchive__b`. More scope creep yet again, but this will save some headaches long term - since Big Objects can't (easily) have new fields added, I wanted to go ahead and get these new fields added to the data model before the first release of the Big Object plugin. I'll probably add some logging overloads in Logger for these items in a future release, but for now, only LogEntryEventBuilder has new methods for HttpRequest & HttpResponse - Updated some internal debug messages in various places to use Logger's logging methods, instead of `System.debug()` - Deprecated the fields `LogEntryDataMaskRule__mdt.ApplyToMessage__c` and `LogEntryDataMaskRule__mdt.ApplyToRecordJson__c` - Fixed test failures that occur when email deliverability is disabled in an org Co-authored-by: James Simone <[email protected]>
As a follow-up to #287 (and part of a larger overhaul of the plugin framework for the upcoming release
v4.7.1
), I've reworked even more of the plugin framework to add support for plugins within theLogBatchPurger
classPlugin Framework Changes
LoggerSObjectHandlerPlugin
. It has been replaced with 2 new interfaces:LoggerPlugin.Batchable
- used to create plugins forLogBatchPurger
(batch deletion job)LoggerPlugin.Triggerable
- used to create plugins forLoggerSObjectHandler
(trigger framework used by Logger)LoggerPlugin__mdt
+ added new fields to control the sort order of plugins used byLogBatchPurger
LoggerSettings__c.DefaultLogPurgeAction__c
- this then sets the default value of the fieldLog__c.LogPurgeAction__c
. This field value isn't used byLogBatchPurger
directly, but it provides a place for plugins to set/read additional actions that can be used within plugin logic (see BigObject changes below for an example)LoggerSettingsController
to query forLoggerParameter__mdt
records that start with 'CustomLogPurgeAction' so that plugins can define & package new actions. Any matching values will be displayed in the LWCloggerSettings
as a picklist option forLoggerSettings__c.DefaultLogPurgeAction__c
LogBatchPurger
now uses a new inner classBatchPurgerInput
to provide contextual info to pluginsLogBatchPurger.BatchPurgerInput
andLoggerSObjectHandler.SObjectHandlerInput
will both need to be changed to top-level classes so that Flow plugins can use them. From this page, Apex-defined types can't be used in Flow unless they are top-level classes. I've already scope-creeped in this branch a lot, so I'll make this change after merging this PR into theplugin-framework-overhaul
branch.@TestVisible
methods used throughout the codebase to mock custom metadata recordsBigObject Plugin Changes
The BigObject plugin has been updated to also work as a batch purger plugin (in addition to being an SObject handler plugin that @jamessimone implemented in #287)
LogEntryArchiveBuilder
into 2 classes:LogEntryArchiveBuilder
- this now only handles converting aLogEntryEvent__e
into aLogEntryArchive__b
record (implemented via Big Object Plugin #287), as well as converting aLogEntry__c
record into aLogEntryArchive__b
record (to cover most of the functionality in Add support for archiving Log__c and LogEntry__c records into a BigObject #128)LogEntryArchivePlugin
- this new class implements the interfacesLoggerPlugin.Batchable
andLoggerPlugin.Triggerable
so archiving can be used as the initial storage place (when a user's save method ==BIG_OBJECT
) as well as a secondary storage place (when aLog__c
hasLogPurgeAction__c
== 'Archive')LogEntryArchive__b
to (more or less) match the list of fields onLogEntryEvent__e
LoggerSettings__c.DefaultLogPurgeAction__c
. WhenLogBatchPurger
executes, any matchingLog__c
/LogEntry__c
/LogEntryTag__c
data will be archived intoLogEntryArchive__b
before being deleted byLogBatchPurger
LogEntryArchive__b
, but I'm guessing we'll probably try a few other variations on what to use within the indexLogEntryArchive__b
records via a new LWC + custom tabSlack Plugin Changes & Log Retention Rules Plugin Changes