Validate ASM installations before loading #10621
Conversation
The check for the specific assemblies tbb.dll and tbbmalloc.dll is generalized to a full file list validation of detected ASM locations. This way, Dynamo is guarded against any incomplete/unusual ASM binary folders that other applications might include. The lists of files for each version were taken from LibG. They cannot be reused from LibG without involving major changes in the preloader, so the lists should be kept in sync as new major release of ASM occur.
|
Creating this as a WIP to get early feedback. Will add unit tests before removing it. |
|
@mmisol I like it - but I want to make sure you saw the comment from @aparajit-pratap suggesting a more strict registry search. |
|
I saw it. I think this approach is safer because we explicitly declare the files we need. Reducing the registry search results to filter out known bad ASM installations gets trickier to update, as we don't know for sure all released products containing ASM. On the other hand, updating the list of ASM binaries for each release of ASM should be easy to do. |
|
@mmisol what are the csv files for? |
|
They are used in the tests. Wanted to provide an input as similar to |
|
I thought about this some more and have more questions. I think when we need to support a new product integration tomorrow, we may need to do more work in addition to this in order to support loading ASM from product x. If, say, a user has x installed on their machine (and no other product), will ASM still be loaded correctly? I think even in that case, Dynamo will try and load ASM from the Revit folder that comes with the x install (as shown by Martin) and fail. Plus there will be no fallback option as Revit or any other product supporting ASM will not be installed on the user's system. If there's another folder location in the x installation directory that happens to have all of the required ASM dll's needed to load correctly, will that folder location be considered in the search? If so, then that would be fine. |
|
Link to self-serve job: https://master-15.jenkins.autodesk.com/view/DYN/job/DYN-DevCI_Self_Service/568/ |
|
@aparajit-pratap Trying to answer your question. |
|
@mmisol Adding another product is not planned until our team got PM confirmation, so let's not include it in the current task scope. |
|
@mmisol @QilongTang my question was not intended at x specifically. I just took it as an example and the reason I took the example of x is it is on the cards for an integration soon. Yes, this task was not about loading ASM from a specific product but we should keep in mind such integrations and the future possibilities of supporting them in our architecture/solution before we realize we need to revisit this time and again. |
|
@aparajit-pratap - but if we need to add product X - we just add X to the list of products to load from - then its install location will be present in the list of ASM installs, and we'll find it later. |
|
@aparajit-pratap hmm Are you proposing a config based list instead of hard coded strings? If that is the case or you have some other ideas around this, let's discuss. I had thought about it before but that approach did not seem safe for me but we can revisit this discussion. |
|
@aparajit-pratap When I said this PR does not go in the direction of "adding support for another product's installation of ASM" I didn't mean it goes against that direction either. If you think about what this PR is doing carefully, this is just a safeguard. Of course, products will need to provide a valid ASM installation for us to pick it up, but that's another story. |
* add namespace conflict tests for short name replacer and node to code (#10611) * Register custom node before package load reset (#10591) This fixes a problem where existing custom nodes in the home workspace became unresolved after a package that contained binaries was loaded. The cause of the problem was that the compiled function was not available at the time of the execution after a VM reset. Now the data is registered on package load, by queueing it in pendingCustomNodeSyncData. This results in a CompileCustomNodeAsyncTask being scheduled before the update of the home workspace graph takes place. * Increase coverage of the Core folder (#10609) * Add a test for the CrashPromptArgs class * Update DynamoCoreTests.csproj * Added NotificationObject tests * Updated the test, NotificationObject coverage at 100% * Changed some access modifiers * Added coverage for Updates/BinaryVersion * Added comments and fixed names * Revert "Added comments and fixed names" This reverts commit 42cd024. * Revert "Added coverage for Updates/BinaryVersion" This reverts commit 679deca. * Increased coverage in CrashPromptArgs * Added some Core coverage * DYN-2560 - Increase the code coverage: DynamoCore/Models Folder First Part (#10612) * DYN-2560 - Increase the code coverage: DynamoCore/Models Folder I started adding just one test method TestOnRequestDispatcherBeginInvoke() for testing the DynamoModelEvents class. * DYN-2560 - Adding test cases for DynamoModelEvents Adding test cases for DynamoModelEvents * DYN-2560 - Adding test cases for DynamoModelEvents I added all the test cases for the all events in the DynamoModelEvent class, i just need to fix the last 6 of them. * DYN-2560 - Adding test cases for DynamoModelEventsArgs I added several test cases for the classes inside the DynamoModelEventsArgs.cs file. ZoomEventArgs TaskDialogEventArgs EvaluationCompletedEventArgs DynamoModelUpdateArgs FunctionNamePromptEventArgs PresetsNamePromptEventArgs ViewOperationEventArgs PointEventArgs WorkspaceEventArgs ModelEventArgs * DYN-2560 - Code Review Comments Based in the comment done by Aaron in the GitHub pull request, I added more description comments for the method TestTaskDialogEventArgs() and also I added comments for a local function * DYN-2560 - Code Review Comments 2 There was a spelling error in two methods for the word "Internally", then I fixed this error in the two places. * Python Engine Enum (#10618) * Cherrypick * Comments * Add unit test * Comments * Handle runtime table gaps on code block deletion (#10605) When the runtime table are built there is an implicit assumption that the code block ids are consecutive. However, that is not always the case, as the deletion of a procedure causes the deletion of its child code blocks, which may generate gaps in the id numbering. In order to make the code resiliant to these gaps, the runtime tables are sized based on the largest code block id, rather than in the amount of code blocks. * Validate ASM installations before loading (#10621) The check for the specific assemblies tbb.dll and tbbmalloc.dll is generalized to a full file list validation of detected ASM locations. This way, Dynamo is guarded against any incomplete/unusual ASM binary folders that other applications might include. The lists of files for each version were taken from LibG. They cannot be reused from LibG without involving major changes in the preloader, so the lists should be kept in sync as new major release of ASM occur. * SQ bug fix (#10622) * (1) Null reference bug fix from SQ dashboard * Add support for debug modes (#10603) * Add support for debug modes * Increase coverage of the Updates folder (#10628) * Add a test for the CrashPromptArgs class * Update DynamoCoreTests.csproj * Added NotificationObject tests * Updated the test, NotificationObject coverage at 100% * Changed some access modifiers * Added coverage for Updates/BinaryVersion * Added comments and fixed names * Added asserts for happy path, changed namespace and deleted a console function * Removed using * Revert "Removed using" This reverts commit 022823d. * Removed a change that should not be there * Added coverage for Updates folder * Tests for CPython3 Engine as well as for IronPython. * [WIP][FEEDBACK] Reduce test time by substantially reducing number of serialization tests. (#10624) * reduce number of serialization tests by factor 3~ * reduce wpf json serialization tests * Handle missing instance calling method statically (#10630) A code block node calling an instance method in its static form would make Dynamo crash if the instance was not provided. An example of this would be calling 'Curve.Patch();'. The cause of the issue was that the default argument was ultimately tried to be interpreted as a pointer. By avoiding that wrong conversion the engine is now able to surface the real problem as a human-readable warning. * Python3 Selection Under Debug Modes (#10629) * Cherrypick Python3 changes * Cleanup * Use Debug Modes * CleanUp * Rename Function * Clean Up * Do not use anouymous function as handler * Revert newer language change * Add adp analytics to Dynamo (#10576) * add ADPTracker register * Fix non-array item search for dot operation (#10633) When an array is passed to the dot operation, it needs to get an item to determine the actual class being processed. This was done in ArrayUtils.GetFirstNonArrayStackValue, but the function only checked the first item of the array and its descendants. This caused the dot operation to failed when passed an array that contained an empty array as its first item. In order to fix the problem, the function was changed to check for non-array items in the entire array. The function should stop as soon as the first non-array item is found. * Addressing some comments * some more comments * Update AssemblyInfo.cs * Update AssemblyInfo.cs * Marking the python node as modified when its engine property is modified Also updated some tests. * Remove unwanted check. * changes to test * Update AssemblySharedInfo.cs * Using python engine's AcquireLock to avoid deadlock. The deadlock was happening only when multiple PythonEngine's are initialized on a thread from 2 different test fixtures in the same run. The main difference is calling this function PythonEngine.BeginAllowThreads(). Also we do not want to initialze the python engine if it is already initialized. Co-authored-by: aparajit-pratap <[email protected]> Co-authored-by: Martin Misol Monzo <[email protected]> Co-authored-by: Bruno Yorda <[email protected]> Co-authored-by: Roberto T <[email protected]> Co-authored-by: Aaron (Qilong) <[email protected]> Co-authored-by: Ashish Aggarwal <[email protected]> Co-authored-by: Tibi <[email protected]> Co-authored-by: Michael Kirschner <[email protected]>
Purpose
The check for the specific assemblies tbb.dll and tbbmalloc.dll is
generalized to a full file list validation of detected ASM locations.
This way, Dynamo is guarded against any incomplete/unusual ASM binary
folders that other applications might include.
The lists of files for each version were taken from LibG. They cannot
be reused from LibG without involving major changes in the preloader,
so the lists should be kept in sync as new major release of ASM occur.
Declarations
Check these if you believe they are true
*.resxfilesReviewers
@DynamoDS/dynamo