Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for isolated entities #358

Merged
merged 5 commits into from
Apr 26, 2024
Merged

Conversation

sebastianburckhardt
Copy link
Member

Adds support for the new entity implementation used by dotnet-isolated.

Also, updates the DF and DT dependencies to 2.13.1 and 2.16.1.

@sebastianburckhardt sebastianburckhardt added this to the 1.4.4 milestone Mar 5, 2024
@sebastianburckhardt sebastianburckhardt modified the milestones: 1.5.1, 1.5.0 Mar 18, 2024
Copy link
Member

@davidmrdavid davidmrdavid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questions but overall looks good

<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="2.12.0" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.DurableTask" Version="2.13.0" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to first release the current DF Extension release prior to getting this out, and then update this dependency to that upcoming release. We had to remove the latest DF Extension release due to a regression in OOProc serialization, so I'd like to avoid upgrading Netherite to depending on that faulty extension.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, maybe that means we should not include this PR in the 1.5.0 release?

Since we probably want this out sooner than later to make progress on the "partition corruption" theme.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we probably want the fix in Azure/azure-functions-durable-extension#2748 to be included so I think it makes sense to wait for the next DF Extension release.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, maybe that means we should not include this PR in the 1.5.0 release?

Yeah I'd say let's hold off, but prioritize this release shortly after.

Comment on lines 85 to 87

internal bool IsSet => this.HasRuntimeStatus || !string.IsNullOrWhiteSpace(this.InstanceIdPrefix)
|| !(this.CreatedTimeFrom is null) || !(this.CreatedTimeTo is null);
|| !(this.CreatedTimeFrom is null) || !(this.CreatedTimeTo is null) || this.ExcludeEntities;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does IsSet represent?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It means "this query has some filtering going on". I will rename it to make that clearer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually looks like this property is not used anywhere (anymore?) so I will remove it altogether.

/// </summary>
public int MaxConcurrentOrchestratorFunctions { get; set; } = 100;

/// <summary>
/// Gets or sets the maximum number of work items that can be processed concurrently on a single node.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's make this more precise by saying "entity" explicitly

Suggested change
/// Gets or sets the maximum number of work items that can be processed concurrently on a single node.
/// Gets or sets the maximum number of Entity work items that can be processed concurrently on a single node.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the comments for activities, entities, and orchestrations.

Comment on lines 858 to 861
if (request.ReleaseOrphanedLocks && status.LockedBy != null)
{
tasks.Add(CheckForOrphanedLockAndFixIt(state, status.LockedBy));
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have we always had this functionality of releasing orphaned locks? Maybe I missed it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not "always", but it has been there for a long time now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, we were always planning to call this automatically every once in a while, but never actually implemented that since it always seemed like it should be done by the same periodic task that does the autopurge, and we haven't gotten around to that one yet.

Comment on lines 846 to 847
// list all entities (without fetching the input) and for each one that requires action,
// perform that action. Waits for all actions to finish after each page.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you clarify this bit about "requiring an action"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rewrote the comments for this function. Also, a bit of simplification.

@sebastianburckhardt sebastianburckhardt modified the milestones: 1.5.0, 1.5.1 Mar 18, 2024
Copy link
Member

@davidmrdavid davidmrdavid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM though there'a conflict in the csproj

/// </summary>
[DataMember]
internal bool ExcludeEntities { get; set; }

Copy link
Contributor

@jaliyaudagedara jaliyaudagedara Apr 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Extra line break

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, just removed it: a6a210e

@davidmrdavid davidmrdavid merged commit 24efd54 into main Apr 26, 2024
2 checks passed
@davidmrdavid davidmrdavid deleted the pr/support-isolated-entities branch April 26, 2024 16:31
@davidmrdavid
Copy link
Member

FYI @sebastianburckhardt that I went ahead and merged this (which I had approved some time back) after resolving some merge conflicts in the csproj.

@jaliyaudagedara
Copy link
Contributor

@sebastianburckhardt @davidmrdavid

Still getting System.NotSupportedException: Durable entities are not supported by the current backend configuration with 1.5.1.

Logged an issue: #390

@davidmrdavid
Copy link
Member

@jaliyaudagedara: Thanks - seems the AssemblyInfo.cs file, which needs to be manually updated on every release, was left intact. As a result, the host process (separate from the .NET isolated process) was loading the wrong version of Netherite, which explains this issue. I'm fixing that here: #393

@jaliyaudagedara
Copy link
Contributor

@davidmrdavid , got it! thanks for the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants