How to call 'GetItemsChangedSink'

[PascDuf]

Hi,

I want to use Vanara to listen to Windows Search events. Listening to notifications doesn't work. I am registering handlers with ISearchCatalogManager.GetItemsChangedSink but when I change a file that is indexed my handler doesn't get called. This is what I am doing:

SearchApi.ISearchManager searchManager;
SearchApi.ISearchCatalogManager catalogManager;

searchManager = (SearchApi.ISearchManager)new SearchApi.CSearchManager();
catalogManager = searchManager.GetCatalog("SystemIndex");

Guid riid = Marshal.GenerateGuidForType(typeof(SearchApi.ISearchItemsChangedSink));
SearchApi.ISearchItemsChangedSink itemsChangedSink;
object itemsChangedSinkObject;

Guid catalogResetSignature;
Guid checkPointSignature;
uint lastCheckPointSignature;


catalogManager.GetItemsChangedSink(
    this,
    riid,
    out itemsChangedSinkObject,
    out catalogResetSignature,
    out checkPointSignature,
    out lastCheckPointSignature);

itemsChangedSink = (SearchApi.ISearchItemsChangedSink)itemsChangedSinkObject;

I pass 'this' because the current class is implementing SearchApi.ISearchNotifyInlineSite. More specifically, these 2 methods:

public void OnItemIndexedStatusChange(SearchApi.SEARCH_INDEXING_PHASE sipStatus, uint dwNumEntries, SearchApi.SEARCH_ITEM_INDEXING_STATUS[] rgItemStatusEntries)
{
}

public void OnCatalogStatusChange(in Guid guidCatalogResetSignature, in Guid guidCheckPointSignature, uint dwLastCheckPointNumber)
{
}

I was expecting OnItemIndexedStatusChange to be called anytime a file in a crawl scope changes of is created. What am I doing wrong?

[dahall]

Have you marked your class as [ComVisible(true)] and setup the assembly to export COM objects?

[PascDuf]

I have isolated the listener class in a separate assembly, I made it COM visible but still, I am not receiving any notification when an indexed file changes. I must be doing something wrong but I don't know what.

[PascDuf]

When I call itemsChangedSink.StartedMonitoringScope I get a "System.NotImplementedException". Looks like this feature is not fully implemented...

[PascDuf]

No but will try that now. Thanks ! Pascal

…

On Tue, Apr 2, 2024 at 1:40 AM David Hall ***@***.***> wrote: Have you marked your class as [ComVisible(true)] and setup the assembly to export COM objects? — Reply to this email directly, view it on GitHub <#447 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BCC7WBZEQFATGK6HINFPQXLY3JAGPAVCNFSM6AAAAABERCNI4WVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DSNZZGM4DO> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Pascal Dufresne Quilt Labs

[dahall]

I'm trying to create a test harness for this. I have the code and supporting interface implementations, but I'm not able to trigger an event to test those interfaces. How are you forcing an index file change? Where did you find documentation or a sample to follow on this? The Microsoft Docs are very unhelpful.

[MegaMagicPower]

I'll try to add some context here as I've also been trying to get this to work for a few days and this was the only reference to "GetItemsChangedSink" in all of Github when I searched. This may or may not be completely accurate as I'm very new to both the search service and interop. This link provides an overview of the windows search service and how it works at a high level:

https://learn.microsoft.com/en-us/windows/win32/search/-search-indexing-process-overview

Its long but from what I understand, the gist is that a producer of events creates an event and puts it somewhere, and periodically the search service will poll these producers for new events and put relevant data from them into the index to be searchable. For our context, the producer is the NTFS filesystem that writes file changes to the change journal, which is then polled and processed by the search service to be placed into the index, which then allows you to search to for files based on their name, content, etc.

The periodic polling however (what they call a "crawl") is not the only way to have the index get updated. Rather than waiting for it to "crawl" on its own looking for new changes, you can explicitly state that changes have occurred and start the process yourself at any time via notifications. This link explains it pretty well:

https://learn.microsoft.com/en-us/windows/win32/search/-search-3x-wds-support

For "Indexer-Managed Notifications", this just means you're sending a notification to force the indexing to happen early, but don't care about what happens after that. The scenario that @PascDuf and I are trying to do is the one titled "Provider-Managed Notifications", where you send a notification but also want updates about the progress of those notifications as the index process is happening.

This is where the ISearchItemsChangedSink and ISearchNotifyInlineSite interfaces come in. ISearchItemsChangedSink will be used as your notification producer. The notification's themselves are the struct SEARCH_ITEM_CHANGE. You create 1 or more of them, put them in a struct, and then pass them into ISearchItemsChangedSink::OnItemsChanged. Doing this should kick off the indexing process and eventually deliver the results of each notification via ISearchNotifyInlineSite::OnItemIndexedStatusChange, which is just a callback method that you implement on the interface. Upon creation of ISearchItemsChangedSink, you pass it your implemented ISearchNotifyInlineSite, which is how it knows where to call back to with the results.

This link is essentially the previous link but expanded to explain how to fit it all together like I've just explained. The "Implementing Provider-managed Notifications" is the relevant section:

https://learn.microsoft.com/en-us/windows/win32/search/-search-3x-wds-notifyingofchanges

With all of that out of the way, I believe @PascDuf 's problem is that he isn't sending notifications; he mistakenly believes that the ISearchNotifyInlineSite callback should be executed even if the index is changed via a "crawl", but I think the callback is only done when you explicitly send a notification. I've tried to do that myself but whenever I try to call "ISearchItemsChangedSink::OnItemsChanged()" with my notification, I get "System.Runtime.InteropServices.COMException: '0x80040E21'", which isn't really very helpful at figuring out what's wrong.

using System.Runtime.InteropServices;
using System.Runtime.InteropServices.Marshalling;
using Vanara.PInvoke;
using static Vanara.PInvoke.SearchApi;

SearchApi.ISearchManager searchManager;
SearchApi.ISearchCatalogManager catalogManager;

searchManager = (SearchApi.ISearchManager)new SearchApi.CSearchManager();
catalogManager = searchManager.GetCatalog("SystemIndex");

Guid riid = Marshal.GenerateGuidForType(typeof(SearchApi.ISearchItemsChangedSink));
SearchApi.ISearchItemsChangedSink itemsChangedSink;
object itemsChangedSinkObject;

Guid catalogResetSignature;
Guid checkPointSignature;
uint lastCheckPointSignature;

TestInlineSite testInlineSite = new TestInlineSite();


catalogManager.GetItemsChangedSink(
    testInlineSite,
    riid,
    out itemsChangedSinkObject,
    out catalogResetSignature,
    out checkPointSignature,
    out lastCheckPointSignature);

itemsChangedSink = (SearchApi.ISearchItemsChangedSink)itemsChangedSinkObject;

SEARCH_ITEM_CHANGE itemChange = new SEARCH_ITEM_CHANGE()
{
    Change = SEARCH_KIND_OF_CHANGE.SEARCH_CHANGE_MOVE_RENAME,
    Priority = SEARCH_NOTIFICATION_PRIORITY.SEARCH_NORMAL_PRIORITY,
    pUserData = 0, //This is supposed to be a "Pointer to user information", but I have no idea what this means or what it wants
    lpwszURL = "file:C:/1new.txt",
    lpwszOldURL = "file:C:/1.txt"
};
System.IO.File.Move("C:/1.txt", "C:/1new.txt");

SEARCH_ITEM_CHANGE[] itemsChangedArray = { itemChange };
uint[] retInts = new uint[1];
HRESULT[] hresults = new HRESULT[1];

itemsChangedSink.OnItemsChanged(1, itemsChangedArray, retInts, hresults);

while (true)
{

}

public class TestInlineSite : ISearchNotifyInlineSite
{
    public void OnItemIndexedStatusChange([In] SEARCH_INDEXING_PHASE sipStatus, [In] uint dwNumEntries, [In][MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 1)] SEARCH_ITEM_INDEXING_STATUS[] rgItemStatusEntries)
    {
        Console.WriteLine("OnItemIndexedStatusChange");
    }
    public void OnCatalogStatusChange(in Guid guidCatalogResetSignature, in Guid guidCheckPointSignature, uint dwLastCheckPointNumber)
    {
        Console.WriteLine("OnCatalogStatusChange");
    }
}

I'm sure I'm just doing something extremely wrong regarding the interop as I don't really understand it and I can't find a solid tutorial explaining the different interop attributes and how it all works. I'm just thankful for libraries like this that have already done all of that for me as I was able to use another function to accomplish sort of what I wanted, which is to monitor when the index gets updated, but I'd rather get the notifications working if I can as its a better fit for what I'm trying to do which is change some files and then wait until the indexing of them occurs before continuing on with further code.

@PascDuf If you just want to monitor the state of the index, I found that you can use ISearchCatalogManager::NumberOfItemsToIndex(). The first parameter, plIncrementalCount, seems to track files that have yet to be indexed. It seems to get set as soon as a file changes and you can simply poll this method every so often until it changes back to 0 to know when all indexing has finished. This isn't as good as having a push notification directed at exactly the files you're changing like ISearchNotifyInlineSite, but is better than nothing.

You could also try the IRowsetEvents interface. This will give you a push notification that you can target on a specific root directory. You can find an example of using this in C++ here, with the relevant method being WatchEvents():

https://github.com/microsoft/Windows-classic-samples/blob/main/Samples/Win7Samples/winui/WindowsSearch/SearchEvents/evtdemo.cpp#L359

[dahall]

I just added a unit test with the code proposed, but could never get OnItemsChanged to not throw an exception.