Can't perform index search on text fields (e.g: archivematica.uuid) when the archivematica dataset is not set as indexable

[photomedia]

There is a bug I just discovered that needs an urgent fix: searching archivematica records by using the archivematica UUID using the "Filter" causes the following error:

Can't perform index search on archivematica.uuid when the archivematica dataset is not set as indexable.

Not only that, because the "Filter" screen "remembers" this search field as being set, now I'm locked out of my Archivematica records screen.

[photomedia]

I figured out how to clear the "Filter" so that at least we can get back to our records display.
We can do this by sending the same request as a "Reset form" on the search filter on the archivematica record screen:
https://[YOUR-REPOSITORY-URL]/cgi/users/home?dataset=archivematica&_listing_order=amid&screen=Listing&amid=&datasetid=&dataobjid=&uuid_merge=ALL&uuid=&is_dirty=EITHER&result_log_result_merge=ANY&result_log_action_merge=ANY&result_log_timestamp=&result_log_comment_merge=ALL&result_log_comment=&_action_reset_filters=Reset+the+form
That has the same effect as resetting the form, and so it wont keep trying to search for the UUID, which causes an error.

[photomedia]

In addition to filtering by archivematica UUID field, the same error shows up if we try to Filter by the Comment field, which is also a text field:

Can't perform index search on archivematica.result_log_comment when the archivematica dataset is not set as indexable

The other fields (eg: AMID) don't thow this error when filtering.

So the question becomes: how do we properly filter/configure a text field so that it can be used with the "Filter"?
Archivematica UUID field is a text field because the UUID includes letters, numbers and dashes.

Perhaps @wfyson, you might know?

The declaration for these fields is here:

EPrintsArchivematica/lib/plugins/EPrints/DataObj/Archivematica.pm

Line 21 in 53b4a0e

{ name => "uuid", type => "text", sql_index => 1 },

[photomedia]

I found a way to resolve the error! I found this thread: https://www.eprints.org/eptech/msg02993.html
And based on that, I see that we need to have the following line, when we create the dataset:
index => 1
So our z_archivematica.pl file (https://github.com/eprintsug/EPrintsArchivematica/blob/master/cfg/cfg.d/z_archivematica.pl) needs to have:

# Add new dataset for tracking archivematica events
$c->{datasets}->{archivematica} = {
	class => "EPrints::DataObj::Archivematica",
	sqlname => "archivematica",
	index => 1
};

Now the question remains, should I use sql_index => 1 or text_index => 1 for the text fields? here:

EPrintsArchivematica/lib/plugins/EPrints/DataObj/Archivematica.pm

Line 21 in 53b4a0e

{ name => "uuid", type => "text", sql_index => 1 },

[photomedia]

I confirm that adding index => 1 resolves the bug, the error message does not show up, but the searching/filtering is not giving me any results when I search by UUID. Perhaps I need to run a 'reindex' command on the "archivematica" dataset?

[wfyson]

Hi @photomedia, sorry for my lack of response previously, I was on annual leave. It sounds like this got sorted however? It's been a very long time since I've looked at this Archivematica plugin so I'm not too familiar with it, but did running a reindex on the "archivematica" dataset fix things?

[photomedia]

Hi @wfyson. The severity of the issue is less now, because after adding the index => 1 to the dataset, when filtering on those text fields, I no longer get an error message that locks the user out of the "manage records" screen for Archivematica records.
However, reindexing the archivematic dataset doesn't fix the issue, or I should rather say that attempting to run reindex on the dataset now gives me errors like this:

 ./epadmin reindex REPOID archivematica 3
DBD::mysql::db do failed: Table 'REPOID.archivematica__rindex' doesn't exist at /opt/eprints3/bin/../perl_lib/EPrints/Database.pm line 3054.
SQL ERROR (do): DELETE FROM `archivematica__rindex` WHERE `amid`='3' AND `field` IN ('amid','datasetid','dataobjid','uuid','result_log','result_log_timestamp','result_log_action','result_log_comment','result_log_result','is_dirty')
SQL ERROR (do): Table 'REPOID.archivematica__rindex' doesn't exist (#1146)
DBD::mysql::db do failed: Table 'REPOID.archivematica__index_grep' doesn't exist at /opt/eprints3/bin/../perl_lib/EPrints/Database.pm line 3054.
SQL ERROR (do): DELETE FROM `archivematica__index_grep` WHERE `amid`='3' AND `fieldname` IN ('amid','datasetid','dataobjid','uuid','result_log','result_log_timestamp','result_log_action','result_log_comment','result_log_result','is_dirty')
SQL ERROR (do): Table 'REPOID.archivematica__index_grep' doesn't exist (#1146)
DBD::mysql::st execute failed: Table 'REPOID.archivematica__rindex' doesn't exist at /opt/eprints3/bin/../perl_lib/EPrints/Database.pm line 1289.
DBD::mysql::st execute failed: Table 'REPOID.archivematica__rindex' doesn't exist at /opt/eprints3/bin/../perl_lib/EPrints/Database.pm line 1289.
DBD::mysql::st execute failed: Table 'REPOID.archivematica__rindex' doesn't exist at /opt/eprints3/bin/../perl_lib/EPrints/Database.pm line 1289.
DBD::mysql::st execute failed: Table 'REPOID.archivematica__rindex' doesn't exist at /opt/eprints3/bin/../perl_lib/EPrints/Database.pm line 1289.

Any idea why the rindex table is missing, and how to get it to be created?

[wfyson]

Aah, I wonder if an "epadmin update <repo_id>" is needed to add the rindex tables to the database first? With the field already present however, I'm not sure if an epadmin update will realise there's anything new to do.

[photomedia]

Hi @wfyson . That worked! I ran epadmin update, it added the dataset, and after that, reindexing solved the issue and now it's filtering as well. Thank you!