Unable to add a search attribute to a fresh dev server

[aromanovich]

What are you really trying to do?

During my e2e-tests I run ya tool temporal server start-dev, and once the server is online I try to add a custom search attribute to it using Operator service API or CLI.

Describe the bug

Adding a search attribute ends with an error unable to add search attributes: Search attribute ... already exists, the server logs a panic: https://gist.github.com/aromanovich/7736838748d6b10c2ba9b9694ee321d9. Despite all that the attribute gets added, but doesn't become available immediately.

All attempts to execute a workflow with the new search attribute fail with Namespace ... has no mapping defined for search attribute ... for around 10 seconds, and only then succeed.

Minimal Reproduction

#!/bin/bash
set -x

temporal server start-dev --namespace default --log-level warn &

until temporal operator cluster health | grep SERVING
do
  sleep 1
done

temporal operator search-attribute create --address localhost:7233 --namespace default --name DeadlineTime --type Datetime

until temporal workflow start --address localhost:7233 --namespace default --task-queue test --type TestWorkflow --search-attribute 'DeadlineTime="2021-06-07T16:46:34.236-08:00"'
do
  sleep 1
done

The output: https://gist.github.com/aromanovich/61400dad2b2a1365011e22e10fb231a2

Environment/Versions

# temporal --version
temporal version 0.8.0 (server 1.20.1) (ui 2.13.3)

[feedmeapples]

The server throwing error "search attribute already exists" (though still adding the attribute) was fixed recently on the server side temporalio/temporal#4148

Keeping the issue open until this is out

[feedmeapples]

The 10 seconds delay is currently by design with SQLite backend. There may be changes in future, at this point would suggest just working around this from the tests

[aromanovich]

was fixed recently on the server side

Thank you for the reference, I'll be waiting for the next release.

10 seconds delay is currently by design with SQLite backend

Is there a chance to avoid this delay in a special case of adding search attributes at the very start of the dev server (as described in #21)?
Probably by writing them to the SQLite database directly before starting the server (totally uneducated guess of course 🙂)?

[bergundy]

The delay isn't by design, it's a bug that was introduced in server 1.20.
You can track this here: temporalio/temporal#4017

[bergundy]

+1 for adding support for #21

[feedmeapples]

The delay isn't by design, it's a bug that was introduced in server 1.20. You can track this here: temporalio/temporal#4017

Thanks for clarifying

If this #21 works for your e2e tests let's then continue there in the thread (and not wait for temporalio/temporal#4017 resolution)