Issue with Docker-based Timesketch Development Environment - Index Not Found Error

[raihalea]

I am currently facing an issue with my Docker-based Timesketch development environment. The system successfully starts, and I am able to upload Plaso files (and CSV files as well) without any problems. However, after the file upload, I continuously encounter the following error messages in the logs when running docker compose logs:

[2024-10-15 20:04:48 +0000] [104] [DEBUG] GET /api/v1/sketches/1/timelines/1/
[2024-10-15 20:04:48,343] timesketch.opensearch/ERROR Unable to count due to an index not found: 25a80a163db248d48647f59683d7511f
[2024-10-15 20:04:53 +0000] [104] [DEBUG] GET /api/v1/sketches/1/timelines/1/
[2024-10-15 20:04:53,046] timesketch.opensearch/ERROR Unable to count due to an index not found: 25a80a163db248d48647f59683d7511f
[2024-10-15 20:04:58 +0000] [104] [DEBUG] GET /api/v1/sketches/1/timelines/1/
[2024-10-15 20:04:58,345] timesketch.opensearch/ERROR Unable to count due to an index not found: 25a80a163db248d48647f59683d7511f

It seems that the system is trying to access an index that does not exist in OpenSearch, causing these errors. Could you please advise on what might be causing this issue and how to resolve it to get the environment functioning correctly?

[jkppr]

Hi @raihalea , let's see if we can get to the bottom of it.

1. Is this with a fresh Timesketch deployment?
2. What release are you deploying?
3. What server/platform are you using?
4. Are you deploying it as a release or dev deployment?
5. Do you use opensearch via the default docker container or a separate instance?
6. Are Timesketch and Opensearch running on the same device or separate servers?
7. Does the timeline ever finish processing? Or is this an infinite issue?
8. Can you share an example file that triggers this behaviour for you?
9. How do you upload the data? (WebUI, dftimewolf, API client, importer client, cli)
10. Do you trigger any analyzers with the upload?

[raihalea]

Hi @jkppr, Thank you for your response.

To answer your questions:

• Yes, this is a fresh Timesketch deployment. I am using it without any modifications from the GitHub master branch.
• I’m attempting a dev deployment on my local machine (x86_64/Windows/WSL). I’ve made sure to allocate sufficient memory and CPU resources for the deployment. I've also confirmed the same issue on another physically different machine with the same setup.
• I'm using the default Docker container for OpenSearch from the Timesketch Docker Compose file without any changes. So, Timesketch and OpenSearch are running on the same instance.
• This is a recurring issue, and Timesketch does not finish processing. (You can refer to the image I attached in the initial discussion.)
• I’ve attached two sample files that trigger the issue. The Plaso file is created using a publicly available dataset, and the CSV file is a simple test sample I created. Both are processed successfully on the released version (20241009), but not on the dev deployment.
  timesketch.csv
  evidences.zip
• As for analyzers, I believe none are triggered during the upload, though I am not entirely sure.

[jkppr]

As a side note: We do not test/support WSL. But that said, I don't see why it should not work.

I just tested your provided files on a fresh dev deployment running on a Ubuntu server and I did not encounter any issues. Both datasets import without issues (tested via the webUI) and I get no index error in the logs. So this seems not to be a general issue and specific to your deployment.

Some more questions and tests to run to understand the issue:

• Just to be on the safe side, can you test if this also happens with a Ubuntu server (e.g. any VPS or VM with sufficient resources) for you?
• How do you upload the data? (WebUI, dftimewolf, API client, importer client, cli)
• What messages do you get for the import in your celery logs?

In general the error indicates a connection problem to the Opensearch instance. Checks you can try to debug this:

• Check your opensearch logs for any errors.
• Shell into your timesketch-dev container and try to verify the connection.
  • From inside the Timesketch container, try to ping the OpenSearch host using ping opensearch
  • Attempt to query the index directly: curl -XGET "http://opensearch:9200/_cat/indices"

[raihalea]

After investigating, I found the cause of the issue. I had forgotten to run ./tsdev.sh celery.

Thank you for your help!