Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review the analysis of Wazuh API usage #5233

Closed
12 tasks done
Tracked by #4155
Desvelao opened this issue Mar 1, 2023 · 6 comments
Closed
12 tasks done
Tracked by #4155

Review the analysis of Wazuh API usage #5233

Desvelao opened this issue Mar 1, 2023 · 6 comments
Assignees

Comments

@Desvelao
Copy link
Member

Desvelao commented Mar 1, 2023

Description

Our friends from @wazuh/framework team performed an analysis of our usage of the Wazuh API in the plugin with the intention to enhance and reduce the requests we do, also reducing their load.

Their analysis has been done in this issue: wazuh/wazuh#10755

The analysis already has some time, so some of the requests could have changed or be removed.

Tasks

In this Epic, we'll need to:

  • Check if the mentioned requests in the analysis still exist in the most recent plugin.
  • Apply improvements: remove duplicated requests or improve them.

Issues

@Desvelao Desvelao added the type/bug Bug issue label Mar 1, 2023
@Desvelao Desvelao self-assigned this Mar 1, 2023
@Desvelao
Copy link
Member Author

Desvelao commented Mar 1, 2023

Comparation against the current 4.5

Agents

Working with agents panel and all related operations.

Status: Done

Agent dashboard

image

Unnecessary queries while accessing this dashboard

2021/11/08 18:37:37 INFO: wazuh-wui 127.0.0.1 "GET //" with parameters {} and body {} done in 0.002s: 308
2021/11/08 18:37:37 INFO: wazuh-wui 127.0.0.1 "GET //" with parameters {} and body {} done in 0.001s: 308
2021/11/08 18:37:37 INFO: wazuh-wui 127.0.0.1 "GET /" with parameters {} and body {} done in 0.005s: 200
2021/11/08 18:37:37 INFO: wazuh-wui 127.0.0.1 "GET /" with parameters {} and body {} done in 0.005s: 200
  • Improvements and suggestions
    1. What's the purpose of these queries? they should be removed.

I could not replicate these requests, but it could be related to getting the version of Wazuh manager.

Multiple queries to list agents

2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"offset": "0", "limit": "15", "sort": "+id", "q": "id!=000"} and body {} done in 0.019s: 200
2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {} and body {} done in 0.011s: 200
2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"limit": "1", "sort": "-dateAdd", "q": "id!=000"} and body {} done in 0.018s: 200
2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"q": "status=active"} and body {} done in 0.015s: 200
  • Improvements and suggestions
    1. One GET /agents query. Then parse specific and needed information like the last registered agent.
    2. No need to sort agent by ID.
Limit Test 1 Test 2 Test 3 AVG
15 0.08046s 0.09946s 0.09922s 0.09304s
1500 0.17242s 0.16282s 0.17384s 0.16969s
15000 1.24544s 1.04256s 1.04288s 1.11029s

NOTE: these values should be considered. Choose between one GET /agents request and parse all the data later on or query depending on the current page.

2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"offset": "0", "limit": "15", "sort": "+id", "q": "id!=000"} and body {} done in 0.019s: 200

This was changed to use the selection of fields that will be displayed in the agent's table. This request is used to build the agents' table.

2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {} and body {} done in 0.011s: 200

This was removed.

2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"limit": "1", "sort": "-dateAdd", "q": "id!=000"} and body {} done in 0.018s: 200

Used to get the last registered agent.

2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"q": "status=active"} and body {} done in 0.015s: 200

I don't know about this, I guess it could be caused by a search using the search bar.

Redundant information: number of agents for each status type

2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents/summary/status" with parameters {} and body {} done in 0.006s: 200
2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents/summary/status" with parameters {} and body {} done in 0.020s: 200
  • Improvements and suggestions
    1. The information obtained with this query can be replaced with the same information retrieved and parsed from GET /agents.
    2. There's no need to do multiple GET /agents/summary/status queries.
2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents/summary/status" with parameters {} and body {} done in 0.006s: 200

The request is only done one time and this doesn't seem to be repeated.

For another hand, to get this data using the GET /agents endpoint and managing the data in the frontend, would force us to get the data of all agents when the related request only gets the data to be displayed on a page of the agent's table. I don't think that approach is better than the current implementation using the GET /agents/summary/status endpoint.

Redundant information: group names and agent OS

2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /groups" with parameters {} and body {} done in 0.013s: 200
2021/11/08 18:38:37 INFO: wazuh-wui 127.0.0.1 "GET /agents/summary/os" with parameters {} and body {} done in 0.009s: 200
  • Improvements and suggestions
    1. The information obtained with this query can be replaced with the same information retrieved and parsed from GET /agents.

I am not sure where these requests are done.

status and details panel behavior

Switching agent status generates a new query

image

2021/11/08 18:55:29 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"q": "id!=000;(status=active)", "offset": "0", "limit": "15", "sort": "+id"} and body {} done in 0.007s: 200
2021/11/08 18:55:34 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"q": "id!=000;(status=disconnected)", "offset": "0", "limit": "15", "sort": "+id"} and body {} done in 0.007s: 200
2021/11/08 18:55:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"q": "id!=000;(status=never_connected)", "offset": "0", "limit": "15", "sort": "+id"} and body {} done in 0.009s: 200
  • Improvements and suggestions
    1. This information could be obtained from the last full agent list retrieved.
    2. In my opinion this behavior should depend on Refresh button only.
    3. There's an issue with the details panel: one agent is disconnected, the table is refreshed by status panel but details is not changing. It's because it needs GET /agents/summary/status.
      image
2021/11/08 18:55:29 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"q": "id!=000;(status=active)", "offset":  "0", "limit": "15", "sort": "+id"} and body {} done in 0.007s: 200

This request is used by the agents' table when using the status filter. I don't think the solution is using the full agent list retrieved previously.

There's an issue with the details panel: one agent is disconnected, the table is refreshed by status panel but details is not changing. It's because it needs GET /agents/summary/status.

You are right, the Details panel uses the data obtained when the Agents page is loaded or when using the Refresh button. If meanwhile, some agent changes its status, the data displayed in the agent's table and the Details panel could not match. I am not sure if we should consider this a bug, we should discuss it.

Agent summary

Redundant information while accessing this dashboard

image

2021/11/09 15:08:26 INFO: wazuh-wui 127.0.0.1 "GET /groups" with parameters {} and body {} done in 0.012s: 200
  • Improvements and suggestions
    1. The purpose of this request should be reviewed.
    2. If group information is needed, It can be retrieved from "GET /agents" with parameters {"agents_list": "002"}.

The request is done in a route controller. I am not sure what is the usage of the retrieved data. We should do a deep analysis of its usage.

I saw a request to GET / done when accessing to agent info/welcome that we should review.

Unnecessary queries while accessing agent operations

image

2021/11/09 14:08:26 INFO: wazuh-wui 127.0.0.1 "GET //" with parameters {} and body {} done in 0.001s: 308
2021/11/09 14:08:26 INFO: wazuh-wui 127.0.0.1 "GET /" with parameters {} and body {} done in 0.005s: 200
2021/11/09 14:22:52 INFO: wazuh-wui 127.0.0.1 "GET /agents/summary/status" with parameters {} and body {} done in 0.010s: 200
2021/11/09 14:22:52 INFO: wazuh-wui 127.0.0.1 "GET /agents/summary/status" with parameters {} and body {} done in 0.006s: 200
  • Improvements and suggestions
    1. What's the purpose of these queries? they should be removed.

I found these requests when accessing to Integrity monitoring module from the agent info/welcome page. I think we should review these requests.

Agent integrity monitoring

GET /syscheck request for each inventory page

image

2021/11/09 16:17:10 INFO: wazuh-wui 127.0.0.1 "GET /syscheck/002" with parameters {"offset": "0", "limit": "15", "sort": "+file", "type": "file"} and body {} done in 0.012s: 200
2021/11/09 16:17:10 INFO: wazuh-wui 127.0.0.1 "GET /syscheck/002" with parameters {"offset": "15", "limit": "15", "sort": "+file", "type": "file"} and body {} done in 0.012s: 200
2021/11/09 16:17:10 INFO: wazuh-wui 127.0.0.1 "GET /syscheck/002" with parameters {"offset": "30", "limit": "15", "sort": "+file", "type": "file"} and body {} done in 0.017s: 200
  • Improvements and suggestions
    1. This situation should be reviewed. It could be just one query with no limit parameter and be parsed later on depending on the current page (benchmark detailed information panel at Security configuration assessment is currently performing that way).
    2. NOTE: this will possibly need a Refresh button to be added.

These requests seem to get the data of each table page of the file integrity monitoring agent inventory. I don't see any problem with the current behavior.

Agent stats

Unnecessary queries while accessing

image

2021/11/09 17:05:57 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "002", "select": "status"} and body {} done in 0.007s: 200
  • Improvements and suggestions
    1. Is it really necessary? It could be obtained from an older GET /agents request.
    2. NOTE: a Refresh button could be useful here.

This request needs a deep analysis. It seems to be related when changing the module. It is defined in the switchSubTab of the agents.js controller.

Agent inventory data

image

Duplicated queries while accessing

2021/11/09 17:30:51 INFO: wazuh-wui 127.0.0.1 "GET /syscollector/002/hardware" with parameters {} and body {} done in 0.014s: 200
2021/11/09 17:30:51 INFO: wazuh-wui 127.0.0.1 "GET /syscollector/002/hardware" with parameters {} and body {} done in 0.024s: 200
2021/11/09 17:30:51 INFO: wazuh-wui 127.0.0.1 "GET /syscollector/002/os" with parameters {} and body {} done in 0.026s: 200
2021/11/09 17:30:51 INFO: wazuh-wui 127.0.0.1 "GET /syscollector/002/os" with parameters {} and body {} done in 0.012s: 200
  • Improvements and suggestions
    1. Review this situation and use one request.

I didn't get the repeated requests. Maybe this could be fixed in the later release of the analysis date.

Unnecessary query while accessing

2021/11/09 17:33:01 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "002", "select": "status"} and body {} done in 0.007s: 200
  • Improvements and suggestions
    1. Is it really necessary? It could be obtained from an older GET /agents request.

This request needs a deep analysis. It seems to be related when changing the module. It is defined in the switchSubTab of the agents.js controller.

Packages: a request depending on current page number

2021/11/09 17:36:03 INFO: wazuh-wui 127.0.0.1 "GET /syscollector/002/packages" with parameters {"limit": "10", "offset": "0"} and body {} done in 0.012s: 200
2021/11/09 17:36:04 INFO: wazuh-wui 127.0.0.1 "GET /syscollector/002/packages" with parameters {"limit": "10", "offset": "10"} and body {} done in 0.009s: 200
2021/11/09 17:36:05 INFO: wazuh-wui 127.0.0.1 "GET /syscollector/002/packages" with parameters {"limit": "10", "offset": "20"} and body {} done in 0.012s: 200
  • Improvements and suggestions
    1. The purpose of these requests should be reviewed. As explained before, doing one request and parsing the output should suffice.

These requests seem to get the data of each table page. I don't see any problem with the current behavior.

Agent configuration

image

Unnecessary queries while accessing this dashboard

2021/11/09 19:06:19 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "001", "select": "status"} and body {} done in 0.010s: 200
  • Improvements and suggestions
    1. Is it really necessary? It could be obtained from an older GET /agents request.

This request needs a deep analysis. It seems to be related when changing the module. It is defined in the switchSubTab of the agents.js controller.

Duplicated queries while accessing this dashboard

2021/11/09 19:06:19 INFO: wazuh-wui 127.0.0.1 "GET /agents/001/group/is_sync" with parameters {} and body {} done in 0.009s: 200
2021/11/09 19:06:19 INFO: wazuh-wui 127.0.0.1 "GET /agents/001/group/is_sync" with parameters {} and body {} done in 0.011s: 200
  • Improvements and suggestions
    1. Review this situation and use one request.

No, apply. Now the request is done to GET /agents?q=id={agent_id}&select=group_config_status

Standby queries

Duplicated queries

2021/11/10 16:15:00 INFO: wazuh-wui 127.0.0.1 "GET /cluster/nodes" with parameters {"select": "name"} and body {} done in 0.027s: 200
2021/11/10 16:15:00 INFO: wazuh-wui 127.0.0.1 "GET /cluster/nodes" with parameters {"select": "name"} and body {} done in 0.023s: 200
  • Improvements and suggestions
    1. Review this situation and use one request.

I could not replicate this case.

Bad parameter usage

2021/11/10 16:15:00 INFO: wazuh-wui 127.0.0.1 "GET /cluster/wd_master_1/stats/remoted" with parameters {"pretty": ""} and body {} done in 0.015s: 200
2021/11/10 16:15:00 INFO: wazuh-wui 127.0.0.1 "GET /cluster/wd_master_1/stats/analysisd" with parameters {"pretty": ""} and body {} done in 0.021s: 200
  • Improvements and suggestions
    1. pretty parameter is not correctly used and should be removed.

I am not sure where this request is done using the pretty parameter. There is some logic in the API Console, but it should remove the pretty parameter.

Possible unnecessary queries

2021/11/10 16:15:00 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"offset": "0", "limit": "1", "q": "id!=000"} and body {} done in 0.010s: 200
2021/11/10 16:15:00 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"offset": "0", "limit": "500", "q": "id!=000"} and body {} done in 0.007s: 200
  • Improvements and suggestions
    1. TBR: I'm not entirely sure what's the purpose of these queries nor the limit: 500 parameter.

I am not sure where is request is done.

@Desvelao
Copy link
Member Author

Desvelao commented Mar 1, 2023

Comparation against the current 4.5

Management

Reviewing the management section: administration items and status and reports

Status: Done

Groups

When adding a new group

image

Issue
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "POST /groups" with parameters {} and body {"group_id": "group2"} done in 0.011s: 200
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "GET /groups" with parameters {"offset": "0", "limit": "10", "sort": "+name"} and body {} done in 0.058s: 200
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "GET /groups" with parameters {"offset": "0", "limit": "10", "sort": "+name"} and body {} done in 0.048s: 200
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "GET /groups" with parameters {"offset": "0", "limit": "10", "sort": "+name"} and body {} done in 0.036s: 200
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "GET /groups" with parameters {"offset": "0", "limit": "10", "sort": "+name"} and body {} done in 0.024s: 200
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "GET /groups" with parameters {"offset": "0", "limit": "10", "sort": "+name"} and body {} done in 0.067s: 200
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "GET /groups" with parameters {"offset": "0", "limit": "10", "sort": "+name"} and body {} done in 0.053s: 200
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "GET /groups" with parameters {"offset": "0", "limit": "10", "sort": "+name"} and body {} done in 0.039s: 200
2021/11/09 09:04:19 INFO: wazuh-wui 172.19.0.105 "GET /groups" with parameters {"offset": "0", "limit": "10", "sort": "+name"} and body {} done in 0.048s: 200
  • 8 requests to GET /groups are done, why? This also happens when refreshing, 6 requests in this case.

I confirm this is happening, we should review it.

Configuration

When clicking on the configuration section

image

Issue
2021/11/08 15:04:35 INFO: wazuh-wui 172.22.0.12 "GET /cluster/status" with parameters {} and body {} done in 0.029s: 200
2021/11/08 15:04:35 INFO: wazuh-wui 172.22.0.12 "GET /cluster/status" with parameters {} and body {} done in 0.028s: 200
2021/11/08 15:04:35 INFO: wazuh-wui 172.22.0.12 "GET /cluster/nodes" with parameters {} and body {} done in 0.054s: 200
2021/11/08 15:04:35 INFO: wazuh-wui 172.22.0.12 "GET /cluster/nodes" with parameters {} and body {} done in 0.016s: 200
2021/11/09 12:21:01 INFO: wazuh-wui 172.19.0.105 "GET /agents" with parameters {"q": "id=000"} and body {} done in 0.014s: 200
  • Improvements and suggestions

    1. Don't use a query to get only one agent. The following will be always faster:
      GET /agents?agents_list=000

    2. The requests used to check cluster health are done twice, could we avoid the second request to each endpoint?

  1. We should change the request
  2. This needs a deep analysis.

When choosing some configurations (global for instance)

image

Issue
2021/11/08 15:13:43 INFO: wazuh-wui 172.22.0.12 "GET /cluster/master-node/configuration/analysis/global" with parameters {} and body {} done in 0.017s: 200
2021/11/08 15:13:44 ERROR: Error connecting with socket
Traceback (most recent call last):
 File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/core/wazuh_socket.py", line 27, in _connect
   self.s.connect(self.path)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/core/configuration.py", line 780, in get_active_configuration
   s = WazuhSocket(dest_socket)
 File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/core/wazuh_socket.py", line 22, in __init__
   self._connect()
 File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/core/wazuh_socket.py", line 29, in _connect
   raise WazuhException(1013, str(e))
wazuh.core.exception.WazuhException: Error 1013 - Unable to connect with socket: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/core/cluster/dapi/dapi.py", line 256, in execute_local_request
    data = await asyncio.wait_for(task, timeout=timeout)
  File "/var/ossec/framework/python/lib/python3.9/asyncio/tasks.py", line 481, in wait_for
    return fut.result()
  File "/var/ossec/framework/python/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/core/cluster/dapi/dapi.py", line 230, in run_local
    data = self.f(**self.f_kwargs)
  File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/rbac/decorators.py", line 406, in wrapper
    result = func(*args, **kwargs) if not skip_execution else None
  File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/manager.py", line 222, in get_config
    data = configuration.get_active_configuration(agent_id='000', component=component, configuration=config)
  File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.2.4-py3.9.egg/wazuh/core/configuration.py", line 782, in get_active_configuration
    raise WazuhInternalError(1121)
wazuh.core.exception.WazuhInternalError: Error 1121 - Error connecting with socket
2021/11/08 15:13:44 INFO: wazuh-wui 172.22.0.12 "GET /cluster/master-node/configuration/mail/global" with parameters {} and body {} done in 0.019s: 500
2021/11/08 15:13:44 INFO: wazuh-wui 172.22.0.12 "GET /security/user/authenticate" with parameters {} and body {} done in 0.171s: 200
2021/11/08 15:13:44 INFO: wazuh-wui 172.22.0.12 "GET /manager/info" with parameters {} and body {} done in 0.044s: 200
2021/11/08 15:13:44 INFO: wazuh-wui 172.22.0.12 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.018s: 200
2021/11/08 15:13:44 INFO: wazuh-wui 172.22.0.12 "GET /cluster/status" with parameters {} and body {} done in 0.020s: 200
2021/11/08 15:13:44 INFO: wazuh-wui 172.22.0.12 "GET /cluster/local/info" with parameters {} and body {} done in 0.017s: 200
2021/11/08 15:13:44 INFO: wazuh-wui 172.22.0.12 "GET /cluster/master-node/configuration/request/remote" with parameters {} and body {} done in 0.016s: 200
2021/11/08 15:13:44 INFO: wazuh-wui 172.22.0.12 "GET /cluster/master-node/configuration/com/logging" with parameters {} and body {} done in 0.015s: 200

This needs a deep analysis.

Status and reports

When clicking on the cluster section

image

Issue
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET /cluster/nodes" with parameters {} and body {} done in 0.012s: 200
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET /cluster/status" with parameters {} and body {} done in 0.015s: 200
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET /cluster/status" with parameters {} and body {} done in 0.015s: 200
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET //" with parameters {} and body {} done in 0.010s: 308
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET /agents" with parameters {} and body {"limit": 1} done in 0.054s: 200
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET /" with parameters {} and body {} done in 0.005s: 200
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET /cluster/local/config" with parameters {} and body {} done in 0.079s: 200
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET /cluster/nodes" with parameters {} and body {} done in 0.089s: 200
2021/11/09 08:05:04 INFO: wazuh-wui 172.19.0.105 "GET /cluster/healthcheck" with parameters {} and body {} done in 0.045s: 200
  • Improvements and suggestions

    1. GET /agents request with limit=1 is used to take the total agents that is shown in the information section. Can this request be changed to GET /agents/summary/status?
      A test has been performed to see if the second endpoint mentioned is faster. This test has been done in a cluster with 3 managers and 1000 agents.
      Suggestion: use GET /agents with limit=1 to take the total of agents instead of GET /agents/summary/status. In this case, the appropriate endpoint is used.
Endpoint Test 1 Test 2 Test 3 Test 4 AVG
GET /agents, limit=1 0.0051 s 0.00301 s 0.00317 s 0.0032 s 0.00362 s
GET /agents/summary/status 0.0199 s 0.01869 s 0.0194 s 0.0189 s 0.0192225 s
  1. I understand that doesn't apply any change.

When clicking on the status section

image

Issue
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /cluster/nodes" with parameters {} and body {} done in 0.017s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /agents/summary/status" with parameters {} and body {} done in 0.027s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /cluster/status" with parameters {} and body {} done in 0.024s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /cluster/status" with parameters {} and body {} done in 0.018s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /manager/info" with parameters {} and body {} done in 0.014s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /overview/agents" with parameters {} and body {} done in 0.043s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /cluster/nodes" with parameters {} and body {} done in 0.014s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /cluster/master-node/status" with parameters {} and body {} done in 0.015s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /cluster/master-node/info" with parameters {} and body {} done in 0.015s: 200
2021/11/09 15:06:27 INFO: wazuh-wui 172.19.0.105 "GET /agents" with parameters {"limit": "1", "sort": "-dateAdd", "q": "id!=000"} and body {} done in 0.020s: 200
  • Improvements and suggestions

    1. GET /overview/agents and GET /agents/summary/status: why is GET /agents/summary/status used? It is not needed as the information obtained in its response is available in the GET /overview/agents response.
    2. "GET /agents" with parameters {"limit": "1", "sort": "-dateAdd", "q": "id!=000"}: this request is used to get the last registered agent. It is not necessary as this information can be obtained from the GET /overview/agents response.
    3. GET /manager/info and GET /cluster/master-node/status + GET /cluster/master-node/info: GET /cluster/node_id/status is needed as it is used to take the running daemons. GET /manager/info is redundant as the node information is obtained in the GET /cluster/node_id/info response.
  1. We could use the data provided by GET /overview/agents request to get the agent status data and could remove the GET /agents/summary/status.
  2. We could use the data provided by GET /overview/agents request to get the last registered agent data and could remove the GET /agents?limit=1&sort=-dateAdd.
  3. We should do a deep analysis.

When changing to another node to see its status, the following logs appear:

image

Issue
2021/11/09 15:35:12 INFO: wazuh-wui 172.19.0.105 "GET /agents/summary/status" with parameters {} and body {} done in 0.022s: 200
2021/11/09 15:35:12 INFO: wazuh-wui 172.19.0.105 "GET /overview/agents" with parameters {} and body {} done in 0.041s: 200
2021/11/09 15:35:12 INFO: wazuh-wui 172.19.0.105 "GET /cluster/worker1/status" with parameters {} and body {} done in 0.040s: 200
2021/11/09 15:35:12 INFO: wazuh-wui 172.19.0.105 "GET /cluster/worker1/info" with parameters {} and body {} done in 0.030s: 200
2021/11/09 15:35:12 INFO: wazuh-wui 172.19.0.105 "GET /agents" with parameters {"limit": "1", "sort": "-dateAdd", "q": "id!=000"} and body {} done in 0.020s: 200
  • Improvements and suggestions

    1. Number 1 explained above.
    2. Number 2 explained above.
  1. Apply the suggestion.
  2. Apply the suggestion.

When clicking on the logs section

image

Issue
2021/11/09 15:45:52 INFO: wazuh-wui 172.19.0.105 "GET /cluster/status" with parameters {} and body {} done in 0.022s: 200
2021/11/09 15:45:52 INFO: wazuh-wui 172.19.0.105 "GET /cluster/nodes" with parameters {} and body {} done in 0.040s: 200
2021/11/09 15:45:52 INFO: wazuh-wui 172.19.0.105 "GET /cluster/status" with parameters {} and body {} done in 0.016s: 200
2021/11/09 15:45:52 INFO: wazuh-wui 172.19.0.105 "GET /cluster/nodes" with parameters {} and body {} done in 0.015s: 200
2021/11/09 15:45:52 INFO: wazuh-wui 172.19.0.105 "GET /cluster/master-node/logs/summary" with parameters {} and body {} done in 0.052s: 200
2021/11/09 15:45:52 INFO: wazuh-wui 172.19.0.105 "GET /cluster/master-node/logs" with parameters {"limit": "100", "sort": "-timestamp"} and body {} done in 0.053s: 200
2021/11/09 15:45:52 INFO: wazuh-wui 172.19.0.105 "GET /cluster/master-node/logs" with parameters {"limit": "100", "sort": "-timestamp"} and body {} done in 0.053s: 200
  • Improvements and suggestions

    1. GET /cluster/master-node/logs is called twice. Remove one of them if it is not necessary.
  1. I confirm this. We should review it.

@Desvelao
Copy link
Member Author

Desvelao commented Mar 1, 2023

Comparation against the current 4.5

Modules

Reviewing the modules section.

Status: Done

When clicking on modules

image

Issue
2021/11/09 15:59:13 INFO: wazuh-wui 172.19.0.105 "GET /agents/summary/status" with parameters {} and body {} done in 0.018s: 200
2021/11/09 15:59:13 INFO: wazuh-wui 172.19.0.105 "GET /agents/summary/status" with parameters {} and body {} done in 0.013s: 200
  • Improvements and suggestions

    1. GET /agents/summary/status is called twice. Remove one of them if it is not necessary.

I could not replicate it.

All modules

When clicking on explore agent.

image

Issue
2021/11/09 11:54:01 INFO: wazuh-wui 172.19.0.105 "GET /agents" with parameters {"offset": "0", "limit": "10", "sort": "+id", "q": "id!=000"} and body {} done in 0.023s: 200
2021/11/09 11:54:04 INFO: wazuh-wui 172.19.0.105 "GET /agents" with parameters {"offset": "30", "limit": "40", "sort": "+id", "q": "id!=000"} and body {} done in 0.031s: 200
  • Improvements and suggestions

    1. If only 10 rows per page are going to be shown, limit must be 10 instead of offset + 10.
      GET /agents?offset=40&limit=10
  1. I confirm this. It could be happening in the agents-selection-table.js component.

Security configuration assessment

When clicking on an agent's policy.

image

Issue
2021/11/11 09:41:10 INFO: wazuh-wui 172.18.0.17 "GET /sca/133" with parameters {"q": "policy_id=cis_debian10"} and body {} done in 0.036s: 200
2021/11/11 09:41:12 INFO: wazuh-wui 172.18.0.17 "GET /sca/133/checks/cis_debian10" with parameters {} and body {} done in 1.699s: 200
  • Improvements and suggestions

    1. GET /sca/133/checks/cis_debian10 does not include limit and offset. Have we taken into account the possibility to use them and have we studied if more than a single request with limit and offset is faster than this? This is what we are doing with other items such as rules, decoders, agents, etc, but note that SCA requests are not the same as those ones.

No apply. This was solved in a previous change.

@Desvelao
Copy link
Member Author

Desvelao commented Mar 1, 2023

Comparation against the current 4.5

Security

Reviewing the security section.

Status: TBR

Users

image

Issue
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /security/roles" with parameters {"sort": "name"} and body {} done in 0.097s: 200
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /security/users" with parameters {"sort": "username"} and body {} done in 0.190s: 200
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /manager/info" with parameters {} and body {} done in 0.100s: 200
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /security/roles" with parameters {"sort": "name"} and body {} done in 0.101s: 200
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /security/users" with parameters {"sort": "username"} and body {} done in 0.128s: 200
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.018s: 200
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /cluster/status" with parameters {} and body {} done in 0.015s: 200
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /security/users/me" with parameters {} and body {} done in 0.105s: 200
2021/11/10 08:06:31 INFO: wazuh-wui 172.18.0.17 "GET /cluster/local/info" with parameters {} and body {} done in 0.015s: 200
  • Improvements and suggestions

    1. I suppose that the GET /security/roles request is used to take the role names using the role ids obtained in each user item from the GET /security/users response. We could use GET /security/roles with the role_ids parameter to show only roles corresponding to the users obtained. Also, the request is repeated.
      GET /security/roles?role_ids=2,3
    2. Repeated request to GET /security/users. Are both of them needed?
    3. TBR: GET /manager/info, GET /agents?agent_list=000 redundant??
    4. TBR: why is GET /cluster/local/info needed?
    5. TBR: why is GET /security/users/me needed?
  1. If we get roles that have the users, then it should be done when paging because the needed roles could change. So the current approach makes sense if we expect the user to go to another tab. For another hand, the request to get the roles is not paging, so this could cause problems when there are more than the default limit (500).
  2. Duplicated requests of GET /security/users and GET /security/roles when going from Security/<any tab> to Security/Users.
  3. I could not replicate it.
  4. I could not replicate it.
  5. I could not replicate it.
  6. I could not replicate it.

❗ The create and edit user has the same problem with the limit of the role.

Roles

image

Issue
2021/11/10 08:39:17 INFO: wazuh-wui 172.18.0.17 "GET /security/roles" with parameters {} and body {} done in 0.095s: 200
2021/11/10 08:39:17 INFO: wazuh-wui 172.18.0.17 "GET /security/roles" with parameters {} and body {} done in 0.118s: 200
2021/11/10 08:39:17 INFO: wazuh-wui 172.18.0.17 "GET /manager/info" with parameters {} and body {} done in 0.081s: 200
2021/11/10 08:39:17 INFO: wazuh-wui 172.18.0.17 "GET /security/policies" with parameters {} and body {} done in 0.140s: 200
2021/11/10 08:39:17 INFO: wazuh-wui 172.18.0.17 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.135s: 200
2021/11/10 08:39:17 INFO: wazuh-wui 172.18.0.17 "GET /security/policies" with parameters {} and body {} done in 0.124s: 200
2021/11/10 08:39:17 INFO: wazuh-wui 172.18.0.17 "GET /cluster/status" with parameters {} and body {} done in 0.120s: 200
2021/11/10 08:39:18 INFO: wazuh-wui 172.18.0.17 "GET /security/users/me" with parameters {} and body {} done in 0.117s: 200
2021/11/10 08:39:18 INFO: wazuh-wui 172.18.0.17 "GET /cluster/local/info" with parameters {} and body {} done in 0.015s: 200
  • Improvements and suggestions

    1. Repeated request to GET /security/roles. Are both of them needed?
    2. I suppose that the GET /security/policies requests are used to show actions, resources and effect of a policy (and the policy names) when placing the mouse cursor on the policy names. In that case, the request is needed, but do we really need to do it twice?
      We could also do the same that was explained above, we could make a request to GET /security/policies with the policy_ids parameter to show only policies corresponding to the roles, in this case.
      GET /security/policies?policy_ids=1,2,3
    3. TBR: GET /manager/info, GET /agents?agent_list=000 redundant??
    4. TBR: why is GET /cluster/local/info needed?
    5. TBR: why is GET /security/users/me needed?
  1. Duplicated requests of GET /security/roles and GET /security/policies when going from Security/<any tab> to Security/Roles.
  2. If we get policies that have the roles, then it should be done when paging because the needed roles could change. So the current approach makes sense if we expect the user to go to another tab. For another hand, the request to get the policies is not paging, so this could cause problems when there are more than the default limit (500).
  3. I could not replicate it.
  4. I could not replicate it.
  5. I could not replicate it.
  6. I could not replicate it.

Policies

image

Issue
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /security/resources" with parameters {} and body {} done in 0.112s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /security/policies" with parameters {} and body {} done in 0.128s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /security/policies" with parameters {} and body {} done in 0.147s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /security/resources" with parameters {} and body {} done in 0.014s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /manager/info" with parameters {} and body {} done in 0.149s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /security/actions" with parameters {} and body {} done in 0.012s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /security/actions" with parameters {} and body {} done in 0.011s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.025s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /cluster/status" with parameters {} and body {} done in 0.017s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /security/users/me" with parameters {} and body {} done in 0.101s: 200
2021/11/10 08:56:58 INFO: wazuh-wui 172.18.0.17 "GET /cluster/local/info" with parameters {} and body {} done in 0.014s: 200
  • Improvements and suggestions

    1. Repeated request to GET /security/policies. Are both of them needed?
    2. The 2 requests to GET /security/resources may not be needed since the actions, resources and effect of a policy are obtained in the GET /security/policies request.
    3. Same as above but for endpoint GET /security/actions.
    4. TBR: GET /manager/info, GET /agents?agent_list=000 redundant??
    5. TBR: why is GET /cluster/local/info needed?
    6. TBR: why is GET /security/users/me needed?
  1. Duplicated requests of GET /security/policies, GET /security/actions, and GET /security/resources when going from Security/<any tab> to Security/Policies.
  2. Do a deep analysis. Some requests and logic could not be used.
  3. I could not replicate it.
  4. I could not replicate it.
  5. I could not replicate it.
  6. I could not replicate it.

When creating a new policy

image

Issue
2021/11/10 09:05:09 INFO: wazuh-wui 172.18.0.17 "GET /security/resources" with parameters {} and body {} done in 0.023s: 200
2021/11/10 09:05:10 INFO: wazuh-wui 172.18.0.17 "GET /security/actions" with parameters {} and body {} done in 0.093s: 200
2021/11/10 09:05:10 INFO: wazuh-wui 172.18.0.17 "GET /security/policies" with parameters {} and body {} done in 0.127s: 200
  • Improvements and suggestions

    1. Why is the request to GET /security/policies needed? The resources and actions are needed in order to show the list of resources and actions to select for the new policy, but there is no need to get all policies.
  1. The request could be removed. It should be required.

Roles mapping

image

Issue
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /security/roles" with parameters {"sort": "name"} and body {} done in 0.103s: 200
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /security/rules" with parameters {"sort": "name"} and body {} done in 0.207s: 200
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /manager/info" with parameters {} and body {} done in 0.123s: 200
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /security/roles" with parameters {"sort": "name"} and body {} done in 0.124s: 200
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /security/rules" with parameters {"sort": "name"} and body {} done in 0.142s: 200
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.020s: 200
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /cluster/status" with parameters {} and body {} done in 0.019s: 200
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /security/users/me" with parameters {} and body {} done in 0.106s: 200
2021/11/10 09:12:15 INFO: wazuh-wui 172.18.0.17 "GET /cluster/local/info" with parameters {} and body {} done in 0.012s: 200
  • Improvements and suggestions

    1. Repeated request to GET /security/roles. Are both of them needed?
    2. Repeated request to GET /security/rules. Are both of them needed?
    3. TBR: GET /manager/info, GET /agents?agent_list=000 redundant??
    4. TBR: why is GET /cluster/local/info needed?
    5. TBR: why is GET /security/users/me needed?
  1. Duplicated requests of GET /security/roles.
  2. Duplicated requests of GET /security/rules.
  3. I could not replicate it.
  4. I could not replicate it.
  5. I could not replicate it.

❗ It seems the Security section could have more problems that were reported. We could extend the analysis.

@Desvelao
Copy link
Member Author

Desvelao commented Mar 1, 2023

Comparation against the current 4.5

Settings

Reviewing the settings section.

Status: TBR

Requests done for all sections in settings

image

Issue
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /security/user/authenticate" with parameters {} and body {} done in 0.171s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /manager/info" with parameters {} and body {} done in 0.041s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.018s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /cluster/status" with parameters {} and body {} done in 0.022s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /security/users/me" with parameters {} and body {} done in 0.101s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /cluster/local/info" with parameters {} and body {} done in 0.013s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /security/user/authenticate" with parameters {} and body {} done in 0.166s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /manager/info" with parameters {} and body {} done in 0.015s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.017s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /cluster/status" with parameters {} and body {} done in 0.021s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /security/users/me" with parameters {} and body {} done in 0.106s: 200
2021/11/10 10:02:10 INFO: wazuh-wui 172.18.0.17 "GET /cluster/local/info" with parameters {} and body {} done in 0.016s: 200
  • Improvements and suggestions

    1. Repeated GET /cluster/local/info. Are both of them needed?
    2. GET /cluster/status repeated. Remove one of them.
    3. TBR: GET /manager/info, GET /agents?agent_list=000 redundant?? Also, both repeated.
    4. TBR: login repeated, why?
  1. I could not replicate it.
  2. I could not replicate it.
  3. I could not replicate it.
  4. If you saw a request to login, this could be caused because the token could have expired.

@Desvelao
Copy link
Member Author

Desvelao commented Mar 1, 2023

Comparation against the current 4.5

Healthcheck

Reviewing the healthcheck done every x time unit.

Status: Done

Issue
2021/11/10 10:25:01 INFO: wazuh-wui 172.18.0.17 "GET /cluster/nodes" with parameters {"select": "name"} and body {} done in 0.027s: 200
2021/11/10 10:25:01 INFO: wazuh-wui 172.18.0.17 "GET /cluster/nodes" with parameters {"select": "name"} and body {} done in 0.017s: 200
2021/11/10 10:25:01 INFO: wazuh-wui 172.18.0.17 "GET /cluster/master-node/stats/remoted" with parameters {"pretty": ""} and body {} done in 0.029s: 200
2021/11/10 10:25:01 INFO: wazuh-wui 172.18.0.17 "GET /cluster/master-node/stats/analysisd" with parameters {"pretty": ""} and body {} done in 0.020s: 200
2021/11/10 10:25:01 INFO: wazuh-wui 172.18.0.17 "GET /cluster/worker2/stats/remoted" with parameters {"pretty": ""} and body {} done in 0.040s: 200
2021/11/10 10:25:01 INFO: wazuh-wui 172.18.0.17 "GET /cluster/worker2/stats/analysisd" with parameters {"pretty": ""} and body {} done in 0.047s: 200
2021/11/10 10:25:01 INFO: wazuh-wui 172.18.0.17 "GET /cluster/worker1/stats/remoted" with parameters {"pretty": ""} and body {} done in 0.033s: 200
2021/11/10 10:25:01 INFO: wazuh-wui 172.18.0.17 "GET /cluster/worker1/stats/analysisd" with parameters {"pretty": ""} and body {} done in 0.030s: 200
  • Improvements and suggestions

    1. Repeated GET /cluster/nodes. Are both of them needed?
    2. The pretty parameter is not being used properly in the requests to GET /cluster/{node_id}/remoted and GET /cluster/{node_id}/analysisd endpoints. Remove the parameter as it is not being used.
  1. I could not replicate it.
  2. I did a fast search and the pretty parameter seems to not be related to the request of frontend. They could be caused the cron jobs to retrieve data of statistics. See server/start/cron-scheduler/predefined-jobs.ts to remove the usage of pretty.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Status: Done
Development

No branches or pull requests

5 participants