Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

install.py can create .env files 0:0 ownership instead of unprivileged user ownership #253

Closed
e101sg opened this issue May 14, 2023 · 10 comments
Closed

install.py can create .env files 0:0 ownership instead of unprivileged user ownership #253

e101sg opened this issue May 14, 2023 · 10 comments

Comments

@e101sg
Copy link

e101sg commented May 14, 2023

Got the installation issue on Ubuntu while installing v23.05.0

After running the auth_setup,docker pull, get an issue when running ./scripts/start. The issue is permission error.
PermissionError: [Error 1] Operation not permitted : 'pcap-capture.env '
I have tried to build from scratch as well..got some Logstash related error at the end (almost) of the build. Not sure, this issue related to new release (23.05.0). Any thoughts highly useful.. (spend the whole weekend for this) ~ Cheers!

Screenshot from 2023-05-14 22-13-36
@mmguero
Copy link
Collaborator

mmguero commented May 15, 2023

Hmmm, looks like somehow there's a scenario where part of the install.py portion which runs as root populated the .env files which ended up in the file ownership not being set correctly for non-root users. I'll look into and see if I can replicate.

In the meantime, doing something like this should solve the problem:

$ cd ~/Documents/Malcolm
$ ./scripts/wipe
$ sudo chown -R user:user ~/Documents/Malcolm
$ ./scripts/wipe
$ ./scripts/start
  1. cd into the Malcolm directory
  2. stop/clean everything out with the wipe command
  3. change ownership of that directory to your user account
  4. clean everything out again after the chown command just for good measure
  5. start malcolm back up again

mmguero added a commit to mmguero-dev/Malcolm that referenced this issue May 15, 2023
@mmguero
Fix file permissions issue on .env files (see cisagov#253)
c3bb897
@e101sg
Copy link
Author

e101sg commented May 15, 2023 •
edited
Loading

yes, this solves the permission issue and Malcolm stated. However, none of the dashboard (Malcolm services on browser) comping up in the browser even after 5 min. Not sure why? i have run the above comments only. ( did not run the from the beginning ie. sudo ./scripts/install.py_). Thanks.

malcolm_wipe

@mmguero mmguero changed the title Installation on using Ubuntu 22.04 install.py can create .env files 0:0 ownership instead of unprivileged user ownership May 15, 2023
@mmguero
Copy link
Collaborator

mmguero commented May 15, 2023

Run ./scripts/status or docker-compose ps -a in your Malcolm directory and let me see the output.

This was referenced May 15, 2023
Merged
Merged
@e101sg
Copy link
Author

e101sg commented May 16, 2023

Sure. Here it is...

Screenshot from 2023-05-16 08-44-12

@mmguero
Copy link
Collaborator

mmguero commented May 16, 2023

My guess is now you're looking at resources (memory) issues for opensearch. What's the system memory (head /proc/meminfo) and what do you have allocated for OpenSearch and LogStash (grep JAVA_OPTS ./config/*.env)?

@e101sg
Copy link
Author

e101sg commented May 16, 2023 •
edited
Loading

Memory should not be issue since the machine is Desktop with 32GB memory. Screen shot.

Screenshot from 2023-05-16 20-37-03

Interestingly, when i try to stop the Malcolm, getting some strange error.. (seeing it first time)
Hope it gives a hint.. what is an issue.

stop_error

@mmguero
Copy link
Collaborator

mmguero commented May 16, 2023

I'd double-check and just make sure (while malcolm is shut down) all files in ~/Documents/Malcolm are owned by your user (sudo chown -R user:user ~/Documents/Malcolm).

As far as the permission denied goes, I've only seen that one other place, it was when a user was running "Ubuntu Core" which (I guess) is an all snap-based Ubuntu distro. That's not what you're using by chance, is it? I was never able to figure out his problem.

mmguero added a commit to idaholab/Malcolm that referenced this issue May 16, 2023
@mmguero
Merge pull request #195 from idaholab/v23.05.1_merge_idaholab
2a253f2 
Malcolm v23.05.1 is a minor release with a few component version updates and bug fixes, particularly to fix an issue with `install.py` where the ownership of `.env` files in the `config` directory may get incorrectly set to `root` rather than the unprivileged user.

cisagov/Malcolm@v23.05.0...v23.05.1

* Enhancements and bug fixes
    - install.py can create .env files 0:0 ownership instead of unprivileged user ownership (cisagov#253)
    -  both zeek and zeek-live containers are trying to pull intel feeds on startup (#196)
    - Make sure a few Arkime fields (`http.xff*`) get created in the index template with the right field types to avoid aggregation query issues
    - Tweaks to convenience scripts (`malcolmmonitor` and `sensormonitor`) in ISO-installed Malcolm and Hedgehog Linux environments
    - Added some `.service` files for the ISO-installed version of Malcolm to be able to feed itself resource statistics via Fluent Bit
    - Documentation updates    

* Component version updates
    - Arkime to [v4.3.1](https://github.com/arkime/arkime/blob/ce8d5d4d1a54a3a9f022bf4b72081f7af666f6e4/CHANGELOG#L33-L44)
    - OpenSearch and OpenSearch Dashboards to [v2.7.0](https://github.com/opensearch-project/opensearch-build/blob/2dbbce4428e583c4cf1f1f867f7591d978395420/release-notes/opensearch-release-notes-2.7.0.md)
    - NetBox to [v3.5.1](https://netbox.dev/blog/posts/netbox-v351-released/)
    - Beats to [v8.7.1](https://www.elastic.co/guide/en/beats/libbeat/8.7/release-notes-8.7.1.html)
mmguero added a commit that referenced this issue May 16, 2023
@mmguero
Merge pull request #254 from cisagov/v23.05.1_merge_cisagov
8221433 
Malcolm v23.05.1 is a minor release with a few component version updates and bug fixes, particularly to fix an issue with `install.py` where the ownership of `.env` files in the `config` directory may get incorrectly set to `root` rather than the unprivileged user.

v23.05.0...v23.05.1

* Enhancements and bug fixes
    - install.py can create .env files 0:0 ownership instead of unprivileged user ownership (#253)
    -  both zeek and zeek-live containers are trying to pull intel feeds on startup (idaholab#196)
    - Make sure a few Arkime fields (`http.xff*`) get created in the index template with the right field types to avoid aggregation query issues
    - Tweaks to convenience scripts (`malcolmmonitor` and `sensormonitor`) in ISO-installed Malcolm and Hedgehog Linux environments
    - Added some `.service` files for the ISO-installed version of Malcolm to be able to feed itself resource statistics via Fluent Bit
    - Documentation updates    

* Component version updates
    - Arkime to [v4.3.1](https://github.com/arkime/arkime/blob/ce8d5d4d1a54a3a9f022bf4b72081f7af666f6e4/CHANGELOG#L33-L44)
    - OpenSearch and OpenSearch Dashboards to [v2.7.0](https://github.com/opensearch-project/opensearch-build/blob/2dbbce4428e583c4cf1f1f867f7591d978395420/release-notes/opensearch-release-notes-2.7.0.md)
    - NetBox to [v3.5.1](https://netbox.dev/blog/posts/netbox-v351-released/)
    - Beats to [v8.7.1](https://www.elastic.co/guide/en/beats/libbeat/8.7/release-notes-8.7.1.html)
@mmguero
Copy link
Collaborator

mmguero commented May 16, 2023

Closing this issue as its initial root cause (the permission of the .env files) has been fixed and released in Malcolm v23.05.1.

@mmguero mmguero closed this as completed May 16, 2023
@e101sg
Copy link
Author

e101sg commented May 17, 2023

Update: Able to see dashboards after successful installation. Thanks a lot for timely help. :) Cheers!

@mmguero
Copy link
Collaborator

mmguero commented May 18, 2023

Glad to hear it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@e101sg @mmguero