Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Queries fail with error socket: too many open files #190

Closed
LalitLab opened this issue Feb 17, 2021 · 1 comment
Closed

Queries fail with error socket: too many open files #190

LalitLab opened this issue Feb 17, 2021 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@LalitLab
Copy link

Branch: https://github.com/turbot/steampipe-plugin-gcp/tree/issue-67

➜  config steampipe query 
Welcome to Steampipe v0.1.3
For more information, type .help

> select * from gcp_iam_role;
Error: pq: rpc error: code = Unknown desc = Get "https://iam.googleapis.com/v1/roles/container.admin?alt=json&prettyPrint=false": dial tcp 216.58.203.42:443: socket: too many open files

Partial results

> select * from gcp_iam_role;
+--------------------------------------------------+---------+-----------------------+----------------+-------+------------------------+--------------+-------------------------------------------------------------------------------------
| name                                             | deleted | role_id               | is_gcp_managed | stage | description            | etag         | included_permissions                                                                
+--------------------------------------------------+---------+-----------------------+----------------+-------+------------------------+--------------+-------------------------------------------------------------------------------------
| projects/niteowl-aaa/roles/gcp_metadata_mk5cnm9m | false   | gcp_metadata_mk5cnm9m | false          | BETA  |                        | BwWrGXTA3dQ= | ["datacatalog.categories.fineGrainedGet","datacatalog.categories.getIamPolicy","data
| projects/niteowl-aaa/roles/CustomRole2           | false   | CustomRole2           | false          |       | Created on: 2020-05-07 | BwWnFW9Fzhs= | ["accessapproval.requests.approve"]                                                 
| projects/niteowl-aaa/roles/gcp_owner_mk5cnm9m    | false   | gcp_owner_mk5cnm9m    | false          | BETA  |                        | BwWrGXTCbrk= | ["datacatalog.categories.fineGrainedGet","datacatalog.categories.getIamPolicy","data
|                                                  |         |                       |                |       |                        |              | omies.list","datacatalog.taxonomies.setIamPolicy","datacatalog.taxonomies.update","i
| projects/niteowl-aaa/roles/TestRole5             | false   | TestRole5             | false          |       | Created on: 2020-06-02 | BwWnFVa70Cg= | ["accessapproval.requests.approve"]                                                 
+--------------------------------------------------+---------+-----------------------+----------------+-------+------------------------+--------------+-------------------------------------------------------------------------------------

> .quit

Solution to the above issue on my system

The existing ulimit value on my system 256

➜  config ulimit -n
256

Changed ulimit value to 1024

config ulimit -n 1024

➜  config ulimit -n
1024

Now ran below queries and table works fine

➜  config steampipe query 
Welcome to Steampipe v0.1.3
For more information, type .help
> select count(*) from gcp_iam_role;
+-------+
| count |
+-------+
| 735   |
+-------+

> select * from gcp_iam_role;
+------------------------------------------------------------+---------+------------------------------------------------------+----------------+-
| name                                                       | deleted | role_id                                              | is_gcp_managed | 
+------------------------------------------------------------+---------+------------------------------------------------------+----------------+-
| projects/niteowl-aaa/roles/gcp_metadata_mk5cnm9m           | false   | gcp_metadata_mk5cnm9m                                | false          | 
| projects/niteowl-aaa/roles/gcp_owner_mk5cnm9m              | false   | gcp_owner_mk5cnm9m                                   | false          | 
|                                                            |         |                                                      |                | 
| roles/accesscontextmanager.policyReader                    | false   | accesscontextmanager.policyReader                    | true           | 
| roles/accesscontextmanager.policyAdmin                     | false   | accesscontextmanager.policyAdmin                     | true           | 
|                                                            |         |                                                      |                | 
| roles/anthosaudit.serviceAgent                             | false   | anthosaudit.serviceAgent                             | true           | 
| roles/apigee.analyticsEditor                               | false   | apigee.analyticsEditor                               | true           | 
| roles/androidmanagement.user                               | false   | androidmanagement.user                               | true           | 
| roles/anthos.serviceAgent                                  | false   | anthos.serviceAgent                                  | true           | 
| roles/accessapproval.approver                              | false   | accessapproval.approver                              | true           | 
| roles/accesscontextmanager.gcpAccessReader                 | false   | accesscontextmanager.gcpAccessReader                 | true           | 
| projects/niteowl-aaa/roles/TestRole5                       | false   | TestRole5                                            | false          | 
| roles/anthosservicemesh.serviceAgent                       | false   | anthosservicemesh.serviceAgent                       | true           | 
| roles/accesscontextmanager.vpcScTroubleshooterViewer       | false   | accesscontextmanager.vpcScTroubleshooterViewer       | true           | 
| roles/artifactregistry.admin                               | false   | artifactregistry.admin                               | true           | 
| projects/niteowl-aaa/roles/CustomRole2                     | false   | CustomRole2                                          | false          | 
| roles/actions.Admin                                        | false   | actions.Admin                                        | true           | 
| roles/apigee.synchronizerManager                           | false   | apigee.synchronizerManager                           | true           | 
| roles/appengine.serviceAdmin                               | false   | appengine.serviceAdmin                               | true           | 


Link I refered for ulimit solution
https://stackoverflow.com/questions/880557/socket-accept-too-many-open-files

@LalitLab LalitLab added the bug Something isn't working label Feb 17, 2021
@johnsmyth johnsmyth changed the title Queries for GCP plugin fails with errror socket: too many open files Queries fail with error socket: too many open files Feb 25, 2021
@johnsmyth
Copy link

Closed in SDK fix - turbot/steampipe-plugin-sdk#45

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants