-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Not able to delete the network #159
Comments
@rajatgupta21 please test it again. I believe this was fixed in current master. |
Please attach logs if possible to understand the issue better. |
Ok here we go, In my extensive testing with latest master code I observed that deletion of bond/vlan (some time with individual or multiple selection), wokd process is getting killed. As per me this is kind of serious issue we should address. |
Do you have any logs? |
wokd is crashing with core dump, some things like this shown: [root@chandra ~]# wokd |
And there was this also but not sure this just because keyboard interrupt CTRL+C: [root@zfwpc159 wok]# wokd |
[root@zfwpc159 ~]# wokd |
I am not expert here but I feel augeas is the culprit |
My first guess is that we're having a double free() in the network parser object (augeas) and this is triggering this error. I'll play with Vlans here and see if I can get the same error |
I actually found one issue in the UI too. So sending the patch for the same. |
OK I now have some confident in reproducing the issue: I am allowing only one interface delete at a time till we have good fix for above mentioned issue. |
How does this multiple parallel delete works? Is this a unique API call or multiple calls? I can look into it |
Sending multiple REST API commands parallel. From UI code, calling REST API go parallel as calling the API doesn't On 22/01/16 9:38 PM, Daniel Henrique Barboza wrote:
|
Perhaps one fix, instead of limiting the multiple delete, would be to fire one single request at a time from the UI? Send the first one, wait for answer. Then send the next one. Repeat. This would be just a temporary workaround, of course, until the backend can handle these multiple requests properly. |
We can do but we need task API implementation to do this I guess. On 22/01/16 10:23 PM, Daniel Henrique Barboza wrote:
|
We have the Task API implemented in Ginger already. But I believe the issue is band-aided for the moment after the recent master changes. |
As suspected this is augeas who is troubling. |
OK. Looking at model/cfginterfaces.py code : parser = augeas.Augeas("/") I have seen the following method but never called in the life time of cfginterfaces.py. @atexit.register I some how feel that we should not create global augeas parser object and use it entire life of the application all the places since such parser may have memory clean up issues and impact adversely. Delete is the case where you play with parser object bit different where you read data using parser and clean tokens etc (ie parser.remove, parser.savae esp in case of bond deletion) ...
By looking at the augeas issue, I would not recommend to create global parser object and clean it off only during the exit. By having parser object global forever will also have adverse impact on performance of wokd, may be due to bad clean up or memory related miss-handling in the augeas code. So suggestion:
Let me know your thoughts !!! |
@atexit.register This will be called, when wokd server is being stopped. @danielhb correct me if i am wrong understanding this. I tried initialiatizing augeas inside all the methods locally and destroy it. This needs more investigation with respect to augeas handling multiple |
@abhiramkulkarni is right about Augeas. We were experiencing an error when exiting wokd due to not cleaning up the augeas parser object. This is why we do this cleanup using atexit.register Now, if Augeas can't handle multiple parsing requests then we have 3 choices:
First alternative is the more sane to try, in my opinion |
I would not consider doing the parsing by hand. Even though this is feasible, you would probably need to deal with the same issues you are having with Augeas, such as using a lock mechanism to make sure you are not changing a file from multiple parsing instances at the same time. |
I guess this is the time we need to engineer better and solve the issue. We have seen double free or corruption errors and wokd killing self is not at all good. As Daniel suggested let us try 1st or 2nd approaches and sort this issue out. |
@chandrureddy , just saw a patch from @jay-katta that implemented lock mechanism in the cfginterface class (Implement locking for network functions). Isn't it enough to solve this issue (solution 2) ? It's not upstream yet but it can be a start |
Click Network→Network Configuration → Delete
error message : Delete interface enccw0.0.1000 failed !GINNET0015E: Error getting information from ifcfg file: 'Error during match procedure!'.
The text was updated successfully, but these errors were encountered: