Twemsentinel : supervisor restart error : nutcracker: ERROR (no such process)

[vishalsharma13]

Hi,

I have just configure the twemsentinel but facing below error while restart the sentinel.

restart
nutcracker: ERROR (no such process)
nutcracker: ERROR (no such process)
restart error

config.yml :-

sentinel_ip: "10.247.74.38"
sentinel_port: "26380"
twemproxy_config_file: "nutcracker.yml"
nutcracker_restart_command: "supervisorctl restart nutcracker"
log_file: "twemsentinel.log"

Please help to resolve the issue.

[yak0]

Hi
You should install nutcracker ( https://github.com/twitter/twemproxy ) and set nutcracker_restart_command in config.yml
I used supervisord for restart nutcracker (http://supervisord.org/)

[vishalsharma13]

I have already installed the nutcracker and run through with below command.

cache 15407 1 0 16:20 ? 00:00:05 /home/cache/.gem/ruby/gems/nutcracker-0.4.1.20/ext/nutcracker/src/nutcracker -c ../conf/nutcracker.yml -p /var/run/nutcracker/nutcracker.pid

I am using supervisorctl to restart the nutcracker service but it gives below error

nutcracker: ERROR (no such process)
nutcracker: ERROR (no such process)

But it unable to find the process id of nutcracker to restart the process.

Also please share the command how to restart the nutcracker through supervisord.

[yak0]

The problem is not about twemsentinel
Your supervisorctl cant find nutcracker process. Did you try "supervisorctl reload" and "supervisorctl update"

Try in terminal "supervisorctl restart all" or "supervisorctl restart nuctracker(or you program name)"

[vishalsharma13]

how to run it through supervisord. We are getting error while running below command.

supervisord restart nutcracker

Error :- positional arguements are not supported.

[yak0]

can you paste supervisord program conf file

[vishalsharma13]

; Sample supervisor config file.
;
; For more information on the config file, please see:
; http://supervisord.org/configuration.html
;
; Note: shell expansion ("~" or "$HOME") is not supported. Environment
; variables can be expanded using this syntax: "%(ENV_HOME)s".

[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
chmod=0777 ; socket file mode (default 0700)
chown=cache:cache ; socket file uid:gid owner
;username=user ; (default is no username (open server))
;password=123 ; (default is no password (open server))

;[inet_http_server] ; inet (TCP) server disabled by default
;port=127.0.0.1:9001 ; (ip_address:port specifier, *:port for all iface)
;username=user ; (default is no username (open server))
;password=123 ; (default is no password (open server))

[supervisord]
logfile=/home/cache/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/home/cache/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
;umask=022 ; (process file creation umask;default 022)
user=cache ; (default is current user, required if root)
;identifier=supervisor ; (supervisord identifier, default is 'supervisor')
;directory=/tmp ; (default is not to cd during start)
;nocleanup=true ; (don't clean up tempfiles at start;default false)
;childlogdir=/tmp ; ('AUTO' child log dir, default $TEMP)
;environment=KEY="value" ; (key value pairs to add to environment)
;strip_ansi=false ; (strip ansi escape codes in logs; def. false)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
user=cache
;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
;username=chris ; should be same as http_username if set
;password=123 ; should be same as http_password if set
;prompt=mysupervisor ; cmd line prompt (default "supervisor")
;history_file=~/.sc_history ; use readline history if available

; The below sample program section shows all possible program subsection values,
; create one or more 'real' program: sections to be able to control them under
; supervisor.

;[program:theprogramname]
;command=/bin/cat ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1 ; number of processes copies to start (def 1)
;directory=/tmp ; directory to cwd to before exec (def no cwd)
;umask=022 ; umask for process (default None)
;priority=999 ; the relative start priority (default 999)
;autostart=true ; start at supervisord start (default: true)
;autorestart=unexpected ; whether/when to restart (default: unexpected)
;startsecs=1 ; number of secs prog must stay running (def. 1)
;startretries=3 ; max # of serial start failures (default 3)
;exitcodes=0,2 ; 'expected' exit codes for process (default 0,2)
;stopsignal=QUIT ; signal used to kill process (default TERM)
;stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)
;stopasgroup=false ; send stop signal to the UNIX process group (default false)
;killasgroup=false ; SIGKILL the UNIX process group (def false)
user=cache ; setuid to this UNIX account to run the program
;redirect_stderr=true ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups=10 ; # of stdout logfile backups (default 10)
;stdout_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default 0)
;stdout_events_enabled=false ; emit events on stdout writes (default false)
;stderr_logfile=/a/path ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups=10 ; # of stderr logfile backups (default 10)
;stderr_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default 0)
;stderr_events_enabled=false ; emit events on stderr writes (default false)
;environment=A="1",B="2" ; process environment additions (def no adds)
;serverurl=AUTO ; override serverurl computation (childutils)

; The below sample eventlistener section shows all possible
; eventlistener subsection values, create one or more 'real'
; eventlistener: sections to be able to handle event notifications
; sent by supervisor.

;[eventlistener:theeventlistenername]
;command=/bin/eventlistener ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1 ; number of processes copies to start (def 1)
;events=EVENT ; event notif. types to subscribe to (req'd)
;buffer_size=10 ; event buffer queue size (default 10)
;directory=/tmp ; directory to cwd to before exec (def no cwd)
;umask=022 ; umask for process (default None)
;priority=-1 ; the relative start priority (default -1)
;autostart=true ; start at supervisord start (default: true)
;autorestart=unexpected ; whether/when to restart (default: unexpected)
;startsecs=1 ; number of secs prog must stay running (def. 1)
;startretries=3 ; max # of serial start failures (default 3)
;exitcodes=0,2 ; 'expected' exit codes for process (default 0,2)
;stopsignal=QUIT ; signal used to kill process (default TERM)
;stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)
;stopasgroup=false ; send stop signal to the UNIX process group (default false)
;killasgroup=false ; SIGKILL the UNIX process group (def false)
user=cache ; setuid to this UNIX account to run the program
;redirect_stderr=true ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups=10 ; # of stdout logfile backups (default 10)
;stdout_events_enabled=false ; emit events on stdout writes (default false)
;stderr_logfile=/a/path ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups ; # of stderr logfile backups (default 10)
;stderr_events_enabled=false ; emit events on stderr writes (default false)
;environment=A="1",B="2" ; process environment additions
;serverurl=AUTO ; override serverurl computation (childutils)

; The below sample group section shows all possible group values,
; create one or more 'real' group: sections to create "heterogeneous"
; process groups.

;[group:thegroupname]
;programs=progname1,progname2 ; each refers to 'x' in [program:x] definitions
;priority=999 ; the relative start priority (default 999)

; The [include] section can just contain the "files" setting. This
; setting can list multiple files (separated by whitespace or
; newlines). It can also contain wildcards. The filenames are
; interpreted as relative to this file. Included files cannot
; include files themselves.

;[include]
;files = relative/directory/*.ini

[yak0]

write this in your supervisord conf file and reload supervisord
[program:nutcracker]
command=(your nutcracker path)/nutcracker -c (twemsentinel path)/nutcracker.yml
stderr_logfile = /var/log/supervisord/nutcracker-err.log
stdout_logfile = /var/log/supervisord/nutcracker-out.log

[vishalsharma13]

Hi,

Thanks for kind support.

Its working now but give below error.

nutcracker: ERROR (not running)
nutcracker: ERROR (abnormal termination)

supervisord Log File :-

2017-01-19 23:24:35,031 INFO exited: nutcracker (terminated by SIGABRT; not expected)
2017-01-19 23:24:42,249 INFO spawned: 'nutcracker' with pid 11327
2017-01-19 23:24:42,264 INFO exited: nutcracker (terminated by SIGABRT; not expected)
2017-01-19 23:24:50,350 INFO spawned: 'nutcracker' with pid 11338
2017-01-19 23:24:50,364 INFO exited: nutcracker (terminated by SIGABRT; not expected)
2017-01-19 23:24:59,381 INFO spawned: 'nutcracker' with pid 11339
2017-01-19 23:24:59,395 INFO exited: nutcracker (terminated by SIGABRT; not expected)
2017-01-19 23:25:09,413 INFO spawned: 'nutcracker' with pid 11705
2017-01-19 23:25:09,429 INFO exited: nutcracker (terminated by SIGABRT; not expected)
2017-01-19 23:25:10,431 INFO gave up: nutcracker entered FATAL state, too many start retries too quickly

[vishalsharma13]

Hi,

After restart through supervisor, server value updated in single quotes like ' 192.168.9.100 ' in nutcracker.yml file . I have tried to resolved but nut able to fix yet.

If you have any idea please help me to resolve.