Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Newer Instances never stopped when stop_new_instances value set to false #127

Closed
lpavan225 opened this issue Sep 26, 2019 · 3 comments
Closed

Comments

@lpavan225
Copy link

lpavan225 commented Sep 26, 2019

Documentation states that

stop_new_instances : Choose whether to stop an instance the first time it is tagged if it is running outside of the running period
Refer : https://s3.amazonaws.com/solutions-reference/aws-instance-scheduler/latest/instance-scheduler.pdf

when this flag is set to FALSE instance should not be stopped the first time it is running outside of the period and the next day the instance should be stopped. but that is not happening.

The Log Always shows the below message: From three days I am seeing the same message as below.

2019-09-24 - 22:05:18.837 - DEBUG : [ Instance EC2:i-01f15732e599d50a4 (v428096-devops_lab-77j919u-BMS-EC2-Stdz-lab) ]
2019-09-24 - 22:05:18.837 - DEBUG : Current state is running, instance type is t2.micro, schedule is "CORP_-1800_newyork_daily_stopAll"
2019-09-24 - 22:05:18.837 - DEBUG : Time used to determine desired for instance is Tue Sep 24 18:05:06 2019
2019-09-24 - 22:05:18.838 - DEBUG : Checking conditions for period "CORP_-1800_daily_stopAll"
2019-09-24 - 22:05:18.838 - DEBUG : [running] Weekday "tue" in weekdays (mon-sun)
2019-09-24 - 22:05:18.838 - DEBUG : [stopped] Time 18:05:06 is outside 18:00:00-18:01:00, returned state is stopped
2019-09-24 - 22:05:18.838 - DEBUG : No running periods at this time found in schedule "CORP_-1800_newyork_daily_stopAll" for this time, desired state is stopped
2019-09-24 - 22:05:18.838 - DEBUG : Desired state for instance from schedule "CORP_-1800_newyork_daily_stopAll" is stopped, last desired state was unknown, actual state is running
2019-09-24 - 22:05:18.838 - DEBUG : New instance "EC2:i-01f15732e599d50a4 (v428096-devops_lab-77j919u-BMS-EC2-Stdz-lab)" will not be stopped until next scheduling period

@DaveO-IRESS
Copy link

We are also seeing this problem (we had assumed it was when enforced is also false, but it may be a wider problem than that).
We need this to be working because we often have instances coming up out of hours and we don't want them to be terminated before their user data script has run - but the next 'out of hours' phase it is fine for them to go down. With this set to true, they get shut down before the user data script can complete.

@lpavan225
Copy link
Author

In this file master/source/code/schedulers/instance_scheduler.py

When ever the schedules finds the new instance its simply skipping that instance and not setting the state of the instance to the state table. So, the instance is forever considered as a new instance and is never being stopped.

This can be fixed by setting the state for the instance at line No 300 in master/source/code/schedulers/instance_scheduler.py

add this line self._instance_states.set_instance_state(instance.id, InstanceSchedule.STATE_RUNNING)

This will fix this issue.

@chaitand28
Copy link

This issue has been fixed in the release 1.3.1. Please deploy the latest template to get the updated code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants