ZTP process doesn't have to start again in case of absence of config_db.json if earlier ZTP process had finished. But it does.

[MaxYaremchuk]

Description
According to ZTP process doesn't have to start again in case of absence of config_db.json if it earlier had finished.
If ztp.status field of local file is either SUCCESS, FAILED or DISABLED, ZTP service exits.
But ZTP starts again no matter whether it has finished earlier.

Steps to reproduce the issue

1. Initiate a ZTP session and let it complete

root@DUT-1:/home/admin# ztp status -v
========================================
ZTP
========================================
ZTP Admin Mode : True
ZTP Service    : Inactive
ZTP Status     : SUCCESS
ZTP Source     : dhcp-opt67 (eth0)
Runtime        : 18m 35s
Timestamp      : 2020-09-08 08:38:58 UTC
ZTP JSON Version : 1.0

ZTP Service is not running

----------------------------------------
01-snmp
----------------------------------------
Status          : SUCCESS
Runtime         : 01s
Timestamp       : 2020-09-08 08:21:52 UTC
Exit Code       : 0
Ignore Result   : False

----------------------------------------
02-script-01
----------------------------------------
Status          : SUCCESS
Runtime         : 17m 06s
Timestamp       : 2020-09-08 08:38:58 UTC
Exit Code       : 0
Ignore Result   : False

----------------------------------------
03-configdb-json
----------------------------------------
Status          : SUCCESS
Runtime         : 02m 12s
Timestamp       : 2020-09-08 08:24:05 UTC
Exit Code       : 0
Ignore Result   : False

----------------------------------------
04-configdb-json
----------------------------------------
Status          : SUCCESS
Runtime         : 02m 14s
Timestamp       : 2020-09-08 08:26:19 UTC
Exit Code       : 0
Ignore Result   : False

----------------------------------------
05-script-02
----------------------------------------
Status          : FAILED
Runtime         : 00s
Timestamp       : 2020-09-08 08:26:19 UTC
Exit Code       : 5
Error           : Plugin failed
Ignore Result   : True

----------------------------------------
06-script-04
----------------------------------------
Status          : SUCCESS
Runtime         : 00s
Timestamp       : 2020-09-08 08:26:19 UTC
Exit Code       : 0
Ignore Result   : False

----------------------------------------
07-script-05
----------------------------------------
Status          : SUCCESS
Runtime         : 01s
Timestamp       : 2020-09-08 08:26:20 UTC
Exit Code       : 0
Ignore Result   : True

----------------------------------------
08-script-06
----------------------------------------
Status          : SUCCESS
Runtime         : 07m 11s
Timestamp       : 2020-09-08 08:33:31 UTC
Exit Code       : 0
Ignore Result   : False

----------------------------------------
09-post-provisioning-script
----------------------------------------
Status          : SUCCESS
Runtime         : 03m 09s
Timestamp       : 2020-09-08 08:36:40 UTC
Exit Code       : 0
Ignore Result   : False
===========================================

2. Delete /etc/sonic/config_db.json file
3. Reboot the switch

Describe the results you received

After rebooting ZTP process has started again.
Earlier ZTP process has successfully finished at 2020-09-08 08:38:58 UTC

ztp status -v
========================================
ZTP
========================================
ZTP Admin Mode : True
ZTP Service    : Active Discovery
Runtime        : 01m 31s
ZTP Status     : Not Started

(04s) Restarting network discovery after link scan

syslog snippet

Sep  8 08:38:58.637108 sonic-step-04 INFO sonic-ztp[4136]: ZTP successfully completed at 2020-09-08 08:38:58 UTC.
Sep  8 08:41:48.246280 sonic INFO sonic-ztp[3930]: ZTP service started.
Sep  8 08:41:48.246781 sonic INFO sonic-ztp[3930]: Checking running configuration to load ZTP configuration profile.
Sep  8 08:41:49.282629 sonic INFO sonic-ztp[3929]: Waiting for system online status before continuing ZTP. (This may take 30--120 seconds).
Sep  8 08:43:05.074860 sonic INFO sonic-ztp[3929]: System is ready to respond.
Sep  8 08:43:05.270558 sonic INFO sonic-ztp[3930]: Link up detected for interface eth0
Sep  8 08:43:05.270992 sonic INFO sonic-ztp[3930]: Restarting network discovery after link scan.
Sep  8 08:43:15.777584 sonic INFO sonic-ztp[3930]: Restarted network discovery after link scan.
Sep  8 08:43:15.860066 sonic INFO sonic-ztp[3930]: Restarting network discovery after link scan.
Sep  8 08:43:27.105760 sonic INFO sonic-ztp[3930]: Restarted network discovery after link scan.
Sep  8 08:43:27.106526 sonic INFO sonic-ztp[3930]: Downloading provisioning data from http://192.168.0.1/ZTP/DUT_ztp.json to /var/run/ztp/ztp_data_opt67.json
Sep  8 08:43:27.442193 sonic INFO sonic-ztp[3930]: Starting ZTP using JSON file /var/run/ztp/ztp_data_opt67.json at 2020-09-08 08:43:27 UTC.
Sep  8 08:43:27.443492 sonic INFO sonic-ztp[3930]: Processing configuration section 01-snmp at 2020-09-08 08:43:27 UTC. 

Describe the results you expected
ZTP process doesn't have to start again in case of absence of config_db.json if it earlier had finished.

Output of show version

SONiC Software Version: SONiC.HEAD.0-fffee7e3
Distribution: Debian 9.12
Kernel: 4.9.0-11-2-amd64
Build commit: fffee7e3
Build date: Sun Jun 21 09:33:14 UTC 2020
Built by: ezrada@r-build-sonic02

Platform: x86_64-mlnx_msn4700-r0
HwSKU: ACS-MSN4700 

sonic_dump_DUT-1_20200908_084455.tar.gz

[rajendra-dendukuri]

Please refer to https://github.com/Azure/SONiC/blob/master/doc/ztp/ztp.md#36-start-and-exit-conditions. The ZTP is starting again as there is no configuration in the switch which means, since ZTP admin mode is enabled, it is the responsibility of ZTP to fetch the lost configuration.

[MaxYaremchuk]

Hi @rajendra-dendukuri
Thank you for having answered.
I'd like to put your attention to the fact that there is ambiguity in the manual.

snippet#1

When the ZTP service starts, it first checks if there already exists a ZTP JSON file locally and if found loads it. 
If ztp.status field of local file is either SUCCESS, FAILED or DISABLED, ZTP service exits.

snippet#2
Test Case #26
Objective: Verify behavior of ZTP when /etc/sonic/config_db.json file is not found during switch bootup
Test Steps:
Initiate a ZTP session and let it complete
Delete /etc/sonic/config_db.json file
Reboot the switch
Expected Results:
ZTP does not start. It stays in completed state.

I believe that if I have problems with misunderstanding due to the lack of accuracy in the documentation, then the same problems may be with other users.
@rajendra-dendukuri could you make changes in aforementioned manual's snippets to make them less confusing ?

[anshuv-mfst]

@rajendra-dendukuri - assigned to the issue.