State pkg.purged doesn't purge packages on at least Debian 8 to 11 #42306

tkaehn · 2017-07-14T10:10:47Z

Description of Issue/Question

State pkg.purged doesn't purge packages on at least Debian 8 and 9

Steps to Reproduce Issue

/srv/salt# cat purge.sls 
---
purge exim:
  pkg.purged:
    - pkgs:
      - exim4
      - exim4-base
      - exim4-config

exim4-daemon-light:
  pkg.purged
/srv/salt# salt-call state.apply purge
local:
----------
          ID: purge exim
    Function: pkg.purged
      Result: True
     Comment: All specified packages are already absent
     Started: 11:24:27.199039
    Duration: 323.465 ms
     Changes:   
----------
          ID: exim4-daemon-light
    Function: pkg.purged
      Result: True
     Comment: All specified packages are already absent
     Started: 11:24:27.526847
    Duration: 8.509 ms
     Changes:   

Summary for local
------------
Succeeded: 2
Failed:    0
------------
Total states run:     2
Total run time: 331.974 ms
/srv/salt# dpkg -l|grep exim
rc  exim4                                     4.89-2+deb9u1                        all          metapackage to ease Exim MTA (v4) installation
rc  exim4-base                                4.89-2+deb9u1                        amd64        support files for all Exim MTA (v4) packages
rc  exim4-config                              4.89-2+deb9u1                        all          configuration for the Exim MTA (v4)
rc  exim4-daemon-light                        4.89-2+deb9u1                        amd64        lightweight Exim MTA (v4) daemon

Versions Report

/srv/salt# salt-call --versions-report
Salt Version:
           Salt: 2016.11.6
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: Not Installed
          gitdb: 2.0.0
      gitpython: 2.1.1
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: 1.3.7
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Jan 19 2017, 14:48:08)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: 2.0.1
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1
 
System Versions:
           dist: debian 9.0 
        machine: x86_64
        release: 4.9.0-3-amd64
         system: Linux
        version: debian 9.0

garethgreenaway · 2017-07-14T16:03:57Z

@tkaehn Thanks for the report. So far I haven't been able to duplicate this particular issue if I rely on Salt to purge them. I do see the same results if I first use apt to remove them and then run the state. Were these packages previous removed manually prior to running the state?

tkaehn · 2017-07-17T05:17:43Z

@garethgreenaway Yes, the packages were removed before (in this case by installing postfix in another state file) and it's the same behaviour here: An installed package is purged, a removed package is not.

tkaehn · 2017-07-19T09:35:43Z

The issue applies for Salt 2017.7.0 as well.

tkaehn · 2017-09-15T06:57:45Z

Still not fixed in 2017.7.1

KumarRajnish · 2017-09-18T16:15:31Z

All, Due to a misconfiguration on our Office365 tenant, all email addressed to your Ruckus Wireless email address ([email protected]) were delivered to a catch all mailbox. This incident occurred between 11 PM to 8 AM (PST). IT is delivering these emails with a “Reply-all” rule to all individuals on the email chain. If you feel you are missing email attachments, please send email to [email protected]. We apologize for any inconvenience this may have caused. Regards, IT From: Thomas Kähn [mailto:[email protected]] Sent: Thursday, September 14, 2017 11:58 PM To: saltstack/salt <[email protected]> Cc: Subscribed <[email protected]> Subject: Re: [saltstack/salt] State pkg.purged doesn't purge packages on at least Debian 8 and 9 (#42306) Still not fixed in 2017.7.1 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#42306 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AHusM_8JSuq8ONyZ-ep00GbsgzqpNScXks5sih_-gaJpZM4OYEkQ>.

tkaehn · 2017-10-10T05:58:43Z

Problem still exists in 2017.7.2.

tkaehn · 2018-04-20T08:24:55Z

Problem still exists in 2018.3.0.

@garethgreenaway the ticket is still labelled as "Info needed". Which info do you need?

tkaehn · 2019-02-26T13:22:00Z

Problem still exists in 2019.2.0:

root@minion:~# cat /srv/salt/purge.sls 
---
mutt:
  pkg.purged
root@minion:~# dpkg -l mutt
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                  Version                 Architecture            Description
+++-=====================================-=======================-=======================-===============================================================================
rc  mutt                                  1.7.2-1+deb9u1          amd64                   text-based mailreader supporting MIME, GPG, PGP and threading
root@minion:~# salt-call state.apply purge test=False
local:
  Name: mutt - Function: pkg.purged - Result: Clean Started: - 13:20:35.103586 Duration: 28.135 ms

Summary for local
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1
Total run time:  28.135 ms
root@minion:~# dpkg -l mutt
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                  Version                 Architecture            Description
+++-=====================================-=======================-=======================-===============================================================================
rc  mutt                                  1.7.2-1+deb9u1          amd64                   text-based mailreader supporting MIME, GPG, PGP and threading

heini · 2019-06-16T10:31:46Z

Any idea when this is going to be fixed? It's nearing its 2nd anniversary...

stale · 2020-01-08T11:10:32Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

heini · 2020-01-08T16:52:23Z

Looks like today's "Stale Bot Day"...

stale · 2020-01-08T16:52:25Z

Thank you for updating this issue. It is no longer marked as stale.

stale · 2020-02-07T17:36:53Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale · 2020-02-11T19:19:11Z

Thank you for updating this issue. It is no longer marked as stale.

sagetherage · 2020-02-11T19:19:26Z

@dmurphy18 can you follow up on this ticket, please?

stale · 2020-03-12T19:50:23Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

heini · 2020-03-13T18:32:22Z

Ping!

stale · 2020-03-13T18:32:25Z

Thank you for updating this issue. It is no longer marked as stale.

sagetherage · 2020-04-14T21:42:28Z

stale bot is disabled so that won't be a problem any longer, kicking this back to triage and will follow up tomorrow.

sagetherage · 2020-04-16T20:41:50Z

I thought it was going through another round of triage, but looks like it is still pending that task. I will follow up with getting it through triage again this week and check back on Friday 2020-04-17

biwhite · 2020-05-12T16:12:29Z

Hitting this now. Package previously removed, but not purged. However pkg.purged is returning 'All specified packages are already absent'
Minion 2019.2.0
Master 3000.2

dmurphy18 · 2023-01-05T20:49:32Z

@tkaehn @biwhite This issue is for Python 2.7 which is no longer supported. But I tried with the current Salt 3005.1 minion (classic packaging) on Debian 10 (Debian 8 & 9 being EOL) which uses Python 3.

Whatever the problem it appears to have been fixed

root@tdeb10:/srv/salt# dpkg -l | grep exim
ii  exim4                                 4.92-8+deb10u7                               all          metapackage to ease Exim MTA (v4) installation
ii  exim4-base                            4.92-8+deb10u7                               amd64        support files for all Exim MTA (v4) packages
ii  exim4-config                          4.92-8+deb10u7                               all          configuration for the Exim MTA (v4)
ii  exim4-daemon-light                    4.92-8+deb10u7                               amd64        lightweight Exim MTA (v4) daemon
root@tdeb10:/srv/salt# salt-call --local state.apply purge
local:
----------
          ID: purge exim
    Function: pkg.purged
      Result: True
     Comment: All targeted packages were purged.
     Started: 13:42:46.168700
    Duration: 5419.589 ms
     Changes:   
              ----------
              installed:
                  ----------
                  exim4:
                      ----------
                      new:
                      old:
                          4.92-8+deb10u7
                  exim4-base:
                      ----------
                      new:
                      old:
                          4.92-8+deb10u7
                  exim4-config:
                      ----------
                      new:
                      old:
                          4.92-8+deb10u7
                  exim4-daemon-light:
                      ----------
                      new:
                      old:
                          4.92-8+deb10u7
              removed:
                  ----------
----------
          ID: exim4-daemon-light
    Function: pkg.purged
      Result: True
     Comment: All specified packages are already absent
     Started: 13:42:52.578854
    Duration: 1.008 ms
     Changes:   

Summary for local
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time:   5.421 s
root@tdeb10:/srv/salt# dpkg -l | grep exim
root@tdeb10:/srv/salt# 
root@tdeb10:/srv/salt# salt-call --local test.versions
local:
    Salt Version:
              Salt: 3005.1
     
    Dependency Versions:
              cffi: Not Installed
          cherrypy: Not Installed
          dateutil: 2.7.3
         docker-py: Not Installed
             gitdb: Not Installed
         gitpython: Not Installed
            Jinja2: 2.10
           libgit2: Not Installed
          M2Crypto: Not Installed
              Mako: 1.0.7
           msgpack: 0.5.6
      msgpack-pure: Not Installed
      mysql-python: Not Installed
         pycparser: Not Installed
          pycrypto: 2.6.1
      pycryptodome: 3.6.1
            pygit2: Not Installed
            Python: 3.7.3 (default, Oct 31 2022, 14:04:00)
      python-gnupg: Not Installed
            PyYAML: 3.13
             PyZMQ: 17.1.2
             smmap: Not Installed
           timelib: Not Installed
           Tornado: 4.5.3
               ZMQ: 4.3.1
     
    System Versions:
              dist: debian 10 buster
            locale: UTF-8
           machine: x86_64
           release: 4.19.0-21-amd64
            system: Linux
           version: Debian GNU/Linux 10 buster
     
root@tdeb10:/srv/salt#

Hence closing this issue. Please open a new issue if this does not resolve the matter for you, given that this issue was originally with Python 2.7 on platforms which are End-Of-Life. Apologies for taking so long to resolve this issue.

tkaehn · 2023-01-05T21:16:24Z

Unfortunately the problem isn't solved at all. A package that is installed is purged. However a package that has already been removed still isn't purged. Here are the steps to reproduce the problem:

root@salt:/srv/salt# salt-call --versions-report
Salt Version:
          Salt: 3005.1
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.0
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
        Python: 3.9.16 (main, Dec 14 2022, 15:47:34)
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-20-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye
 
root@salt:/srv/salt# apt install goaccess
[...]
root@salt:/srv/salt# dpkg -r goaccess
(Reading database ... 56678 files and directories currently installed.)
Removing goaccess (1:1.4-1) ...
Processing triggers for man-db (2.9.4-2) ...

root@salt:/srv/salt# dpkg -l goaccess
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-============================================================
rc  goaccess       1:1.4-1      amd64        log analyzer and interactive viewer for the Apache Webserver

root@salt:/srv/salt# cat purge_goaccess.sls 
---
goaccess:
  pkg.purged

root@salt:/srv/salt# salt-call state.apply purge_goaccess test=False --state-verbose=True
local:
----------
          ID: goaccess
    Function: pkg.purged
      Result: True
     Comment: All specified packages are already absent
     Started: 22:06:14.850577
    Duration: 23.219 ms
     Changes:   

Summary for local
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1
Total run time:  23.219 ms

root@salt:/srv/salt# dpkg -l goaccess
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-============================================================
rc  goaccess       1:1.4-1      amd64        log analyzer and interactive viewer for the Apache Webserver

root@salt:/srv/salt# apt install goaccess
[...]
root@salt:/srv/salt# dpkg -l goaccess
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-============================================================
ii  goaccess       1:1.4-1      amd64        log analyzer and interactive viewer for the Apache Webserver

root@salt:/srv/salt# salt-call state.apply purge_goaccess test=False --state-verbose=True
local:
----------
          ID: goaccess
    Function: pkg.purged
      Result: True
     Comment: All targeted packages were purged.
     Started: 22:06:50.293949
    Duration: 4116.195 ms
     Changes:   
              ----------
              installed:
                  ----------
                  goaccess:
                      ----------
                      new:
                      old:
                          1:1.4-1
              removed:
                  ----------

Summary for local
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:   4.116 s

tkaehn · 2023-01-05T21:19:58Z

Please re-open this issue.

dmurphy18 · 2023-01-05T23:42:02Z

@tkae Well from #5503 and its associated #4972, looks like functionality was broken long ago circa 2016.11 or before, probably in the transition from apt.py module to aptpkg.py, which is very long ago.

This will take some time to wade through to see the breakage, will re-open
Have to check if the removal of this functionality was intentional or not, which is problematic given the age of the issue, Salt version. 2016.11.6

dmurphy18 · 2023-01-06T18:38:28Z

Looks like there is a problem in handling function _find_remove_targets given its comment about returning a dict, but it sometimes returns a state return object and other times a list, it and its usage needs to be refactored.
see https://github.com/saltstack/salt/blob/master/salt/states/pkg.py#L2901-L2903

Note that pkg.purge from execution module does purge packages which are removed.
Problem is with the state pkg.purged not completing the purging of the list of packages due to internal function stated above

dmurphy18 · 2023-01-06T18:44:24Z

Steps to reproduce.

apt install exim4
salt-call --local state.apply remove
dpkg -l | grep exim (should see packages are rc)
salt-call --local state.apply purge
dpkg -l | grep exim (should see packages are still rc - when should have been purged from machine)
salt-call --local pkg.purge exim4
dpkg -l | grep exim (should see package exim4 is purged from the machine)

The same results on Debian or Ubuntu

root@david:/srv/salt# dpkg -l | grep exim
rc  exim4                                      4.90.1-1ubuntu1.10                              all          metapackage to ease Exim MTA (v4) installation
rc  exim4-base                                 4.90.1-1ubuntu1.10                              amd64        support files for all Exim MTA (v4) packages
rc  exim4-config                               4.90.1-1ubuntu1.10                              all          configuration for the Exim MTA (v4)
rc  exim4-daemon-light                         4.90.1-1ubuntu1.10                              amd64        lightweight Exim MTA (v4) daemon
root@david:/srv/salt# cat remove.sls 
removed exim:
  pkg.removed:
    - pkgs:
      - exim4
      - exim4-base
      - exim4-config

exim4-daemon-light:
  pkg.removed
root@david:/srv/salt# cat purge.sls 
purge exim:
  pkg.purged:
    - pkgs:
      - exim4
      - exim4-base
      - exim4-config

exim4-daemon-light:
  pkg.purged
root@david:/srv/salt#

pkg.purged should purge installed and removed packages on the machine similar to execution module command pkg.purge

garethgreenaway added this to the Blocked milestone Jul 14, 2017

garethgreenaway added the info-needed waiting for more info label Jul 14, 2017

noelmcloughlin mentioned this issue Mar 3, 2019

Fix pkg.installed not removing 'winbind' packages saltstack-formulas/samba-formula#56

Merged

stale bot added the stale label Jan 8, 2020

stale bot removed the stale label Jan 8, 2020

stale bot added the stale label Feb 7, 2020

sagetherage assigned dmurphy18 Feb 11, 2020

stale bot removed the stale label Feb 11, 2020

stale bot added the stale label Mar 12, 2020

stale bot removed the stale label Mar 13, 2020

sagetherage added needs-triage and removed info-needed waiting for more info labels Apr 14, 2020

sagetherage unassigned dmurphy18 Apr 14, 2020

sagetherage removed this from the Blocked milestone Apr 14, 2020

Akm0d added P3 Priority 3 State-Module team-state and removed needs-triage labels Apr 17, 2020

Akm0d added this to the Approved milestone Apr 17, 2020

sagetherage removed the P3 Priority 3 label Jun 3, 2020

sagetherage removed the team-state label May 24, 2021

dmurphy18 self-assigned this Dec 5, 2022

dmurphy18 closed this as completed Jan 5, 2023

tkaehn changed the title ~~State pkg.purged doesn't purge packages on at least Debian 8 and 9~~ State pkg.purged doesn't purge packages on at least Debian 8 to 11 Jan 5, 2023

dmurphy18 reopened this Jan 5, 2023

dmurphy18 added Regression The issue is a bug that breaks functionality known to work in previous releases. Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged labels Jan 5, 2023

dmurphy18 added Bug broken, incorrect, or confusing behavior and removed Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged labels Jan 6, 2023

dmurphy18 mentioned this issue Jan 10, 2023

State pkg.purged doesn't purge removed packages on at least Debian 8 to 11 #63455

Merged

3 tasks

Ch3LL closed this as completed in #63455 Jan 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

State pkg.purged doesn't purge packages on at least Debian 8 to 11 #42306

State pkg.purged doesn't purge packages on at least Debian 8 to 11 #42306

tkaehn commented Jul 14, 2017

garethgreenaway commented Jul 14, 2017

tkaehn commented Jul 17, 2017

tkaehn commented Jul 19, 2017

tkaehn commented Sep 15, 2017

KumarRajnish commented Sep 18, 2017 via email

tkaehn commented Oct 10, 2017

tkaehn commented Apr 20, 2018

tkaehn commented Feb 26, 2019

heini commented Jun 16, 2019

stale bot commented Jan 8, 2020

heini commented Jan 8, 2020

stale bot commented Jan 8, 2020

stale bot commented Feb 7, 2020

stale bot commented Feb 11, 2020

sagetherage commented Feb 11, 2020

stale bot commented Mar 12, 2020

heini commented Mar 13, 2020

stale bot commented Mar 13, 2020

sagetherage commented Apr 14, 2020

sagetherage commented Apr 16, 2020

biwhite commented May 12, 2020

dmurphy18 commented Jan 5, 2023

tkaehn commented Jan 5, 2023

tkaehn commented Jan 5, 2023

dmurphy18 commented Jan 5, 2023 •

edited

Loading

dmurphy18 commented Jan 6, 2023

dmurphy18 commented Jan 6, 2023

State pkg.purged doesn't purge packages on at least Debian 8 to 11 #42306

State pkg.purged doesn't purge packages on at least Debian 8 to 11 #42306

Comments

tkaehn commented Jul 14, 2017

Description of Issue/Question

Steps to Reproduce Issue

Versions Report

garethgreenaway commented Jul 14, 2017

tkaehn commented Jul 17, 2017

tkaehn commented Jul 19, 2017

tkaehn commented Sep 15, 2017

KumarRajnish commented Sep 18, 2017 via email

tkaehn commented Oct 10, 2017

tkaehn commented Apr 20, 2018

tkaehn commented Feb 26, 2019

heini commented Jun 16, 2019

stale bot commented Jan 8, 2020

heini commented Jan 8, 2020

stale bot commented Jan 8, 2020

stale bot commented Feb 7, 2020

stale bot commented Feb 11, 2020

sagetherage commented Feb 11, 2020

stale bot commented Mar 12, 2020

heini commented Mar 13, 2020

stale bot commented Mar 13, 2020

sagetherage commented Apr 14, 2020

sagetherage commented Apr 16, 2020

biwhite commented May 12, 2020

dmurphy18 commented Jan 5, 2023

tkaehn commented Jan 5, 2023

tkaehn commented Jan 5, 2023

dmurphy18 commented Jan 5, 2023 • edited Loading

dmurphy18 commented Jan 6, 2023

dmurphy18 commented Jan 6, 2023

dmurphy18 commented Jan 5, 2023 •

edited

Loading