Anyway to use Dockge to consolidate Stacks?

[silverjerk]

⚠️ Please verify that this bug has NOT been raised before.

• I checked and didn't find similar issue

🛡️ Security Policy

• I agree to have read this project Security Policy

📝 Describe your problem

When first deploying to my local environment I was using the command line and manually adding .env and compose files; I later installed Portainer and began managing Stacks from there.

My original appdata stack contains many of my initial self-hosted services. As a result, Portainer has no access to this stack. And with my current Dockge install, it cannot access appdata or any of the Portainer stacks.

Current setup looks like this:

appdata (10 containers)
dockge (1 container)
heimdall (1 container)
homarr (1 container)

Ultimately what I'm trying to do is move all Docker management away from Portainer, and find a way to access the appdata stack (which Dockge currently displays a status of "exited" and Portainer cannot access), and generate a stack file so I can regain access to my original deployment script and reuse it as necessary.

Is it at all possible to workaround Dockge not having access to my original appdata and Portainer stacks? Is it also possible to generate a stack script from the original stack that was deployed?

📝 Error Message(s) or Log

No response

🐻 Dockge Version

1.4.2

💻 Operating System and Arch

DSM 7

🌐 Browser

Arc

🐋 Docker Version

No response

🟩 NodeJS Version

No response

[huuscript]

Have you look in /data/compose/
Also, you can paste the docker run commands into “Docker Run” and click convert to compose in dockge

[trythatagain]

When using portainer, if you added a stack manually by using add stack and then pasted in a compose file, then you can use the same compose file in Dockge, by using the '+compose' button. Plus as 'huuscript; says have a look at where you stored locally (rleft hand path) to 'data/' from your portainer compose file.

If you used a appdata folder in portainer, then all your data is there and you just point dockge there from your dockge.yml file.

Might be worth posting your compose.yml that span up portainer and you compose.yml file that span up dockge.
I've done this from OMV7, but I'm not sure I understand all your terminology, so seeing the 2 compose files, should make it easier.

[silverjerk]

I'm likely not expressing the goal well here, and I think this is probably out of scope of this discussion thread, but I thought I'd add some detail for the sake of getting additional insight.

In Portainer, I've lost access to editing my media stack (compose file of "appdata" added below); this is a common issue when updating Portainer to the latest version, and one of the main reasons I'd like to move away from the app, as I've run into other issues with its use.

So in Portainer, I can no longer modify the stack, and in Dockge I can see the stack properly now by pointing Dockge to it, but it shows as a single "appdata" stack that is exited. I think this is probably the default behavior, since some of the services within the stack are technically offline as I don't run some of them.

In Portainer, my stack list is as follows:

appdata
dockge
heimdall
homarr
prowlarr
docmost
outline
traefik
syncthing

What I'm trying to do is aggregate my appdata stack so that all of my services (minus a few that I'd like to keep separate) can be spun up or down in one command; I think I resolved my other major issue, which was finding a way for other containers to use the appdata_default network (I want them to be able to communicate easily with one another, and to make reverse proxying easier throughout).

Secondarily, I'm slightly confused on the best methodology for file structure when using Dockge; currently all of my docker folders, as above, are going into the appdata folder, and then I create a separate folder with the compose file within Dockge's stack folder. This seems duplicative and like I'm missing the methodology here. It feels like Dockge wants to be the parent directory and the application folders live within the stacks directory alongside their compose files?

TLDR: I want to clean up this server and use Dockge as my main deployment method, as many of my stacks and/or containers are becoming disorganized (and some of that due to me trying to contend with issues that arose while using Portainer).

Appdata stack attached.

Attached a secondary example of another stack I installed with Dockge (syncthing).

media-compose.txt
syncthing-compose.txt

[huuscript]

I too running Dockge & Portainer in the same Docker and here is what i learn and how i use it.

1. Portainer and Dockge can not use the same compose yaml file as the folder structure is different. Portainer put its compose file in folder name in number sequence where as Dockge put them in folder name of the container. So keeping them in the same location can cause unpredictable results.
2. Deploy containers via Dockge then portainer will loose management to that container compose and vice versa they can see each other containers but can not compose it.

I delete portainer via CLI and deployed the Portainer container from Dockge, so now i can update potainer with one click instead of going through CLI and deployed Dockge from Portainer for same reason.

I still keep portainer around to quickly remove orphans volume and images and network.

I dont know why you loose control of the stack when updating portainer though, is it because your compose file of the stack is not store at the default portainer location?

[trythatagain]

As with most things Homelab, there is nearly always more than one way of doing something. :)
As I mentioned I use OMV7, it changed from portainer to it's own plugin, which is a lot better in my opinion, it can do a lot more and its easy to understand and has 'global env file'.
But I've always kept my compose files in a folder called compose, with container-name subfolder, like Dockge and OMV7.
I also keep all my app data (container data) in a folder called appdata, with container-name subfolders.

Plus I also keep 1 big yml file with all the container compose.yml file data, just makes it easy when searching for certain things.

As you may know you can create multiple docker networks, so I generally create a docker network first (named mynet) before creating any containers . Then I add the following to each container.

    networks:
      - mynet

networks:    
  mynet:
    external: true

There's a fair bit to learn in docker, but also lots and lots of youtube videos and articles about.

But it sounds as though you're on the right track, so good luck and have fun!