Symlink target issue #7

tlartaud · 2015-03-23T17:12:48Z

Hi there,

Thank you for for your node. its usefull.

I'm using some Chrooted environments on my remote, so my deploy user, and apache, use different root paths. shipit-shared us the "deployTo" path which is for the deploy user. The apache user may have some really different rights on some servers configurations.

Is there a way to allow shipit-shared to use symlinks on files as this :
ln -s ../../shared/.env .env
replacing shared by grunt.shipit.config.shared.baseDir

instead of
ln -s /httpdocs/shared/.env .env

Thank you by advance

The text was updated successfully, but these errors were encountered:

timkelty · 2015-03-23T21:27:44Z

Should be, I'll look into it tonight.

timkelty · 2015-03-24T16:11:38Z

@tlartaud Would it work for you if shipit.config.shared.baseDir would allow for an absolute path (starting with a /), in which case it wouldn't be relative to the deployTo path?

So your grunt.shipit.config.shared.baseDir would be /httpdocs/shared

timkelty · 2015-03-24T16:54:00Z

Let me know if that solves your issue.

I published this to https://github.com/timkelty/shipit-shared/tree/2.1.0

tlartaud · 2015-03-25T09:51:55Z

I've already tried the absolute path. The absolute path might NOT work because my deploy user do not have the same paths to the website, than the apache user.

Example
Web site path as apache user : /var/www/vhosts/domain.com/subdomain.com
Web site path as chrooted / deploying user : /subdomain.com

Well, if i do this, the symlink created will be :
ln -s /subdomain.com/shared/.env .env
which is not correct ! The page will be executed from http, and the script will be executed by apache.

So, the following paths would work
ln -s /var/www/vhosts/domain.com/subdomain.com/shared/.env .env
or
ln -s ../../shared/.env .env

In your shipit shared option, can't you add, next to the "baseDir" option, a new option called 'basePath', which could be optionnal, and which could allow user to ovveride the default deploy path with a custom one, different from the deploy path into shipit ?

PS : i will try again with your last update to see if it changes something.
Thank a lot !

timkelty · 2015-03-25T13:47:22Z

I'm not seeing how using an absolute path and this latest update wouldn't fix this.

In your shipitfile:

deployTo: '/var/www/vhosts/domain.com/subdomain.com',
shared: {
    baseDir: '/subdomain.com/shared'
}

Note, this will only work with the latest version. It basically does what you suggested, but without the additional setting. If the path is relative (not starting with /), it will make it relative to the deployTo path, but if it is absolute, it will take it as is.

tlartaud · 2015-04-09T09:42:59Z

Hi,

I apologize for the answer time .. i've been really busy theses last days.

I tried your solution, with last updates. Symlink creation works fine, but results paths are not correct for my server configuration.

This is my config file

{
  "options": {
    "workspace": "tmp/workspace",
    "repositoryUrl": "[email protected]:namespace/project.git",
    "url": "http://domain.dev",
    "deployTo": "tmp/deployTo",
    "ignores": [
      ".git",
      ".idea",
      "node_modules",
      "tmp",
      "public/app/uploads",
      "public/.htaccess"
    ],
    "keepReleases": 2,
    "branch": "develop",
    "shallowClone": true,
    "shared": {
      "baseDir": "shared",
      "dirs": [
        "public/app/uploads"
      ],
      "files": [
        ".env",
        "public/.htaccess"
      ]
    }
  },

  "staging": {
    "servers": "[email protected]",
    "deployTo": "/staging.domain.org",
    "url": "http://staging.domain.org"
  },

  "production": {
    "servers": "[email protected]",
    "deployTo": "/httpdocs",
    "url": "http://domain.org",
    "branch": "master"
  }
}

As you can see, i am deploying with shipit, using a chrooted user.
When shipit deploy to /staging.domain.org, the corresponding path for apache on my Plesk server is /var/www/vhosts/domain.org/staging.domain.org.
However, symlinks are read by apache. Your script, for example for the .htaccess file, create symlinks like this

.htaccess -> /staging.domain.org/shared/public/.htaccess

On my server configuration, this is not correct, and it should look like this instead :

.htaccess -> /var/www/vhosts/domain.org/staging.domain.org/shared/public/.htaccess

or

.htaccess -> ../../../shared/public/.htaccess

Maybe that in your configuration file, you could add an option to allow people like me which use a chrooted user to deploy, to prepend a custom path to symlinks you create.
It could look something like this

    "shared": {
      "baseDir": "shared",
      "prependSymlinkPaths": "/var/www/vhosts/domain.org",
      "dirs": [
        "public/app/uploads"
      ],
      "files": [
        ".env",
        "public/.htaccess"
      ]
    }

Do you think it is possible ? It would be really helpful.
Thanks a lot

tlartaud · 2015-04-09T09:57:57Z

I also tried to change "baseDir": "shared" to "baseDir": "/var/www/vhosts/domain.org/staging.domain.org/shared" but it does not work. Returns the following error

Warning: Command failed: /bin/sh -c ssh [email protected] "mkdir -p /var/www/vhosts/domain.org/staging.domain.org/shared/public/app/uploads"
mkdir: cannot create directory '/var/www': Permission denied
 Use --force to continue.

Which is normal, because i am connected with a chrooted user which do not have access to /var/www, because his chrooted path is /var/www/vhosts/domain.org/

I only need to prepend symlinks paths with my own custom path, which is only required for apache user.
Is that possible ? Thanks a lot.

timkelty · 2015-04-09T12:28:24Z

I see now, thanks.
Yes, the second example is what I was expecting (setting baseDir to /var/www/vhosts/domain.org/staging.domain.org/shared), but I see now your permissions problem.

I'll take a look at your PR and merge it in.

tlartaud · 2015-04-09T13:30:00Z

Thank you :)

timkelty · 2015-04-09T21:06:56Z

If I understand correctly, your issue is with the permissions on the mkdir -p command, right?

If you set your baseDir to /var/www/vhosts/domain.org/staging.domain.org/shared, wouldn't it work if you could optionally skip the shared:create-dirs? You could test this by running shipit shared:link.

That way, once, you could either create the dirs manually, or run shipit shared:dirs with the non-chrooted user, and then never do it again.

I'm just wondering if an option to skip the creating of the dirs every time is a better option, rather than adding such a specific option as you've suggested.

Let me know what you think.

timkelty · 2015-04-09T21:14:28Z

If that suggestion works, I'd rather add that option, as it's not so specific, and because I'm not thrilled about the fact that shipit-shared already hooks into published for you with no option not to.

So we might have an option that was onPublished that defaulted to shared, but in your case you would change it to shared:link, or it could be false for no deploy hook at all.

tlartaud · 2015-04-11T10:22:06Z

dude, do you know what a chroot is ?

"If I understand correctly, your issue is with the permissions on the mkdir -p command, right?"
What ?
No .

My issue is about paths written inside symlinks files that are not correct. Did you read my post. I d'ont know how i can explain this better than with full explanation + a pull request....

tlartaud · 2015-04-11T10:31:51Z

its NOT a permission issue. It is chroot.

Server structure

- / (root path for apache user)
- etc
- [ others folders ... ]
- var
  - www
     - vhosts
        - domain.org (Chroot path = chrooted_user cant go in any parent folder of this one)
           - etc
           - var
           - [others folders...]
           - staging.domain.org

As you can see in that structure, the chrooted user cant access any of the parent folder of domain.org.

"That way, once, you could either create the dirs manually, or run shipit shared:dirs with the non-chrooted user, and then never do it again."
If i use shipit-shared, this is for auto-generating the base structure. if it doesnt work, or if i need to manually create some stuffs, the node will be useless for me and i may stop use it, and create my own.

tlartaud · 2015-04-11T10:41:54Z

"I'm just wondering if an option to skip the creating of the dirs every time is a better option, rather than adding such a specific option as you've suggested."

i think that if i add /var/www/vhosts in baseDir, i will always have some problems every time i try to run a command from your script, because /var/www/vhosts DOES NOT EXISTS in the chrooted environment. It's not that the user does not have the permission to access the folder.

thats why i suggested an option specific to chrooted environments, because if does ONLY affect datas INSIDE symlinks files.
When connected as a chrooted user, the baseDir to the Symlink is correct and is /staging.domain.org/current/public/.htaccess, however, his content should be /var/www/vhosts/domain.org/staging.domain.org/shared/public/.htaccess
The baseDir is actually correct if i let the option baseDir: "shared". I should not change it.

I know theses explanations are not very clear, and i aploogize for that. i'm not really good in english, and i try to explain myself as I can, sorry if I confused you.

timkelty · 2015-04-13T15:26:34Z

I know what a chroot is, I was just misunderstanding your issue.

I think it might make more sense just to simplify the options a bit.

shipit.config.shared.path would now just be the full path to the remote shared dir, defaulting to path.join(shipit.config.deployTo, 'shared')
shipit.config.shared.symlinkPath would be the source for the created symlinks, defaulting to shipit.config.shared.path. In your case, this is the option you'd want to change:

    "shared": {
      "symlinkPath": "/var/www/vhosts/domain.org/shared/",
      "dirs": [
        "public/app/uploads"
      ],
      "files": [
        ".env",
        "public/.htaccess"
      ],
      "staging": {
        "servers": "[email protected]",
        "deployTo": "/staging.domain.org",
        "url": "http://staging.domain.org"
      },
    }

This config should mkdir -p everything in /staging.domain.org/shared/, but the source path for the created symlinks would be /var/www/vhosts/domain.org/shared/.

Does that look like it will work for you?
88b6a3b

tlartaud · 2015-04-16T10:49:03Z

Hey there,

I just tried to deploy a new fresh website with your last node version.
It now works like a charm, and i really appreciate it. Thanks a lot.

The only "bug" that is still remaining, is that symlinks are not created if the source path (eg. /staging.domain.com/shared/public/.htaccess) does not exists.
Wouldn't it be a great idea to keep the same behavior for files and folders ?
I mean, when we create a symlink to a folder, the target folder is created if it does not exists. It could be really helpful that the files are also created if missing.
Or maybe that we could have options like below, to set an option for each file or folder, to tell shipit-shared if it should be created if unexistent.

"shared": {
      "baseDir": "shared",
      "symlinkPath": "/var/www/vhosts/domain.com",
      "dirs": [
        { "path": "public/app/uploads", "createIfMissing": true },
        { "path": "public/app/cache", "createIfMissing": true },
        { "path": "public/app/wp-rocket-config", "createIfMissing": false }
      ],
      "files": [
        { "path": ".env", "createIfMissing": true },
        { "path": "public/.htaccess", "createIfMissing": true },
        { "path": "public/app/advanced-cache.php", "createIfMissing": false }
      ]
    }

What do you think ?

PS : Anyway, it actually works if i create all the shared files before deploying, but it could be great to have the symlinks created even if the linked file does not exist, in some cases.

Thanks a lot for your updates and time.

tlartaud · 2015-04-16T11:00:30Z

It also give me another idea, you could really improve our development workflow, by allowing us to upload the missing files from our local environment, so we could have some option like below

"shared": {
      "baseDir": "shared",
      "symlinkPath": "/var/www/vhosts/domain.com",
      "dirs": [
        { "path": "public/app/uploads", "createIfMissing": true },
        { "path": "public/app/cache", "createIfMissing": true },
        { "path": "public/app/wp-rocket-config", "createIfMissing": false }
      ],
      "files": [
        { "path": ".env", "createIfMissing": true, "localPath": "remotes_files/production/.env" },
        { "path": "public/.htaccess", "createIfMissing": true, "localPath": "remotes_files/production/public/.htaccess"  },
        { "path": "public/app/advanced-cache.php", "createIfMissing": false}
      ]
    }

This could be really awesome, because with that options, we don't even need to connect via SSH or FTP to our server to copy or create the missing and required files into the shared folder.

What do you think about this ? :)

tlartaud · 2015-04-16T11:32:31Z

Well,

I think i spoked to fast ..

Checkout 7-chroot branch, ran npm install, deployed.
Absolutely no symlink are created for the files. However, it work for the folders.

My complete config file

{
  "options": {
    "workspace": "tmp/workspace",
    "repositoryUrl": "[email protected]:namespace/project.git",
    "url": "http://domain.dev",
    "deployTo": "tmp/deployTo",
    "ignores": [
      ".git",
      ".idea",
      "node_modules",
      "tmp",
      "public/app/uploads",
      "public/.htaccess",
      "public/app/cache",
      "public/app/advanced-cache.php",
      "public/app/wp-rocket-config"
    ],
    "keepReleases": 2,
    "branch": "develop",
    "shallowClone": true,
    "shared": {
      "baseDir": "shared",
      "symlinkPath": "/var/www/vhosts/domain.com",
      "dirs": [
        "public/app/uploads",
        "public/app/cache",
        "public/app/wp-rocket-config"
      ],
      "files": [
        ".env",
        "public/.htaccess",
        "public/app/advanced-cache.php"
      ]
    }
  },

  "staging": {
    "servers": "[email protected]",
    "deployTo": "/staging.domain.com",
    "url": "http://staging.domain.com"
  },

  "production": {
    "servers": "[email protected]",
    "deployTo": "/httpdocs",
    "url": "http://domain.com",
    "branch": "master"
  }
}

Am i doing something wrong ? As i can see in your last post, you wrapped the "staging" array into "shared", but i think its a mistake, right ?

The result on remote server

bash-4.3$ cd public
bash-4.3$ ls -al
total 24
drwxrwxr-x 4 chrooted_user psacln 4096 Apr 16 11:24 .
drwxrwxr-x 6 chrooted_user psacln 4096 Apr 16 11:24 ..
drwxrwxr-x 6 chrooted_user psacln 4096 Apr 16 11:24 app
-rwxrwxr-- 1 chrooted_user psacln  124 Apr 16 11:24 index.php
drwxr-xr-x 5 chrooted_user psacln 4096 Apr 16 11:24 wp
-rwxrwxr-- 1 chrooted_user psacln  434 Apr 16 11:24 wp-config.php
bash-4.3$ cd app
bash-4.3$ ls -al
total 28
drwxrwxr-x  6 chrooted_user psacln 4096 Apr 16 11:24 .
drwxrwxr-x  4 chrooted_user psacln 4096 Apr 16 11:24 ..
lrwxrwxrwx  1 chrooted_user psacln   50 Apr 16 11:24 cache -> /staging.domain.com/shared/public/app/cache
drwxrwxr-x  2 chrooted_user psacln 4096 Apr 16 11:23 languages
drwxrwxr-x  2 chrooted_user psacln 4096 Apr 16 11:23 mu-plugins
drwxrwxr-x 22 chrooted_user psacln 4096 Apr 16 11:23 plugins
drwxrwxr-x  3 chrooted_user psacln 4096 Apr 16 11:23 themes
lrwxrwxrwx  1 chrooted_user psacln   52 Apr 16 11:24 uploads -> /staging.domain.com/shared/public/app/uploads
lrwxrwxrwx  1 chrooted_user psacln   61 Apr 16 11:24 wp-rocket-config -> /staging.domain.com/shared/public/app/wp-rocket-config

As you can see, symlinks for folders are created, but the symlink for public/.htaccess is missing.
However, the .htaccess file on my shared folder exists

bash-4.3$ cd /staging.domain.com/shared/public
bash-4.3$ ls -al
total 16
drwxr-xr-x 3 chrooted_user psacln 4096 Apr 16 10:34 .
drwxr-xr-x 3 chrooted_user psacln 4096 Apr 16 10:31 ..
-rw-r--r-- 1 chrooted_user psacln  235 Apr 16 10:34 .htaccess
drwxr-xr-x 5 chrooted_user psacln 4096 Apr 16 10:40 app

Do you have an idea of what is going on ?
Thanks by advance

timkelty · 2015-04-16T11:34:48Z

Nope, but I'll look today.

This is a good excuse to get some tests in here so we don't keep running into this stuff, too :).

tlartaud · 2015-04-16T11:36:41Z

Ok, i know why it failed.

My console

Running "shared:link:files" task
Running "if [ -e /var/www/vhosts/domain.com/.env ]; then if ! [ -L /staging.domain.com/current/.env ]; then if [ -d /staging.domain.com/current/.env ]; then rm -rf /staging.domain.com/current/.env; fi; ln -s /var/www/vhosts/domain.com/.env /staging.domain.com/current/.env; fi; fi" on host "staging.domain.com".
Running "if [ -e /var/www/vhosts/domain.com/public/.htaccess ]; then if ! [ -L /staging.domain.com/current/public/.htaccess ]; then if [ -d /staging.domain.com/current/public/.htaccess ]; then rm -rf /staging.domain.com/current/public/.htaccess; fi; ln -s /var/www/vhosts/domain.com/public/.htaccess /staging.domain.com/current/public/.htaccess; fi; fi" on host "staging.domain.com".
Running "if [ -e /var/www/vhosts/domain.com/public/app/advanced-cache.php ]; then if ! [ -L /staging.domain.com/current/public/app/advanced-cache.php ]; then if [ -d /staging.domain.com/current/public/app/advanced-cache.php ]; then rm -rf /staging.domain.com/current/public/app/advanced-cache.php; fi; ln -s /var/www/vhosts/domain.com/public/app/advanced-cache.php /staging.domain.com/current/public/app/advanced-cache.php; fi; fi" on host "staging.domain.com".
Shared files symlinked on remote.

The wrong argument is

if [ -e /var/www/vhosts/domain.com/.env ];

and should be

if [ -e /domain.com/.env ];

EDIT : After thinking about it, i may correct myself, this previous test should be simply removed, as it force the symlink creation to already have a file in place before creating it. Thats why it fail in some cases. It could work for my .htaccess if i remove it from the ignoresarray, but as developpers, we often have some files like advanced-cache.php which are file generated by wp plugins, and which are gitignored, so theses files won't exists after an RSync copy. It could be great to create symlinks even if any file exists in that location path yet.

So, the first ran command should be

if ! [ -L /staging.domain.com/current/.env ]; then if [ -d /staging.domain.com/current/.env ]; then rm -rf /staging.domain.com/current/.env; fi; ln -s /var/www/vhosts/domain.com/.env /staging.domain.com/current/.env; fi

EDIT 2 : You should use if ! [ -h instead of if ! [ -L because -h will return true even if the targeted symlink (/shared/public/.htaccess) does not exists.

tlartaud · 2015-04-16T12:45:09Z

I think that ideally, the result command should be this

# If target file is not yet a symlink, OR if target is a symlink with wrong target path (usefull if symlink config changed or if we aldready have a symlink stored inside the repository)
if ( ! [ -h /staging.domain.com/current/.env ] ) || ( [ -h /staging.domain.com/current/.env ] && [ "$(readlink -n /staging.domain.com/current/.env)" != "/var/www/vhosts/domain.com/.env" ] ); then

    # If target exists (folder, file, symlink), remove it
    if [ -e /staging.domain.com/current/.env ]; then
        rm -rf /staging.domain.com/current/.env
    fi

    # Force symlink creation
    ln -sf /var/www/vhosts/domain.com/.env /staging.domain.com/current/.env
fi

Untested but it should work well.

tlartaud · 2015-04-16T13:57:52Z

Final solution tested and working for me. I changed link.js line 27 to 32 https://github.com/timkelty/shipit-shared/blob/issues/7-chroot/tasks/shared/link.js#L27 to this.

    return shipit.remote(
      sprintf('if ( ! [ -h %(target)s ] ) || ( [ -h %(target)s ] && [ $(readlink -n %(target)s ) != %(source)s ] ); then if [ -e %(target)s ]; then rm -r %(target)s; fi; ln -s %(source)s %(target)s; fi', {
        source: path.join(shipit.sharedSymlinkPath, filePath),
        target: path.join(shipit.currentPath, filePath)
      })
    );

And in init.js

shipit.sharedSymlinkPath = path.join( shipit.config.shared.symlinkPath, shipit.sharedPath) || shipit.sharedPath;

And you can replace all

link(path,true);
link(path,false);

by

link(path);

( -r argument on a file won't throw an error )

Alternatively, you can choose to not join paths in init.js, but you should warn users that they need to add the correct value in each environnments, like this

{
  "options": {
    ...,
    "shared": {
      "baseDir": "shared",
      "symlinkPath": "/var/www/vhosts/domain.com
      "dirs": [
        "public/app/uploads",
        "public/app/cache",
        "public/app/wp-rocket-config"
      ],
      "files": [
        ".env",
        "public/.htaccess",
        "public/app/advanced-cache.php"
      ]
    }
  },

  "staging": {
    ...,
    "deployTo": "/staging.domain.com",
    "shared": {
      "symlinkPath": "/var/www/vhosts/domain.com/staging.domain.com"
    }
  },

  "production": {
    ...,
    "deployTo": "/httpdocs",
    "shared": {
      "symlinkPath": "/var/www/vhosts/domain.com/httpdocs"
    }
  }
}

Everything is ok now

bash-4.3$ cd /staging.domain.com/current/public/app
bash-4.3$ ls -al
total 40
drwxrwxr-x  6 chrooted_user psacln 4096 Apr 16 13:55 .
drwxrwxr-x  4 chrooted_user psacln 4096 Apr 16 13:55 ..
lrwxrwxrwx  1 chrooted_user psacln   96 Apr 16 13:55 advanced-cache.php -> /var/www/vhosts/domain.com/staging.domain.com/shared/public/app/advanced-cache.php
lrwxrwxrwx  1 chrooted_user psacln   83 Apr 16 13:55 cache -> /var/www/vhosts/domain.com/staging.domain.com/shared/public/app/cache
drwxrwxr-x  2 chrooted_user psacln 4096 Apr 16 13:52 languages
drwxrwxr-x  2 chrooted_user psacln 4096 Apr 16 13:52 mu-plugins
drwxrwxr-x 22 chrooted_user psacln 4096 Apr 16 13:55 plugins
drwxrwxr-x  3 chrooted_user psacln 4096 Apr 16 13:52 themes
lrwxrwxrwx  1 chrooted_user psacln   85 Apr 16 13:55 uploads -> /var/www/vhosts/domain.com/staging.domain.com/shared/public/app/uploads
lrwxrwxrwx  1 chrooted_user psacln   94 Apr 16 13:55 wp-rocket-config -> /var/www/vhosts/domain.com/staging.domain.com/shared/public/app/wp-rocket-config

timkelty · 2015-04-16T17:46:20Z

Nice!

Hmmm....I think I'm going to opt to not do the path joining in init.js. A little more config, but it just seems more clear what is going on, that you can fully override the sharedPath and the sharedSymlinkPath.

timkelty · 2015-04-16T18:01:09Z

I like the creation/upload ideas you had:
#9
#10

timkelty · 2015-04-16T18:06:53Z

@tlartaud I pushed those changes, can you make sure they work for you?
https://github.com/timkelty/shipit-shared/tree/issues/7-chroot

tlartaud · 2015-04-17T09:13:48Z

"I like the creation/upload ideas you had"

Yep :)
At first, i created my own solution based on shipit which was working, until i found your node. I like the config array implementation you did so i switched to your solution. I had also implemented a system to let me upload the files if they exist locally. This was really usefull.
I just hope i can do so with your node soon :)

tlartaud · 2015-04-17T10:17:29Z

PS : I'm sorry, i just deleted my 2 olds comments. i was looking at the 7-chroot and didn't see the changes you made. Sorry.

I will make a last test with your merged version and let you know if it works for me.
Thanks for listening to me :) i appreciate a lot.

tlartaud · 2015-04-17T12:12:28Z

Removed my node_modules folder
Ran npm install
Checked out shipit-shared master branch in node_modules + npm install
Ran grunt publish:staging
There were still some hard errors that i've fixed. Use this code and everything will be fine

return shipit.remote(
      sprintf('if ( ! [ -h "%(target)s" ] ) || ( [ -h "%(target)s" ] && [ $(readlink -n "%(target)s" ) != "%(source)s" ] ); then if [ -edfh "%(target)s" ]; then rm -r "%(target)s"; fi; ln -s "%(source)s" "%(target)s"; fi', {
        source: path.join(shipit.sharedSymlinkPath, filePath),
        target: path.join(shipit.releasesPath, shipit.releaseDirname, filePath)
      })
    );

I fixed some typos errors in the bash command.

_Important:_
You can notice that i removed shipit.currentPath. Don't use this ! This is creating a just so LOL issue. deploy:clean is the last task, and is creating the symlink deployTo/current from the last release folder.
If you use the path deployTo/current BEFORE the deploy:clean task (and actually, shipit-shared task start before deploy:clean), you will not be working in the last release folder, but in the old one... HUH ^^

Success !

bash-4.3$ ls -al /staging.domain.com/current/public/app
total 40
drwxrwxr-x  6 chrooted_user psacln 4096 Apr 17 12:05 .
drwxrwxr-x  4 chrooted_user psacln 4096 Apr 17 12:05 ..
lrwxrwxrwx  1 chrooted_user psacln   97 Apr 17 12:05 advanced-cache.php -> /var/www/vhosts/domain.com/staging.domain.com/current/public/app/advanced-cache.php
lrwxrwxrwx  1 chrooted_user psacln   84 Apr 17 12:05 cache -> /var/www/vhosts/domain.com/staging.domain.com/current/public/app/cache
drwxrwxr-x  2 chrooted_user psacln 4096 Apr 17 12:05 languages
drwxrwxr-x  2 chrooted_user psacln 4096 Apr 17 12:05 mu-plugins
drwxrwxr-x 22 chrooted_user psacln 4096 Apr 17 12:05 plugins
drwxrwxr-x  3 chrooted_user psacln 4096 Apr 17 12:05 themes
lrwxrwxrwx  1 chrooted_user psacln   86 Apr 17 12:05 uploads -> /var/www/vhosts/domain.com/staging.domain.com/current/public/app/uploads
lrwxrwxrwx  1 chrooted_user psacln   95 Apr 17 12:05 wp-rocket-config -> /var/www/vhosts/domain.com/staging.domain.com/current/public/app/wp-rocket-config

The config file used

{
  "options": {
    "workspace": "tmp/workspace",
    "repositoryUrl": "[email protected]:project/namespace.git",
    "url": "http://domain.dev",
    "deployTo": "tmp/deployTo",
    "ignores": [
      ".git",
      ".idea",
      "node_modules",
      "tmp",
      "public/app/uploads",
      "public/.htaccess",
      "public/app/cache",
      "public/app/advanced-cache.php",
      "public/app/wp-rocket-config"
    ],
    "keepReleases": 2,
    "branch": "develop",
    "shallowClone": true,
    "shared": {
      "baseDir": "shared",
      "dirs": [
        "public/app/uploads",
        "public/app/cache",
        "public/app/wp-rocket-config"
      ],
      "files": [
        ".env",
        "public/.htaccess",
        "public/app/advanced-cache.php"
      ]
    }
  },

  "staging": {
    "servers": "[email protected]",
    "deployTo": "/staging.domain.com",
    "url": "http://staging.domain.com",
    "shared": {
      "symlinkPath": "/var/www/vhosts/domain.com/staging.domain.com/shared"
    }
  },

  "production": {
    "servers": "[email protected]",
    "deployTo": "/httpdocs",
    "url": "http://domain.com",
    "branch": "master",
    "shared": {
      "symlinkPath": "/var/www/vhosts/domain.com/httpdocs/shared"
    }
  }
}

I hope that helps

timkelty · 2015-04-17T12:31:02Z

@tlartaud Ok, I pushed that to master and issues/7-chroot, give it a shot.

Also - I don't think "baseDir": "shared", is doing anything in your config :)

tlartaud · 2015-04-17T12:32:25Z

Yep, you are totally right, i've just added it back for sample purposes.
I'll check out your last changes and let you know.

Thanks.

tlartaud · 2015-04-17T12:54:47Z

@timkelty sorry, last fix

    return shipit.remote(
      sprintf('if ( ! [ -h "%(target)s" ] ) || ( [ -h "%(target)s" ] && [ $(readlink -n "%(target)s" ) != "%(source)s" ] ); then rm -r "%(target)s" 2> /dev/null; ln -s "%(source)s" "%(target)s"; fi', {
        source: path.join(shipit.sharedSymlinkPath, filePath),
        target: path.join(shipit.releasesPath, shipit.releaseDirname, filePath)
      })
    );

tlartaud · 2015-04-17T13:19:59Z

@timkelty I am just wondering myself if it wouldn't be better to create symlinks which looks like this

target : /staging.domain.com/releases/8957436508605
source : /staging.domain.com/current or /var/www/vhosts/domain.com/staging.domain.com/current

I didn't test yet, but maybe that if the source symlink include current instead of releases/8957436508605, it will prevent the release name to be registered in the database (for example when a php script read a file and store his dirname() path in the DB). So, all the dirname paths registered in the DB would contain only current and not releases/8957436508605 in path.

Actually, this is just a supposition, i didn't test, and i don't know if a <?php dirname($symlink_path); ?> return the full path, or simply the path included in the symlink file.
That could help users to not have wrong paths problem when pushing a new release with an unchanged database.

timkelty · 2015-04-17T13:30:58Z

@tlartaud Ok, Pushed that last fix.

tlartaud · 2015-04-17T14:13:46Z

@timkelty Thanks a lot. It works fine now.

I've tested the thing about dirname(), and it always return the full path, so we can't make php use current instead of releases/8957436508605, it's up to the user to make sure that any release number is stored inside the database ...

Maybe that you should prefer this in init.js so the option is not required (i think grunt will throw an error if shipit.config.shared.symlinkPath does not exist)

shipit.sharedSymlinkPath = ( 'symlinkPath' in shipit.config.shared ) ? shipit.config.shared.symlinkPath : shipit.sharedPath;

It will prevent users that update your node without adding the new (optional) symlinkPath config to get an error.

tlartaud · 2015-04-17T14:18:08Z

Just a strange behaviour,
i made a mistake in my config file, and i had

"shared": {
      "symlinkPath": "/var/www/vhosts/domain.com/staging.domain.com/current"
    }

I replaced it with the correct one and merged and pushed to all branches

"shared": {
      "symlinkPath": "/var/www/vhosts/domain.com/staging.domain.com/shared"
    }

The strange thing is that i had to totally remove current/ and releases/ folders on my remote and deploy again, otherwise, each time i deployed, i got symlinks created with current value instead of shared. Really strange. Dunno why, but i think it should be more tested :)

Anyway, thanks a lot.

timkelty · 2015-04-17T14:21:20Z

@tlartaud Grunt, or JS in general should be fine with

shipit.sharedSymlinkPath = shipit.config.shared.symlinkPath || shipit.sharedPath;

Since shipit.config.shared is always going to be defined at that point, shipit.config.shared.symlinkPath will just be undefined, and it will take shipit.sharedPath

timkelty · 2015-04-17T14:23:03Z

Glad we worked this one out!
Next on my list is to get some test in here so working through this won't be so tedious.

timkelty · 2015-04-17T14:23:42Z

@tlartaud in the link method, should the rm -r be rm -rf?

tlartaud · 2015-04-17T14:30:29Z

@timkelty

yeah, maybe for rm -rf. It depends on the server configuration. In my case, i am connected as a chrooted user, so all the file are owned by him, so i don't have this problem, rm -r will always be working for me, maybe not for others servers configurations.
yeah, im sure you are right for shipit.sharedSymlinkPath = shipit.config.shared.symlinkPath || shipit.sharedPath;, i'm not a developper, i am mainly a designer, and even if i am used to develop, i do not know all the basics ^^. If this syntax does not throw an error when option is missing, my last comment about it is useless ^^

tlartaud mentioned this issue Apr 9, 2015

Adding prependSymlinksPath #8

Closed

timkelty pushed a commit that referenced this issue Apr 16, 2015

Suggested changes to link: #7 (comment)

610bb4a

timkelty mentioned this issue Apr 16, 2015

Option to create files if they don't exist #10

Closed

timkelty closed this as completed in 88b6a3b Apr 16, 2015

timkelty reopened this Apr 16, 2015

tlartaud closed this as completed Apr 17, 2015

Symlink target issue #7

Symlink target issue #7

Comments

tlartaud commented Mar 23, 2015

timkelty commented Mar 23, 2015

timkelty commented Mar 24, 2015

timkelty commented Mar 24, 2015

tlartaud commented Mar 25, 2015

timkelty commented Mar 25, 2015

tlartaud commented Apr 9, 2015

tlartaud commented Apr 9, 2015

timkelty commented Apr 9, 2015

tlartaud commented Apr 9, 2015

timkelty commented Apr 9, 2015

timkelty commented Apr 9, 2015

tlartaud commented Apr 11, 2015

tlartaud commented Apr 11, 2015

tlartaud commented Apr 11, 2015

timkelty commented Apr 13, 2015

tlartaud commented Apr 16, 2015

tlartaud commented Apr 16, 2015

tlartaud commented Apr 16, 2015

timkelty commented Apr 16, 2015

tlartaud commented Apr 16, 2015

tlartaud commented Apr 16, 2015

tlartaud commented Apr 16, 2015

timkelty commented Apr 16, 2015

timkelty commented Apr 16, 2015

timkelty commented Apr 16, 2015

tlartaud commented Apr 17, 2015

tlartaud commented Apr 17, 2015

tlartaud commented Apr 17, 2015

timkelty commented Apr 17, 2015

tlartaud commented Apr 17, 2015

tlartaud commented Apr 17, 2015

tlartaud commented Apr 17, 2015

timkelty commented Apr 17, 2015

tlartaud commented Apr 17, 2015

tlartaud commented Apr 17, 2015

timkelty commented Apr 17, 2015

timkelty commented Apr 17, 2015

timkelty commented Apr 17, 2015

tlartaud commented Apr 17, 2015