Force:source:pull does not respect source tracking locations outside of "packagedirectory/main/default/"

[ImJohnMDaniel]

Summary

Since the incorporation of the new Source Tracking logic, we have seen that when we have a project with metadata in the source code repo and that metadata is outside of packagedirectory/main/default/ folder, when you make a change to that metadata inside the scratch org, the next time you run force:source:pull, the CLI will extract the metadata again and place it under the packagedirectory/main/default/ folder.

Steps To Reproduce:

Repository to reproduce: dreamhouse-lwc -- ImJohnMDaniel's fork

The issue is not reproducible on the standard dreamhouse-lwc app because all source for that project exists under the packagedirectory/main/default/ folder.

I have created a fork of the dreamhouse-lwc repo, then moved all source code of the project up one level/out of the "default" folder to demonstrate the issue.

1. Clone the dreamhouse-lwc -- ImJohnMDaniel's fork repo locally
2. run the "bin/install-scratch" scripts
3. Inside the scratch org, find the TestPropertyController Apex class file
4. Edit that file from the scratch org UI. Place a simple comment in the class or some other benign change.
5. Save the changed Apex file
6. Execute CLI command force:source:pull

It's at this point, you should see a duplicate of the TestPropertyController Apex class file now show up in the force-app/main/default/classes/ folder. The original copy of the Apex class is still present in the force-app/test/classes/ folder.

Expected result

The CLI should not create duplicates of existing metadata in the packagedirectory/main/default/ folder when that metadata already exists outside of the "default" folder.

Actual result

The CLI is creating duplicates of existing metadata in the packagedirectory/main/default/ folder.

System Information

{
        "cliVersion": "sfdx-cli/7.148.0",
        "architecture": "darwin-x64",
        "nodeVersion": "node-v16.14.2",
        "pluginVersions": [
                "@dx-cli-toolbox/sfdx-toolbox-package-utils 0.8.4 (link) /Users/john/workspace/_cli-related/sfdx-toolbox-package-utils",
                "@dx-cli-toolbox/sfdx-toolbox-utils 0.1.3 (link) /Users/john/workspace/_cli-related/sfdx-toolbox-utils",
                "@oclif/plugin-autocomplete 1.2.0",
                "@oclif/plugin-commands 1.3.0 (core)",
                "@oclif/plugin-help 3.3.1 (core)",
                "@oclif/plugin-not-found 1.2.6 (core)",
                "@oclif/plugin-plugins 1.10.11 (core)",
                "@oclif/plugin-update 1.5.0 (core)",
                "@oclif/plugin-warn-if-update-available 1.7.3 (core)",
                "@oclif/plugin-which 1.0.4 (core)",
                "@salesforce/sfdx-plugin-lwc-test 0.1.7 (core)",
                "@salesforce/sfdx-scanner 2.13.1",
                "alias 2.0.0 (core)",
                "apex 0.11.0 (core)",
                "auth 2.0.2 (core)",
                "community 1.1.4 (core)",
                "config 1.3.30 (core)",
                "custom-metadata 1.1.0 (core)",
                "data 0.6.13 (core)",
                "generator 1.2.2 (core)",
                "info 2.0.0 (core)",
                "limits 2.0.0 (core)",
                "org 1.11.2 (core)",
                "salesforce-alm 54.2.0 (core)",
                "schema 2.0.0 (core)",
                "sfdmu 4.13.4",
                "sfdx-cli 7.148.0 (core)",
                "shane-sfdx-plugins 4.43.0",
                "├─ @mshanemc/plugin-streaming 1.1.7",
                "└─ @mshanemc/sfdx-sosl 1.1.0",
                "signups 1.0.0 (core)",
                "soqlx-opener 0.1.12",
                "source 1.9.3 (core)",
                "telemetry 1.4.0 (core)",
                "templates 54.3.0 (core)",
                "trust 1.1.0 (core)",
                "user 1.7.1 (core)"
        ],
        "osVersion": "Darwin 21.4.0"
}

Additional information

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[ImJohnMDaniel]

This is definitely a breaking change with the new version of the source-tracking code base. I have various dev teams seeing this issue. They are using versions 7.145, 7.146, and 7.148. The only developer not seeing this behavior is currently on version 7.123

[ImJohnMDaniel]

When I switch back to force:source:legacy:push/pull commands, the CLI behaves as expected -- an edit to the TestPropertyController Apex class file in the org does not cause the creation of a new Apex file under the "default" folder. Instead, the original file is updated.

This is definitely a breaking change

[uip-robot-zz]

This issue has been linked to a new work item: W-11047555

[ImJohnMDaniel]

@mshanemc -- I have been testing with v7.149.0 and it does appear to have corrected this issue. Thanks for the help!

Please feel free to close the issue at your discretion

[iowillhoit]

Thanks @ImJohnMDaniel, sfdx version 7.149.1 was promoted to latest yesterday. Closing this issue now 👋

[ImJohnMDaniel]

Yup. Saw that too. Thanks for the help!!