Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allow multiple spec files to be provided to XcodeGen #1270

Merged
merged 5 commits into from
Nov 2, 2022
Merged

allow multiple spec files to be provided to XcodeGen #1270

merged 5 commits into from
Nov 2, 2022

Conversation

skofgar
Copy link
Contributor

@skofgar skofgar commented Sep 30, 2022 •
edited
Loading

TL;DR: Generate 10 projects in less than 1 second instead of 8 by allowing XcodeGen to accept multiple project files instead of generating them one by one.

When generating multiple projects it can take a long time. For example, when generating 10 projects by invoking XcodeGen 10 times sequentially via bash, the script takes about 8 seconds, even with caching turned on.

Why is this a problem?

When switching branches and checking out new branches of a repo that contains multiple XcodeGen project files, this causes substantial delays and slows down engineers.

Investigation

Looking at the profiler, it looks like the slowest part is just starting up XcodeGen. The "initialization" period can take up to 0.5 seconds.

Proposed Solution

Allowing XcodeGen to accept multiple spec files, can significantly speed up the generation. In our tests generation of the same 8 projects was less than 3 seconds without caching and with caching less than 1 second (sometimes just 0.2 seconds).

Approach

  • allow specs to be provided as a comma separated list of specs instead of just one
xcodegen --spec path/to/first/project.yml,path/to/another/project.yml --use-cache

@skofgar skofgar marked this pull request as ready for review September 30, 2022 21:47
@skofgar
Copy link
Contributor Author

skofgar commented Oct 7, 2022

Hi @yonaskolb may I request a review for this PR?

yonaskolb
yonaskolb approved these changes Nov 2, 2022
View reviewed changes
Copy link
Owner

@yonaskolb yonaskolb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool, thanks @skofgar!

@yonaskolb yonaskolb merged commit 3e9fd04 into yonaskolb:master Nov 2, 2022
mcfans added a commit to iftechio/XcodeGen that referenced this pull request Feb 22, 2023
@mcfans
Squashed commit of the following:
b4e0f01 
commit bb02e01
Author: takeshi-1000 <[email protected]>
Date:   Thu Dec 8 05:15:58 2022 +0900

    Fixed logic related to order of elements in localization directory Path array

commit bad37c9
Author: takeshi-1000 <[email protected]>
Date:   Thu Dec 8 05:09:44 2022 +0900

    Revert "Fixed some logic in PBXVariantGroup."

    This reverts commit ea402c8.

commit ea402c8
Author: takeshi-1000 <[email protected]>
Date:   Wed Dec 7 09:23:02 2022 +0900

    Fixed some logic in PBXVariantGroup.
    Try to make the logic simpler so that an error does not occur under the Linux environment.

commit b9359fb
Author: takeshi-1000 <[email protected]>
Date:   Tue Dec 6 19:00:20 2022 +0900

    fix fixtures test

commit ca25eca
Author: takeshi-1000 <[email protected]>
Date:   Tue Dec 6 12:33:23 2022 +0900

    fix compilation error in ci enviroment

commit a40341f
Author: takeshi-1000 <[email protected]>
Date:   Mon Dec 5 21:05:49 2022 +0900

    add test

commit cb28c35
Author: takeshi-1000 <[email protected]>
Date:   Mon Dec 5 21:05:41 2022 +0900

    update SourceGenerator, PBXProjGenerator, SourceType

    ## SourceGenerator
    - adapt SourceGenerator to TargetSourceFilterable
    - refactor variant group logic
    - add logic where you can add target membership to another target

    ## PBXProjGenerator
    - apply. new PBXProjGenerator

    ## SourceType
    - add new sourceType

commit 8a33b01
Author: takeshi-1000 <[email protected]>
Date:   Mon Dec 5 21:05:08 2022 +0900

    add PBXVariantGroupGenerator, TargetSourceFilterable

commit e7f7537
Author: Mathieu Olivari <[email protected]>
Date:   Thu Nov 3 01:05:46 2022 -0700

    Fix includes related issues and improve their performances (yonaskolb#1275)

    * Fix recursive include path when relativePath is not set

    If relativePath is not set on a particular include, the first level of
    include will currently work, but starting at the second level of
    iteration, the computed include path will fail as relativePath will be
    appended over and over onto the filePath. We're fixing that recursion
    problem here and adding the corresponding tests to make sure it doesn't
    happen again.

    * Include projectRoot in include paths

    The projectRoot setting (when specified) is currently ignored when
    computing the include paths. We're fixing that in that commit.

    * Use memoization during recursive SpecFiles creation

    SpecFile objects are created by recursive through includes. On a large
    project with programatically generated SpecFile, it is not rare to have
    hundreds of SpecFiles, creating a large web of include dependencies.
    In such a case, it is not rare either for a particular SpecFile to be
    included by multiple other SpecFiles. When that happens, XcodeGen
    currently creates a SpecFile object every time a SpecFile gets included,
    which can lead to an exponential growth of includes.

    I have seen hundreds of files being turned into hundred of thousands of
    SpecFile object creations, which leads to an impractical XcodeGen run of
    tens of minutes.

    This change adds memoization during SpecFile recursion, in order to
    reuse the previously created SpecFiles, if available, instead of
    re-creating them.

    * Update CHANGELOG.md

    Add the following changes to the changelog:
    * b97bdc4 - Use memoization during recursive SpecFiles creation
    * a6b96ad - Include projectRoot in include paths
    * 557b074 - Fix recursive include path when relativePath is not set

commit 3e9fd04
Author: Roland <[email protected]>
Date:   Wed Nov 2 07:42:22 2022 +0100

    allow multiple spec files to be provided to XcodeGen (yonaskolb#1270)

    * allow spec to be a comma separated list of specs instead of one

    * update readme and --spec command documentation

    * update Changelog

    * print project name

commit 87d7c7e
Author: Yonas Kolb <[email protected]>
Date:   Sat Oct 1 20:50:18 2022 +1000

    Update CHANGELOG.md

commit 435c194
Author: SofteqDG <[email protected]>
Date:   Sat Oct 1 11:23:09 2022 +0300

    Extend possible paths for SettingsPresets (yonaskolb#1135)

    * Search for presets in Bundle.main.resourcesPath dir (if exists)

commit ed5ec74
Author: Craig Siemens <[email protected]>
Date:   Wed Sep 28 22:08:37 2022 -0600

    Added scheme generation for aggregate targets (yonaskolb#1250)

    * Updated SchemeGenerator to generate schemes for all projectTargets.

    * Added changelog entry

commit 6f33172
Author: Bobby Sudekum <[email protected]>
Date:   Fri Sep 9 01:43:39 2022 -0700

    Add enableGPUFrameCaptureMode to Scheme (yonaskolb#1251)

commit ebf70f1
Author: Yonas Kolb <[email protected]>
Date:   Fri Aug 19 00:53:34 2022 +1000

    Update to 2.32.0

commit 594c67f
Author: freddi(Yuki Aki) <[email protected]>
Date:   Fri Aug 12 15:21:43 2022 +0900

    Add `enable` option for `include` to enable optional including for addtional spec (yonaskolb#1242)

    * add new option enable for include of spec

    * fix to see the environment variable when parsing include

    * add test for include with environment variable

    * fix how to parse boolean value

    * add spec about enable for include

    * add Change Log

    * fix the number of PR in changelog

    * fix include test to make more clear

    * fix test to focus enable option more

    * fix english error

    * fix to expand variable only one time

    * add new test case by setting environment object as NO

commit e9295f1
Author: Steven Sheldon <[email protected]>
Date:   Thu Aug 11 05:45:06 2022 -0700

    Fix profile action to not run frameworks (yonaskolb#1245)

    * Fix profile action to not run frameworks

    * Add PR number to changelog

    * Update CHANGELOG.md

    Co-authored-by: Yonas Kolb <[email protected]>

commit ac525a4
Author: Shinolr <[email protected]>
Date:   Tue Aug 9 22:32:33 2022 +0800

    remove redundant bracket (yonaskolb#1243)

commit 34f50d6
Author: Isaac Halvorson <[email protected]>
Date:   Mon Aug 1 17:27:14 2022 -0500

    Correct name of package in example yaml (yonaskolb#1240)

    Hey there, I just noticed that one of the example yaml snippets had the wrong package name specified.

commit ff552f3
Author: antonsergeev88 <[email protected]>
Date:   Sun Jul 31 11:33:20 2022 +0300

    Handle mlmodelc as a single unit (yonaskolb#1237)

    * Handle mlmodelc as a single unit

    * Add mlmodelc support in changelog

commit de2a537
Author: Yonas Kolb <[email protected]>
Date:   Sun Jul 24 16:10:15 2022 +1000

    Update to 2.31.0

commit 24572da
Author: Aleksei Sapitskii <[email protected]>
Date:   Sun Jul 24 09:08:33 2022 +0300

    Added duplicate dependencies validation (yonaskolb#1234)

    **Reason**
     - More strict validation of added dependencies

    **Contents**
     - Added changelog entry
     - Added check for duplicates in validation stage
     - Added test

commit da8aad0
Author: matsuji <[email protected]>
Date:   Thu Jul 21 20:25:34 2022 +0900

    Add a new CopyFilesBuildPhase, "Embed ExtensionKit Extensions" (yonaskolb#1230)

    * Embed ExtensionKit Extensions

    * Fix explicitFileType for extensionKit

    * Update ChangeLog

    * Fix if statement structure

    * Add a new example extension to Tests/Fixtures/TestProject/

    * Update Tests/Fixtures/TestProject/Project.xcodeproj

    * Comment out example for extension kit extension in Tests/Fixtures/TestProject/

    * Update Tests/Fixtures/TestProject/Project.xcodeproj

commit c1d5c65
Author: Yonas Kolb <[email protected]>
Date:   Sat Jul 16 16:57:26 2022 +1000

    Update to 2.30.0

commit c082bc0
Author: Aleksei Sapitskii <[email protected]>
Date:   Sat Jul 16 09:46:42 2022 +0300

    Fix XcodeGen building after XcodeProj update to 8.8.0 (yonaskolb#1228)

    * Fix XcodeGen building after XcodeProj update to 8.8.0

    **Reason**
    - XcodeProj has been updated and has API breaking changes

    **Content**
    - Added new enum case handling in `Linkage`
    - Renamed the enum case name for `XCWorkspaceDataFileRef.init`

    * add new product type to docs

    * update changelog

    Co-authored-by: Yonas Kolb <[email protected]>

commit 19817f3
Author: Luca Bartoletti <[email protected]>
Date:   Sat Jul 16 07:46:31 2022 +0100

    Fix `watchapp2-container` product name (yonaskolb#1219)

    The correct name is `application.watchapp2-container`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yonaskolb yonaskolb yonaskolb approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@skofgar @yonaskolb