Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Suggestion] Support for multiple projects with deliverable output #101

Closed
wants to merge 6 commits into from

Conversation

Vidarls
Copy link
Contributor

@Vidarls Vidarls commented Oct 16, 2014

Sometimes it is reasonable to collect multiple projects that creates separate deliverables in the same solution.
For our case the concrete use case is feature verticals. I.E We try to do narrow verticals with both server side, client side (and some times other stuff) together. Some UI projects will then depend on several of the Client packages from these vertical projects in order to function.

Currently the scaffolding project focuses on single project libraries. This suggestion implements support for multiple projects with separate deliverables.

Overview of changes:

  1. Each project now uses the default output dirs from Visual Studio (bin/Debug, bin/Release), the build process copies the binaries to /bin/[projectName] during build.
  2. Documentation generation of reference docs is modified to use the new subfolder structure in the bin folder
  3. The nuget task now iterates through the subfolders of the bin folder and generates one nuget package for each project
  4. The nuget task now automatically includes any nuget dependencies based on the paket.referense file of each project (if present), and adds it to the package.

These changes should not be breaking for single project libraries (biggest change is one extra folder layer in bin folder), but it enables basic multi project.

Todo

This is just an initial suggestion, mainly to get some feedback on whether I'm way off scope (and should keep this stuff in my own fork) or not. So some stuff is still pending, and some changes may need to be made.

  1. The nuget dependency thingy might belong in a FAKE.PaketHelper rather than in the project scaffold
  2. There is no support for projects with executable deliverables (.exe files)
  3. I need to figure out a way of adding nuget dependencies for referenced projects:
    If a project references another project in the solution. (IE the UI project references the messaging project) you don't want to have to deal with nuget during dev, but when some other project references the UI project, it should have a dependency on the messaging deliverable.

Allows for multiple projects in the src folder, each can be released as a nuget package
Nuget packaging now automatically infers dependencies based on paket settings.
@forki
Copy link
Member

forki commented Oct 16, 2014

Currently I'm trying to document the current status of the project - see
http://fsprojects.github.io/ProjectScaffold/index.html
I think we should wait until I'm ready, then we have hopefully the same
understanding on how things work,
Then we improve it with your ideas. I think we need to iron out a lot of
use cases in regards of C# projects.

2014-10-16 15:35 GMT+02:00 Vidar Løvbrekke Sømme [email protected]:

Sometimes it is reasonable to collect multiple projects that creates
separate deliverables in the same solution.
For our case the concrete use case is feature verticals. I.E We try to do
narrow verticals with both server side, client side (and some times other
stuff) together. Some UI projects will then depend on several of the Client
packages from these vertical projects in order to function.

Currently the scaffolding project focuses on single project libraries.
This suggestion implements support for multiple projects with separate
deliverables.

Overview of changes:

  1. Each project now uses the default output dirs from Visual Studio
    (bin/Debug, bin/Release), the build process copies the binaries to
    /bin/[projectName] during build.
  2. Documentation generation of reference docs is modified to use the
    new subfolder structure in the bin folder
  3. The nuget task now iterates through the subfolders of the bin
    folder and generates one nuget package for each project
  4. The nuget task now automatically includes any nuget dependencies
    based on the paket.referense file of each project (if present), and adds it
    to the package.

These changes should not be breaking for single project libraries (biggest
change is one extra folder layer in bin folder), but it enables basic multi
project.

Todo

This is just an initial suggestion, mainly to get some feedback on whether
I'm way off scope (and should keep this stuff in my own fork) or not. So
some stuff is still pending, and some changes may need to be made.

  1. The nuget dependency thingy might belong in a FAKE.PaketHelper
    rather than in the project scaffold
  2. There is no support for projects with executable deliverables (.exe
    files)
  3. I need to figure out a way of adding nuget dependencies for
    referenced projects: If a project references another project in the
    solution. (IE the UI project references the messaging project) you don't
    want to have to deal with nuget during dev, but when some other project
    references the UI project, it should have a dependency on the messaging
    deliverable.

You can merge this Pull Request by running

git pull https://github.com/Vidarls/ProjectScaffold multiproject

Or view, comment on, or merge it at:

#101
Commit Summary

  • Support for multiple projects with deliverable output

File Changes

Patch Links:


Reply to this email directly or view it on GitHub
#101.

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 16, 2014

I currently have a forked branch that works with single project C# libraries (dll deliverables) planning to look into mixed solutions soon 😄

@forki
Copy link
Member

forki commented Oct 16, 2014

woah curly braces.

Joking aside the goal is to make this also the goto place for C# only projects.

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 16, 2014

Trying to encourage my team to use whatever tool is best, (particularly to experiment with F#) so having a low barrier for mixed solutions is important. (And not a lot is required to support it, it was easy to support C# only, I just haven't done the mixed part yet.)

@forki
Copy link
Member

forki commented Oct 16, 2014

yes what I mean is: we want the C# community to use this (and improve it for their needs).

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 16, 2014

It would seem I have a case of it works on my machine. @forki Can you have a look at the log and see if you can bring some light on what I am doing wrong? (not allowed to access the default bin folders it seems)

@forki
Copy link
Member

forki commented Oct 16, 2014

mhm looks like it's not a relative path

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 16, 2014

should probably have a leading . then.

* For dependencies between projects in the same solution, a nuget dependency to the generated package for the projects are added to the nuspec
* For projects with Exe output type, All files in the relevant bin folder are added to the nuget package, no dependencies. Cause dealing with dependencies in a deploy situation is not really fun.
@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 17, 2014

I know you are putting a lot of effort into docs now @forki , but this is stuff we're going to use internally anyway, and I'd like to share it in case any of it is interesting for the community.

Following are added:

Inter project dependencies

If any library type project have dependencies (project references) to other library projects in the same solution, the dll's are not included in generated nuget package, but a dependency is added to the nuspec.

Example: Simple.Data by Mark Rendle (https://github.com/markrendle/Simple.Data) have a core lib and multiple providers and adapters that have different interdependencies. Each of those are published as separate packages with nuget dependencies declared on each other.
This feature would automate the nuget package generation with the correct dependency settings.

Executable projects

When packing an application type project as a nuget it is usually for some sort of automated deploy. In my opinion, dependencies should be resolved at packaging stage (when you have some sort of oversight) and not at deploy time. The strategy for creating nuget packages when project output type is set to Exe is therefor different than for Libraries. For Exe type projects, all files in the bin folder are added to the nuget package, and no dependencies are added to the nuspec.

Concerns

  • A lot of assumptions are made ( from our internal point of view) so they need to be validated
  • A lot of code in the build template, if these are feature you want in the scaffolding, we should consider adding a helper lib, or seeing if anything belongs in Paket, or FAKE as helpers there.

@forki
Copy link
Member

forki commented Oct 17, 2014

I can assure you @agross and I are working on a solution to this. :-)
Generating nuspecs based on dependencies file and paket.references is
planned for paket 0.9 and fake vnext.

It's cool to see people are interested.
On Oct 17, 2014 2:48 PM, "Vidar Løvbrekke Sømme" [email protected]
wrote:

I know you are putting a lot of effort into docs now @forki
https://github.com/forki , but this is stuff we're going to use
internally anyway, and I'd like to share it in case any of it is
interesting for the community.

Following are added:
Inter project dependencies

If any library type project have dependencies (project references) to
other library projects in the same solution, the dll's are not included in
generated nuget package, but a dependency is added to the nuspec.

Example: Simple.Data by Mark Rendle (
https://github.com/markrendle/Simple.Data) have a core lib and multiple
providers and adapters that have different interdependencies. Each of those
are published as separate packages with nuget dependencies declared on each
other.
This feature would automate the nuget package generation with the correct
dependency settings.
Executable projects

When packing an application type project as a nuget it is usually for some
sort of automated deploy. In my opinion, dependencies should be resolved at
packaging stage (when you have some sort of oversight) and not at deploy
time. The strategy for creating nuget packages when project output type is
set to Exe is therefor different than for Libraries. For Exe type
projects, all files in the bin folder are added to the nuget package, and
no dependencies are added to the nuspec.
Concerns

  • A lot of assumptions are made ( from our internal point of view) so
    they need to be validated
  • A lot of code in the build template, if these are feature you want
    in the scaffolding, we should consider adding a helper lib, or seeing if
    anything belongs in Paket, or FAKE as helpers there.


Reply to this email directly or view it on GitHub
#101 (comment)
.

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 17, 2014

Good to hear, let me know if I can help out

@forki
Copy link
Member

forki commented Oct 17, 2014

Yes we will probably need people who are willing to test alpha versions and
also check if our ideas work "in the real world" (tm).
On Oct 17, 2014 3:10 PM, "Vidar Løvbrekke Sømme" [email protected]
wrote:

Good to hear, let me know if I can help out


Reply to this email directly or view it on GitHub
#101 (comment)
.

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 17, 2014

Do you have any idea of when you will get to these parts?

(I hate getting these questions myself, sorry) But If it is in the near future, like a couple of weeks, I'd rather spend some time helping out getting a nice open source version going, than hacking on my own private fork (that I then have to maintain myself).

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 17, 2014

Also, that build failed for some weird reason before it even got to testing the actual code.

@forki
Copy link
Member

forki commented Oct 17, 2014

We will discuss the strategy this weekend. I will then log some issues into
paket issue tracker. I will probably flag some as up-for-grabs.

Actually I think a basic version should not be too hard. We have most of
the functionality already in paket's internals.
On Oct 17, 2014 3:19 PM, "Vidar Løvbrekke Sømme" [email protected]
wrote:

Also, that build failed for some weird reason before it even got to
testing the actual code.


Reply to this email directly or view it on GitHub
#101 (comment)
.

Changed how libdirs are resolved for generating API docs with
the new bin folder hierarchy.
@forki
Copy link
Member

forki commented Oct 23, 2014

ccing @agross

I think we are ready to start this.
==> reading your ideas/code

// Copies binaries from default VS location to exepcted bin folder
// But keeps a subdirectory structure for each project in the
// src folder to support multiple project outputs
Target "CopyBinaries" (fun _ ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this step needed? I assume only for the test step, right? why not change the search pattern there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two reasons:

  1. You can add new projects without changing default settings.
  2. Even with the default settings, I like to have my binaries in one place after build

But this is all just for convenience based on peronal preferences, more than actual requirements.
Big thing is not to have all bins for all projects in a single messy bin folder with no proj structure

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are building opnionated solution layout so opions are good.

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 28, 2014

Thanks @forki. Works as a charm 😄

(Also, adding the scaffolding project as another remote, and merging, works really great. Thanks for the tip.)

| Minimum(semver) when (semver.ToString()) = "0" -> Minimum(getCurrentlyInstalledVersion p.Name resolved)
| version -> version

let getReferencedPackagesFromProject projectRefFile =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly, it gets me the names. But I also need any version restriction from paket.dependencies

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, just reading the reference file directly is actually less code than using GetPackageHull (as I do not require indirect dependencies)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes you are right that was the wrong one.
Please try fsprojects/Paket@667a1e6#diff-99b45a5cb27d99b9474be5250b7d8f99R16

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 29, 2014

Tried to simplify a bit now and found that:

  • To loose the xpath stuff from build.template I would need to get target framework from the project file (Paket)
  • To simplify the find-project-dependencies code It would be nice with some shortcut to get all project dependencies with version requirements.

Just had time for a quick look tonight, will continue tomorrow I hope.

@forki
Copy link
Member

forki commented Oct 30, 2014

I think we came to the point where the discussions is hard to follow.
Please resubmit the pull request with the current changes and we will start from there. Ok?

@forki
Copy link
Member

forki commented Oct 30, 2014

@Vidarls
Copy link
Contributor Author

Vidarls commented Oct 30, 2014

Thanks for all the good help, closing this while integrating all the new Paket goodness 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants