Make "Get package" available in REPL

[forki]

See fsharp/fsharp.github.io@2951570

How should it look like?

[rneatherway]

We were talking about enabling this natively in FSI. Perhaps something like:

#r "nuget://<package name>/<path>/assembly.dll

How do you see this integrating with Paket?

[forki]

Yes that would be the best solution, but that's really far away since it
has to go into compiler and then deployed. The Paket project could
contribute code for this.

But I'm more talking about a solution which could be enabled today.
Something like #r referencing a paket lib and calling functions.
On Sep 19, 2014 5:34 PM, "Robin Neatherway" [email protected]
wrote:

We were talking about enabling this natively in FSI. Perhaps something
like:

#r "nuget:////assembly.dll

How do you see this integrating with Paket?

—
Reply to this email directly or view it on GitHub
#124 (comment).

[rneatherway]

Yeah I agree, it might take a while. Providing an easy API from Paket is a great idea as a first step. Currently Paket is distributed as a single executable right? A simple approach might be:

#r "Paket.exe"
Paket.Fetch "FSharp.Data"
#r "<paket dir>/FSharp.Data.dll"

or similar, where Paket would just place all the fetched dlls in a particular directory. It's a shame that #r doesn't accept string variables, otherwise Paket could just return a string hiding the package directory structure to be passed onto #r.

[bartelink]

Any tab-completion-for-files story for #r ? (Paket keeps stuff like NuGet does in a packages off project root with a [versionless, in constrast to NuGet defaults] subdir per NuGet), i.e. the location might be
#r "../packages/FSharp.Data/FSharp.Data.dll"

[rneatherway]

Tab completion for the filesystem would be simple to add to FCS, but then needs to be exposed in every editor. If #r could just accept a string variable we could hide the filesystem details.

[bartelink]

Fair point. I was missing a /lib in there too (and relative paths are never fun). Would you envisage it as a good idea to have the area into which stuff gets fetched as being a temp folder or actually in a project folder (e.g. for fsi in VS) ? Whether that is necessary would influence whether the #r "Paket.exe" bit requires passing of a storage area. Assuming not, could it look like this?

`#r "Paket.exe"
#r <| Paket.Open "FSharp.Data" "lib/FSharp.Data.dll"
// OR // #r (Paket.Open "FSharp.Data" "lib/FSharp.Data.dll")
// OR // Paket.Open "FSharp.Data" "lib/FSharp.Data.dll" |> #r

[rneatherway]

No sadly you can only pass it a literal string.

Another option is for paket to have a command so that running paket repl or paket fsi from the command line would launch fsi with -r:<assembly> for each of the assemblies specified in paket.references or paket.dependencies. An optional argument could be used to select a particular .NET framework version, using the latest as the default. We could also look at loading the solution/project output assemblies if one is found in the current directory. The code I use for project inspection in FSharp.AutoComplete would handle some of this.

An editor can then easily be configured to use an alternative fsi, which in this case would be paket itself.

[bartelink]

I got that its only a literal but was working on the basis that such an impediment could easily be overcome using the Art of the Possible / SEP fields :)

@rneatherway Interesting idea. In principle the dependencies file is not important - it can have mutually exclusive packages and/or very broadly scoped packages.

Which leaves paket.references as the primary authority. In principle a paket-wrapped FSI could work but the following would need to be figured out:

1. Are we attempting to work on the fly in temp dirs? If so, target FW is a key ingredient and paket.references does not include versions either so hard to imagine that's sufficient info. Also a key aspect of many packages is source files (and paket is also v likely to have various ways of grabbing and consuming file(s) from gists/fssnip/github etc. (either finessed into nuget packages or not, TBD). So, Q: is dealing with this a) important and b) can much useful be achieved? (I think no)

Bottom line: Paket could extract a nupkg to a temp place but you have to wonder what's Paket-specific about that.

1. assuming we're talking about starting in a project context, is it possible to have supplied:
   • target fw
   • project (folder) path - would allow derivation of either projectname.fsproj.paket.references / paket.references
   • sln folder path and/or walk up to paket.dependencies file [which is assumed to be adjacent to packages folder and/or would refer to same]
     If it is, Paket can easily derive the relevant set of dlls and sources. Is it?

Side things are:

1. Would there be a neat way for a paket update/install to trigger invalidation / prompt for restart of session from in FSI ?
   • any command worthy of restarting FSI would also touch the .fsproj (which begs the question, could the IDE offer to restart FSI sessions when you say OK, reload project)
2. Does FSI lock DLLs or shadow copy? Paket only does in-place updates
3. Any typical paket install/update emits all useful info into the fsproj - can stuff not work from there?
4. Paket doesn't really have an equivalent of Install-Package X -Version Y - you add to deps and/or refs and tell it "make it so" by an overall install/update.

Bottom line is in a sln/proj case, the fsproj always has authoritative info so the info as a whole might as well be sourced there and as long as the DLLs are not being locked, paket can update v easily.

I personally would love to have a magic button to load/reload up my project in a FSI because having to copy refs to clipboard etc. makes it so much harder to Just Do.

Paket can then simply grow a "paket add" of some kiind which triggers download/install and a relevant reload as per above.

---

All of this is thinking aloud from someone that doesnt use FSI enough. @ptrelford Any ideas/dreams ? You tried Paket yet?

[rneatherway]

I got that its only a literal but was working on the basis that such an impediment could easily be overcome using the Art of the Possible / SEP fields :)

OK sorry! In that case it's a neat suggestion, but if we assume we can modify FSI, I would go the whole hog and add nuget/project support directly.

To answer some of your points/questions:

• I do not imagine working in a temp dir, but rather in the context of a particular solution, project or just paket.* file. If the context is a solution or project, this can be used to determine an exact set of assemblies to include. So, to your question "is it?", I believe it is.
• I believe FSI now supports shadow copy.
• Yes we could work directly from the fsproj, but the argument for it being in Paket is then slightly weaker, although I believe still worthwhile.
• Not sure about triggering updates/reloads of FSI. I agree it would be very nice to at least offer the chance to restart with new assemblies.

[bartelink]

@rneatherway Apologies for not replying earlier. I was hammocking #154 and #166

Firstly re

but the argument for it being in Paket is then slightly weaker,

I dont think anyone wants anything in Paket per se - people just want stuff to work. Paket is happy to helper and its nice is stuff is composable but it def doesn't 'want'/need it ? i.e. I dont know what you mean but assume you mean nothing!

It seems to me that the "magic import" #124 (comment) : would mesh well with #154 if it can project out packages (see also my monolog 'with' @jeffhandley in https://jabbr.net/#/rooms/fsharp today)

The problem with

#r "nuget://<package name>/<path>/assembly.dll

or

#r <| Paket.Open "FSharp.Data" "lib/FSharp.Data.dll"

Is

• its actually messy mutually exclusive subdirs under lib
• it doesnt address Source packages.
• it doesnt deal with >1 DLL

#154 and projecting as nupkgs would allow 2 broad approaches:

1. work off Paket's knowledge:

   #r <| Paket.Open "PACKAGENAME" "$SolutionFolder" "$(TargetPlatform)"

   Q: Could easily be >1 assembly, magic-#r would need to deal with that
   Q: wouldnt you want to be able to #load source items in it too ?

   So maybe a magic shellesque backtick operator in FSI could do:

   `$(SolutionRoot)\tools\paket.exe emitfsi "PACKAGENAME" "$SolutionFolder" "$(TargetPlatform)" sources=true allDlls=true`
   

   could expand the packages and emit:

   #r "../../packages/lib/net40/Dll1.Data.dll"  
   #r "../../packages/lib/net40/Dll2.Data.dll"
   #load "../../packages/content/helpers/helpers.fsx"  
   

2. Let Paket project into the .fsproj, run that via commandline and/or VS commandlined Ctrl-Alt-A to run paket install and work from there within VS:

   • shadow copying makes this doable
   • fsi reload is useful feature anyway
     Q: that doesnt give a good standalone FSI story
     Q: how does one mix fsx and DLLs then (i.e. have to go ferriting around in trees to find DLLs, copy those, then find fsxes)

---

So, circling back around, it seems to me that something in Paket like:

paket.exe emitfsx --sources

could generate something like

  #r "../../packages/lib/net40/Dll1.Data.dll"  
  #r "../../packages/lib/net40/Dll2.Data.dll"
  #load "../../packages/content/helpers/helpers.fsx"  

And then one could use that in commnandline fsi via cut and paste:

paket.exe emitfsx --sources | clip

And with a bit of work, if fsxs supported a #n or shelly backtick construct they could call out to paket a bit like this:

   #do "../../tools/paket.exe emitfsx FSharp.Data ---platform=$(TargetPlatform) -sources"

OR

   `../../tools/paket.exe emitfsx ---platform=$(TargetPlatform) FSharp.Data --sources`

where

• $(TargetPlatform) comes from VS
• --sources controls whether/how to include snippets etc

Thoughts?

[rneatherway]

I dont think anyone wants anything in Paket per se - people just want stuff to work. Paket is happy to helper and its nice is stuff is composable but it def doesn't 'want'/need it ? i.e. I dont know what you mean but assume you mean nothing!

If this feature works directly from the .fsproj then it isn't Paket-specific, hence the argument for it being incorporated into Paket as opposed to some other tool is weaker than a feature that uses the paket.* files.

It seems to me that the "magic import" #124 (comment) ...

I'm discounting all features that require modification to FSI. The turnaround time and upgrade delay would leave this feature unreliable for years to come.

I also am not a fan of VS-specific solutions. I usually use Emacs, and I think Paket could be a great way to continue to improve the .NET development experience on Unix.

That said, I think emitting an fsx with the reference closure and optionally launching fsi including that fsx is a good option. I have an MSBuild task here that actually constructs such an fsx as part of the build. I also have code in the Emacs support for F# that could be used to do the same thing.

[bartelink]

If this feature works directly from the .fsproj then it isn't Paket-specific, hence the argument for it being incorporated into Paket as opposed to some other tool is weaker than a feature that uses the paket.* files.

Ah, I was thinking of tool as a standalone exe / DLL. In the case of emitting to the fsproj, I'd picture the things picking it up being the e.g. VFPT which I don't see as fitting the term "tool" - I call that "integrated into one or more plugins and/or the project system". (Not arguing, just explaining why the comment didnt initially make sense to me)

AIUI #166 would deal with the files being present in the file system and in the project

I also am not a fan of VS-specific solutions. I usually use Emacs, and I think Paket could be a great way to continue to improve the .NET development experience on Unix.

I appreciate that. I'm effectively in the same camp despite on the surface being in a different world. I like and use VS but I like having a proper build too and I cant imagine using any variant of the NuGet tooling in VS again - i.e. Package Manager Console and its stuff makes no sense.

The view I take is that the more general/composable something like this is, the wackier a bot @mathias-brandewinder and his like can write .

I'm discounting all features that require modification to FSI. The turnaround time and upgrade delay would leave this feature unreliable for years to come.
That said, I think emitting an fsx with the reference closure and optionally launching fsi including that fsx is a good option. I have an MSBuild task here that actually constructs such an fsx as part of the build. I also have code in the Emacs support for F# that could be used to do the same thing.

Then perhaps... is there/should we request a #include facility in FSI that would allow an FSX to pull in another one?

If you had that, then you could tell paket install to generate fsxi files per project. These files would have #r and/or #load statements. They could be genned into the packages folders.

Then .fsx files can #include "../../packages/FSharp.Data/all.fsix"

In the meantime and/or for generalized builds you can have a task or a script that will CPP-stlye preprocess fsx files if it's not there yet.

i.e. Just because everyone can't use it now shouldnt rule out considering it as part of the solution.

---

Another thing to consider in all this is that a package can have nested dependencies - e.g. if you want to stand up an in memory owin host, the unit of what you want loaded into your fsx context might be 15 nupkgs worth of stuff (maybe source, maybe DLLs, prob a combo).

[rneatherway]

Then perhaps... is there/should we request a #include facility in FSI that would allow an FSX to pull in another one?

You can use #load, this is what I do. My MSBuild task generates a __References__.fsx file and I then put #load "__References__.fsx" at the top of each script file. So this wouldn't require an upgrade to FSI.

I would like to see a native solution in FSI for other purposes as well, but my opinion is that it is best to keep these proposals separate.

[bartelink]

BTW Since this thread started, we've gained paket add which can quickly get a package and have it ready to go - I think it addresses a lot of the desires in here- you played with it (coz I havent really :P)

---

OK so.... what do we do with this issue to bring it to a close?

Seems to me that if someone who has done fsx scripting formulated an appropriate proposal, i.e. what subsets of stuff would you want - individual DLLs ? Source files? Both? Separable?

then Paket could generate a set of e.g. references-dlls.fsx. references-sources.fsx on every paket add paket install (and via #166 also wired into VS package restore appropriately)

then scripts can #load them (so you're confirming #load can execute #rs in the content?)

but for multi-framework packages references-net40-dlls.fsx, references-net40-sources.fsx which opens up questions about how you write the script in a general fashion - can you pass in an arg and then #load "../../packages/FSharp.Data/references-($TargetPlatform).fsx ?

---

@forki @agross @ilkerde @isaacabraham @tpetricek @ptrelford @ovatsus I'm out of names to pull in off the top of my head but I think there's enough food for thought in this thread for someone to pull out something useful for VFPT, FSI, Paket or something else to start doing which others can then use and/or build on

[rneatherway]

then scripts can #load them (so you're confirming #load can execute #rs in the content?)

Yes, I do this regularly.

but for multi-framework packages references-net40-dlls.fsx, references-net40-sources.fsx which opens up questions about how you write the script in a general fashion - can you pass in an arg and then #load "../../packages/FSharp.Data/references-($TargetPlatform).fsx ?

Not currently.

I would imagine that Paket would generate the references according to the current configuration, which could optionally be overridden. This could be done manually with say paket emitfsx or automatically during other commands.

[rneatherway]

I can put together a PR for this if it sounds acceptable to someone with commit access.

[forki]

what would you do exactly?

[rneatherway]

I suggest the following, which does not require modification of FSI:

Add a new command emitfsx to Paket. When this command is invoked, generate a file named paket.fsx that contains a list of #r directives for all the assemblies used by the current solution. Working with scripts would then just require a #load "paket.fsx" at the top.

If this works well, the generation could be made automatic -- to be run when add, update etc are run and keep paket.fsx consistent with the current state.

[forki]

it sounds good.

2014-09-29 16:56 GMT+02:00 Robin Neatherway [email protected]:

I suggest the following, which does not require modification of FSI:

Add a new command emitfsx to Paket. When this command is invoked,
generate a file named paket.fsx that contains a list of #r directives for
all the assemblies used by the current solution. Working with scripts would
then just require a #load "paket.fsx" at the top.

If this works well, the generation could be made automatic -- to be run
when add, update etc are run and keep paket.fsx consistent with the
current state.

—
Reply to this email directly or view it on GitHub
#124 (comment).

[bartelink]

@rneatherway Sounds cool.

One thing though is that I don't get is

according to the current configuration

Are you suggesting to process the xml and determine the prop value and use that as input to the process?

If so, that
a) prob wont have existing code (Am I right @forki ?)
b) will be different to the way things have worked to date (a paket install should never normally cause any diff beyond a restoration of files (beyond the diff that the target platform change itself would cause))
c) would v likely be the correct experience and hence something I'd fully support :)

How would a non-default gen become a persistent synced thing I wonder? Perhaps the (#154 There you go again @forki) paket.dependencies file could include a emitfsx [<TargetPlatformName>] element per paket definition ?

Looking forward to this. I dont use the REPL half enough.

And then I'll be atting-in @ptrelford after the impl is done to see if he'd use it to walk the code to generate test function reload lists etc. as part of a smooth testing experience etc. (IOW are there other such 'standard' fsx snippets anyone can think of?)

[rneatherway]

I have a strategy for this and hope to submit the PR soon. Just ironing out a few bugs in the project-walking code.

@bartelink I am going to process the project files using the MSBuild API. The current configuration is provided by the solution file, which I will look for in the current directory.

[forki]

Sounds exciting
On Oct 15, 2014 6:52 PM, "Robin Neatherway" [email protected]
wrote:

I have a strategy for this and hope to submit the PR soon. Just ironing
out a few bugs in the project-walking code.

@bartelink https://github.com/bartelink I am going to process the
project files using the MSBuild API. The current configuration is provided
by the solution file, which I will look for in the current directory.

—
Reply to this email directly or view it on GitHub
#124 (comment).

[bartelink]

Interesting; sounds great. Really looking forward to using all of this!

[forki]

I am going to process the project files using the MSBuild API

What does this mean?

[rneatherway]

Please see my just added PR #303

This isn't ready to be merged, but is working and shows you what I mean. I couldn't decipher the command-line parser (NDesk.Options is pretty nice, flexible and simple) so at the moment it uses the nuget option to specify the project file of interest.

Call with:

paket.exe emitfsx nuget <path/to/my.fsproj>

and it prints the full list of assemblies with #r needed for a fsx script to explore that project.