Any plans to include an upload component with example?

[Billybobbonnet]

I am thinking of something basic to add a picture to a user profile, similar to what meteoris provide (using collectionFS with the gridFS adapter). A progress bar would be great.

I could include it in the admin panel example (or a dedicated one) if you want.

[perak]

Great, let's do it.

1. If you want to create example, please create dedicated "example-upload" or something similar, and add it there. Please add minimum packages and code in order file upload to work. I'l try to use your code and add functionality into "kitchen". And your example into official examples. I'l add you into contributors list.
2. If you are ambitious to help more:

What I actually planned is to add new "input" control type "upload" into forms which automatically adds required packages into app (without explicitly specifying them). If you want to play with that, you can find form template in ~/.meteor-kitchen/templates/bootstrap/form.html and add something like:

<div id="form-input-upload" class="form-group FIELD_GROUP_CLASS FIELD_ID">
    <label for="FIELD_NAME">FIELD_TITLE</label>
    <div class="input-div">
... upload component here ...
    </div>
</div>

please leave uppercase strings as is - they are automatically replaced by generator

Code that reads from data on submit is located in ~/.meteor-kitchen/templates/bootstrap/form_utils.js

Packages that will be automatically included by generator are listed in:

~/.meteor-kitchen/templates/bootstrap/form.json

Usage of this would be like other input types:

{
  "name": "image_field",
  "type": "string" // ???
  "input": "upload",
 ...
}

And I will add required functionality into generator core if needed by your instructions.

What do you think?

[Billybobbonnet]

1. Once the upload input file control will be created, I'll do that.
2. I'm new to js, as well as Meteor & Github and I'm struggling to get a progress bar on my upload using cfs:ui. For some reason, I cant get the id of my uploading FS.file so the progress bar wont show.

As soon as I've solved that, I'll try to make that input control following your instructions, and send you a PR (my first one, actually).

[xumx]

I have used this package to great success. Maybe you can consider using that

https://atmospherejs.com/edgee/slingshot

[Billybobbonnet]

I would like to use slingshot with a local GridFS at first in order to be able to switch to S3 later with the direct upload benefits. From my (little) understanding, I just need to wrap the server side gridFS code into the Slingshot directive. Is that correct? I'm gonna try anyway :)

[Billybobbonnet]

I gave up with slingshot since it doesn't look like it's easily adaptable to GridFS.

I discovered a package that makes everything I expect from a file upload feature. I has no built-in file ownership constrains so it could be integrated with meteor-kitchen user-account system.

A few details from the package readme:

The file-collection package also provides a simple way to enable secure HTTP (GET, POST, PUT, DELETE) interfaces to your files, and additionally has built-in support for robust and resumable file uploads using the excellent Resumable.js library.

The file-collection is built on to of Meteor.Collection but it does not support the same options, so when I load it, I get an error 8 from meteor:

Error: The global definition of Mongo.Collection has changed since the file-collection package was loaded. Please ensure that any packages that redefine Mongo.Collection are loaded before file-collection.

Where should I declare the file collection if I want to load it correctly?

[Billybobbonnet]

@perak @xumx I'm making progress. I adapted this meteor-file-collection package (mentioned above) to the meteor-kitchen setup. Before I go further, I wanted to ask your opinion on the best way to do things. Here is what I have in mind:

• I create an new "input" control type "upload" which contains two parts:
  • a non displayed read-only text input used to add a fileUrl field in the collection attached to the form (e.g. invoices)
  • an "uploader" template which deals with the fileCollection. It takes care of the upload and the authorizations. Once the upload is completed, it fills the fileUrl string field with the uploaded file URL.
• Additionally, I add a special collection named "Files" where we store the file passed by the "uploader" template and define the .allow authorizations.

The pros:

• I provides fancy resumable uploads with multiple files select, image/video preview, drag&drop zone, and progress bar.
• It creates an additional "file" layer to meteor-kitchen which can be used to handle the files assets seamlessly
• It stores only the URL in the regular Meteor.Collection

Several cons:

• A cookie up-to-date with the Meteor Auth token of the logged-in user is needed so that the read/write allow rules on the server can verify the userId of each HTTP request. (any other way to achieve that?)
• Resumable.js generates 404 errors because it tests the existence of chunks on the server before sending them. See Error 404 messages on client when uploading a file vsivsi/meteor-file-collection#5
• It requires several packages: jQuery-cookies, meteor-file-collection, numeral

[perak]

@Billybobbonnet nice to see you are making progress! 👍

Can you please deploy (to meteor) simple example, I wish to see how that works.

But, need to say, I am not happy about 404 errors, using cookies... but in worst case, we can use your solution as transitional solution - currently we don't have any upload possibility, so everything goes.

P.S. I'l upload latest version of meteor-kitchen later today (v.0.9.33) your loadinTemplate is included ;)

[perak]

Important! @Billybobbonnet Be careful - if you install latest version, your ~/meteor-kitchen directory will be overwritten - wach out and don't loose your work on upload!!!

[perak]

@Billybobbonnet please let me know if you need, in order upload field type to work, to add new properties to "field" object (and if some strings from template should be replaced by kitchen while generating code, similar to already existing FIELD_NAME, FIELD_TITLE, etc.).

[Billybobbonnet]

@perak I made you an example here: https://github.com/Billybobbonnet/meteor-kitchen-upload-example

See the differences with the minimal example with account system between the two previous commits. I only added a file collection before building the example.

In this version, you can upload an image with preview, drag & drop, to the server on the user_details page. Several things to be noted:

• I also made a live example. I send you the url by email.
• the 404 errors could be changed in 204 (no content) but I don't know how. See How to silence 404 logging? 23/resumable.js#127
• The use of the cookie bugs me and I think we could use the auth token instead. But it is out of my league at the moment
• As I struggled on the form_utils.js modification, I realized it would be better to send you first what I have and let you see where we go from there. An {{> uploader}} template is added to the user_settings form but it doesn't update the user.image (string URL) property yet on form validation.
• Basically, it creates the "Files" API, implements some basic authentication rules (see server/collection/files.js) and handle the uploads client-side (chunking, progress, etc.).
• In the live example there is no file filter or size limit. You're welcome to put a big file in order to see the progress bar behavior but pls remove it afterwards.

[Billybobbonnet]

@Billybobbonnet please let me know if you need, in order upload field type to work, to add new properties to "field" object (and if some strings from template should be replaced by kitchen while generating code, similar to already existing FIELD_NAME, FIELD_TITLE, etc.).

I did not pay attention to this message. I guess it is up to what you decide. The minimal set would be just the FIELD_FILE string containing the file URL. We could add some options:

• file extension filters (client and server-side checks)
• file size filters (only client side I guess)
• file existence server-side check for the big files (would require to hash the file client-side and compare MD5). I've read somewhere it could be done in approx 10sec/500mb.
• single/multiple file upload option

[perak]

Great! I am busy with my job at the moment but will take a look later (can't wait!) :)

P.S. you got tails from git merge conflicts <<<<<<< HEAD in multiple source files!

[Billybobbonnet]

P.S. you got tails from git merge conflicts <<<<<<< HEAD in multiple source files!

lol yes, I still need to learn to properly use git.

[perak]

:) no worries, take your time - I'l take a look later today (in few hours).

[perak]

Any chance to see live example?

P.S. did you resolved merge conflicts? If so, please push your code.

TNX

[Billybobbonnet]

the merge conflicts are solved in a fresh repo: https://github.com/Billybobbonnet/meteor-kitchen-upload-example

live example: kitchenupload.meteor.com.

[perak]

Looks good, thanks!

I'l clean-up the code little bit and add it into official distribution, but not before next weekend. Keep me updated if you make more progress on this.

👍

[Billybobbonnet]

I have added an upload percentage display and a set of filters for files in client/lib/files_utils.js.

What I plan/want to add is (including items above) :

• a preview of the file using FileReader (see http://jsfiddle.net/LvsYc/638/) but I failed my firsts attempts.
• file extension filters (client is done, server-side checks remains)
• file size filters (only client side I guess)
• file existence server-side check for the big files (would require to hash the file client-side and compare MD5). I've read somewhere it could be done in approx 10sec/500mb.
• single/multiple file upload option with global upload progress
• time remaining

Concerning the 404 errors, making it 204 could not be a solution (see 23/resumable.js#188) or it would require to fork resumable.js.

Concerning the cookie, it seems possible to use tokens instead (see https://github.com/vsivsi/meteor-file-collection#http-authentication)

[Billybobbonnet]

@perak Have you already changed the code? If not, I plan to add a mini api to the uploader (file size, file type filter and single/multiple upload.), an HTML5 thumbnail generator (video & image) and a file preview system based on file types.

What do you think I should do?

1. I let you handle in kitchen the generation of the uploader code depending on the media type to be uploaded
   or 2) I make an all-purpose version where we only change the input parameters (e.g. {{ > uploader multiple videos}}?

The second solution would allow re-usability but it would require to make sub-templates in order to avoid loading all the file types filtering and preview code every time. I'm getting better but it wont be an easy ride :) By the way, when blaze has a view in a {{# if }} statement, is the subtemplate loaded in the client browser by default, even if not rendered? I would make my proposition less interesting.

I'm also not clear on how to integrate it in the form_utils.js file. I would request to make one check per file filter type : I get the file URL string in img src preview, video src preview if available/possible or a placeholder, other files in a placeholder. I am also considering to preview pdf (see here, even if it might be a little overkill). Is it worth making it now or will you change the form validation procedure soon?

What do you think? Are you ok with my approach? > generic file api with types field, kitchen gets back only the file URL and provide attached (inherited?) permissions.

Another update: I asked file-collection package creator @vsivsi how to get rid of the cookie but it could be not as easy as I said. (see vsivsi/meteor-file-collection#46)

[perak]

Hey,

I didn't change code yet... but file upload is must have feature and that's next step. Currently bussy with adding cool new feature to the GUI - "build and run app in the cloud", pre-pre-alpha preview here:

;)

[Billybobbonnet]

Yes! That's looking great :)

If you have a moment this upcoming week, tell me in what direction to go (see my questions above). I'll work on other topics and switch on uploader as soon as I have your instructions.

[perak]

OK. Thank you for your work. Also, it looks like next week I'l have more free time than usual and will focus to help you and to merge your contributions. 👍

[vsivsi]

Hi, I'm the author of meteor-file-collection. It is not necessary to use an HTTP cookie to pass the authentication token during file upload. The HTTP header method works just fine. See my comment here: vsivsi/meteor-file-collection#46 (comment)

However, do note that without using a cookie, the only way to restrict HTTP GET access to stored files by Meteor userId is by handling all of your own file transfers using XMLHttpRequest or equivalent.

Basically, if you want to just use <a href="..."> or <img src="...">, etc to refer to file-collection stored assets that are securely restricted to one or more Meteor userIds, then you need to use the authentication cookie, because there's no other way to set custom HTTP header information using HTML. However, if all of your HTTP GETs are being done using Javascript, then use of cookies is not necessary.

[perak]

Thank you @vsivsi for detailed info!

[Billybobbonnet]

@perak, @vsivsi (in case it would interest you as an example), I added several features to https://github.com/Billybobbonnet/meteor-kitchen-upload-example. This is a fresh repo so you can see the difference with the minimal user account example.

First, a few things that need to be said:

• I used the cookie version of the upload (see an example of token based authentication vsivsi/meteor-file-collection#46 (comment))
• the source might have errors "as-is" since the .meteor folder has not been updated. I'll try to make it right tomorrow.
• the live demo is updated at http://kitchenupload.meteor.com. The uploader parameters are set to fileType: video, sizeLimit: 200MB, mode: "single". Please remove the files you uploaded.

Second, the new features:

• I added parameters to the template so it can be loaded as follows {{ > uploader filetypefilter="video" mode="single" sizelimit="200" }}
  • mode /values: single or multiple/ default: single
  • filetypefilter/values: video, audio, document, subtitle, image/default: none
  • sizelimit/values; any integer in MB/default:unlimited
• I added an error management which selectively refuse files depending on the fileType/fileSize/uploadMode settings. It displays an alert within the file uploading queue while allowing the remaining selected files to be added (in case of mode="multiple")
• I added a video preview based on Mime type (mp4, ogg, webm).

Third, what is still missing:

• The template has still some graphical glitches (e.g. non loaded video preview creates overlapped DOM elements), even if the ui as been improved
• the 404 errors are still here. It should be possible to change them into 204 but it would require to (slightly) change resumable.js.
• I want to add image and video previews of the files before the upload using fileinput.files>dataUrl. They will switch on the server url when the file is uploaded. Ideally, but it is a low priority addition, I'd like to add pdf first page preview
• I want to generate image thumbnails and video posters&thumbnails using canvas and add them to the file item.
• No integration with form_utils.js yet.
• The form is submitted when a file has been added. I guess it has to do with the above item.

Lastly, it was a long but fun day. I would be glad to have your feedback on my way of doing things since I'm a just starting to code in javascript. I'm not a trained developer, I just learned it out of curiosity and mostly with local desktops applications.

[perak]

@Billybobbonnet thank you for your efforts. Sorry for my unresponsiveness - I am moving to a new apartment. Please little bit more patience, I am max interested in implementing upload feature into meteor-kitchen, and your work is great contribution.

[Billybobbonnet]

Thanks @perak. Take your time and enjoy your new apartment 😄 . I'll wait for your feedback.

[Billybobbonnet]

damn, looks like i lost the right version of .meteor folder. Current repo is working but I encounter some duplicate id errors from mongo leading to extra 403 errors. I'll make a fresh version of it manually later.

[vsivsi]

@Billybobbonnet FWIW, I've successfully made the upload testChunk 404s go away in version 1.1.0 of file-collection, due for release very soon. 204s work just as well, and don't generate the distracting console noise. If you want a preview, the changes are on the dev_1.1 branch. https://github.com/vsivsi/meteor-file-collection/tree/dev_1.1

[perak]

@vsivsi that's wonderful!

[Billybobbonnet]

@vsivsi this is great! And it looks like you had not to fork resumable, you kept it as a submodule and changed file-collection. It's even better 😄

I'll try your branch asap to test it.

[Billybobbonnet]

@perak I have made (yet) another fresh repo with the right code. (see https://github.com/Billybobbonnet/meteor-kitchen-upload-example)

@vsivsi I tried and managed to edit your package code directly inside my .meteor folder but meteor always restore the former version when the project is built. I guess I'll have to wait for your release.
I am surprised to see how straightforward it is. This might explain why resumable's people don't want to make it 204 by default (see 23/resumable.js#188).

[vsivsi]

If you manually edit the versions file in your app's meteor directory, you can specify vsivsi:[email protected] which will force it to use the cloned repo in your packages directory.

[vsivsi]

Just to close the loop on this, file-collection released version 1.1.0 with the above referenced changes last week. Enjoy!

[perak]

@vsivsi just in time! Thanks.

[perak]

@Billybobbonnet at last! I just uploaded version 0.9.39 with upload support. At the end, I decided to use CollectionFS. Upload feature is not perfectly implemented yet (especially front-end) but it works :)

You can see live example here: http://generator-upload.meteor.com/
And source code here: https://github.com/perak/kitchen-examples/tree/master/example-upload

:)

[Billybobbonnet]

nice job @perak ! This is looking clean.

Have you tried cfs-ui for the progress bar? Will you use collectionFS for s3 as well? (it would require to pipe all your uploads through your servers)

One quick feedback too: it does not work on firefox (last version) yet. I let a failed upload with "what is coding" as title.

Anyway, this is nice to see this happening :-) congrats mate!

ps: feel free to close the isue, or tell me if you want to.

[bob360]

Very Very interesting,
May I suggest example-upload to be flexible to give developer a choice to setup their own cloud. This can be set ,on a config/ json/ settings.js file or sets off commands to both client and server. I only need that example-upload (to cloudinary) to finish my project.Please keep me posted.