New AssetLib Workflow

[nonunknown]

Describe the project you are working on:

Not Related to a project, Applicable to all community

Describe the problem or limitation you are having in your project:

The current asset store, is working fine, I'm not sure but I think Godot Maintainers pay a server to maintain it, and it relies on only
few, if not one, moderator.

Describe the feature / enhancement and how it helps to overcome the problem or limitation:

My proposal is an enhancement to the assetlib, that fully relies on github (the majority of godot users use it).
This enhancement will give the following advantages:

• Fast Server Response
• No needed to pay hosting
• Easy Moderation due to relying on github PRs (Pull Requests)
• Better workflow

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

Resuming, I've made a plugin (Will post here later as this proposal evolves) that works as MVP (minimum viable product) so the reader can follow reading, and doing at the same time

PS: Github personal token, from tests I've done today wasn't necessary, so it will be optional in a settings tab.

Step - 1

• After user Install and activate the plugin, will be a list with all registered assets. Since its not relied on a server, this information has to be in somewhere, in this case the plugin uses this url:

https://github.com/nonunknown/godot-assetlib-x/blob/data/repos.json

• PS - in case this is approved, the godot engine github account would need a new repo called: assetlib, there all information will be stored in a json file, like the example above.

The file above will list all assets available in the assetstore.

Submit asset workflow

In the actual workflow the user must go to asset store page, register (again?), fulfill lots of informations and wait for approval.

The new workflow

User goes to go to his github repo lets say: UserA/PluginA or UserB/PluginB
create a new branch (required pattern): assetstore

This branch must not contain master branch's files, but these:

• imgs/ - Images to show in asset lib's details
• asset.json - Used by assetlib plugin
• description.MD - The description is a MD(MarkDown) file, for better organization
• icon.png - assetlib icon to show
• dependencies.json - other plugin dependencies (optional file)

• Asset.json

{
	"name":"Required Asset Name here",
	"author":"Required Name here",
	"category":"Required Category here",
	"license":"Required License here", 
	"godot_version": 0, 
	"version": 0,
	"videos": ["Optional urls here"],
	"commit": "latest"
}

PS: For auto-detect values read The Future section

After making this new branch, the user goes to GodotEngine's AssetLib repo and create a pull request, this pull request must modify the repos.json file only including the existing repo url as the following example:

before PR

[
	"nonunknown/godot-sheet-manager",
	"usera/pluginA"
]

after PR

[
	"nonunknown/godot-sheet-manager",
	"usera/pluginA",
	"userb/pluginB"
]

So the branch assetlib containing these files will be automatically accessed by the plugin's when it requests repos.json

Dependencies

One new feature, requested by many people is dependencies, since this relies on github a way to solve this problem, is adding
a new file to assetlib branch of the user's plugin.

• Dependencies.json

[
	{
		"asset": "userC/pluginC",
		"commit": "latest"
	},
	{
		"asset": "userD/pluginD",
		"commit": "HUASD3357784DAS"
	}
	
]

Before downloading any dependency, the assetlib must check if it already exists and the commit hash.

Example Repos:

https://github.com/nonunknown/Godot-3D-text-plugin/tree/assetstore
https://github.com/nonunknown/godot-sheet-manager/tree/assetstore

The future

• In order to automate the process, a github action can be used to check if the PR inserted url has a assetlib branch with the pattern mentioned above.
• since the plugin uses github API , a button: "Submit asset" can be created, this will let users submit asset automatically by selecting
  the folder and telling the program the repo URL and then the assetlib branch will created with required files, and everything can be
  done inside the engine

This project solves the following issues

#142 - Add dependency handling and enable users to organize and publish addons and addon-collections as packages
#831 - Add support for global/universal addons / In a proper release
#554 - Allow extracting asset library assets into subfolders
#139 - Add full Markdown support for use in browsing, creating, and editing projects and addons / Partially if this is only a plugin, fully if its turn core

If this enhancement will not be used often, can it be worked around with a few lines of script?:

This Feature will be use by every single godot user

Is there a reason why this should be core and not an add-on in the asset library?:

This question is answered by the pros mentioned above

[JotaFaD]

I may be wrong, but I think the asset library is hosted alongside the godotengine website, which is done for free by TuxFamily.

[DrMoriarty]

@nonunknown Hi! This proposal is very interesting, but what do you think of asset dependencies?
I developed a lot of godot modules, especially for iOS and Android which are wrap common used SDKs. And they sometimes depends of each other so it will be very difficult to operate them in current Asset Library.
The second issue that for special version of godot someone should use special version of binary module. The current Asset Library (and the your proposal) have no option to operate with engine and asset versions.

So, I'm already thinking to make my own plugin management system like the SDKBOX for cocos2d-x.

[YuriSizov]

Tying into a third-party ecosystem is a bad idea, especially for an open source project. Current asset library can be modified to use any provider with no modification of the client application.

You can make a fully functioning plugin that would replace the default Asset Library tool with a GitHub only tool (or it can be used alongside). Openness of Godot allows you to do it and freely distribute it to anybody needing it for their workflow. But I don't think it's a good addition to the core.

[nonunknown]

@pycbouh Well, I cant see the logic in why it can not rely on github, since the engine itself is hosted here!

@DrMoriarty Well this is one of the best things that can happen, actually I've already have an idea on this, but forgot to mention, I'm updating the proposal here....

[YuriSizov]

Well, I cant see the logic in why it can not rely on github, since the engine itself is hosted here!

The engine can be "hosted" anywhere at no loss. The beauty of a distributed VCS. It can in fact have a mirror on any number of Git platforms and even have a self-hosted one. And, frankly, where the code is made publicly available doesn't matter as much as the restriction your proposal would put on the actual engine and its usage.

[Calinou]

Keep in mind many people would prefer hosting their assets elsewhere than on GitHub. On the current asset library, we have a few dozen assets which are hosted on GitLab or Bitbucket. We don't want to stiffle competition 🙂

[aaronfranke]

and it relies on only few, if not one, moderator.

That's pretty much the situation, yes, but I have no problem handling the entirety of moderation on the asset library. It's not much.

[willnationsdev]

I believe it is generally better to preserve the service-agnostic approach to integrating. If you want a more direct relationship with GitHub rather than having to go through the Asset Library, I would suggest having #12 implemented instead and then adding GitHub as a registered service via a plugin.

[bojidar-bg]

Consider also the discussion in godotengine/godot-website#82: it is going to be much simpler to move the asset library to Github pages or netlify or some other static website host, than to make use of the Github API in the editor. That way, it would also allow custom asset libraries to be added with ease, as they would be just a HTTP URL pointing to a standardized folder structure of JSON and image files.

[DrMoriarty]

Hi All! Recently I implemented my own plugin management system for my own modules. It mostly designed for process binary precompiled modules for third-party SDKs for iOS and Android.
It has dependencies, autoloading gd wrappers, export hooks, uniform module settings for app keys and so on. It can be used as GUI addon and as CLI utility (for example when you want build your project with CI and don't want to store all binary frameworks in github repo. They can be automatically downloaded before your project was build.)

Full info and package list: https://drmoriarty.github.io/nativelib/
GUI Addon: https://godotengine.org/asset-library/asset/824
CLI utility: https://github.com/DrMoriarty/nativelib-cli

What do you think of it?

[Calinou]

Closing due to lack of support (see the above comments).

[aaronfranke]

@pycbouh: Current asset library can be modified to use any provider with no modification of the client application.

Note for the record: This wasn't possible when you posted this comment, but this is possible in Godot 3.3 and later:

[YuriSizov]

@aaronfranke I think I was talking about the backend, as providers (here referring to GitHub, GitLab, etc) are handled there and the editor only receives a URL to the archive. I didn't mean that you can access a different backend from the editor.

At the time of writing that was indeed only hardcoded to accept either the official backend or localhost.

[DrMoriarty]

@aaronfranke Could you point me to the Asset Library API documentation?
This https://docs.godotengine.org/en/stable/tutorials/assetlib/index.html has no mentions about it.

[aaronfranke]

@DrMoriarty I don't think there is documentation for the API, but the Godot Asset Library is planned to be rewritten soon, so it likely won't get documented until after it's rewritten.

[Calinou]

@DrMoriarty I don't think there is documentation for the API, but the Godot Asset Library is planned to be rewritten soon, so it likely won't get documented until after it's rewritten.

There is: https://github.com/godotengine/godot-asset-library/blob/master/API.md

The new asset library rewrite also has an OpenAPI specification file: https://github.com/Calinou/godot-asset-library-laravel/blob/master/public/developer/v1/openapi.yaml