implement loading from resources, reloading/saving, some api changes

docs/adr/0002-recipe-rewrite.md

@@ -1,7 +1,7 @@
 # 2. Recipe rewrite
 
 Date: 2024-11-03
-Last update: 2024-11-03
+Last update: 2024-11-08
 
 **DO NOT rely on any APIs introduced until we finish the work completely!**
 
@@ -32,9 +32,9 @@ Slimefun, focusing on
   Slimefun recipes
 - Performance: Should not blow up any servers
 
-The new system should also be completely backwards compatible with the old.
+The new recipe system should also be completely backwards compatible.
 
-## API Changes
+## API Additions
 
 ### 5 main recipe classes
 
@@ -61,7 +61,7 @@ An `RecipeOutputItem`s controls how an output is generated when the recipe is
 crafted. It can be a single item (see `RecipeOutputItemStack`, `RecipeOutputSlimefunItem`),
 or a group of items each with a certain weight of being output (see `RecipeOutputGroup`).
 
-#### Examples
+#### Examples (pseudocode)
 
 Here are the inputs and outputs of the recipe for a vanilla torch
 
@@ -104,9 +104,9 @@ This is the public interface for the recipe system, there are methods here to ad
 load, save, and search recipes. It also stores a map of `MatchProcedures` and
 `RecipeType` by key for conversions from a string
 
-## JSON Serialization
+### JSON Serialization
 
-All recipes should be able to be serialized to and deserialized
+All recipes are able to be serialized to and deserialized
 from JSON. The schemas are shown below.
 
 Here, `key` is the string representation of a namespaced key
@@ -124,7 +124,8 @@ Here, `key` is the string representation of a namespaced key
 }
 ```
 
-The recipe deserializer also needs a `__filename` field, which is inserted when the file is read, so it doesn't (and shouldn't) be in the schema
+The recipe deserializer technically needs a `__filename` field, but it is
+inserted when the file is read, so it isn't (and shouldn't) be in the schema
 
 `RecipeInput`
 
@@ -174,7 +175,7 @@ The recipe deserializer also needs a `__filename` field, which is inserted when
 }
 ```
 
-*In addition to those schemata, items can be in short form:
+*In addition to those schemas, items can be in short form:
 
 - Single items: `<namespace>:<id>|<amount>`
 - Tags: `#<namespace>:<id>|<amount>`
@@ -183,22 +184,76 @@ The recipe deserializer also needs a `__filename` field, which is inserted when
 
 The 5 main recipe classes are all polymorphic, and subclasses can be used in their
 stead, and should not affect the recipe system (as long as the right methods are
-override, see javadocs)
+overriden, see javadocs)
 
 ### Custom serialization/deserialization
 
 The default deserializers recognize subclasses with custom deserializers by
-the presence of a `class` field in the json, which is the key of a
+the presence of a `class` field in the json, which should be the key of a
 custom deserializer registered with Slimefun's `RecipeService`.
 For custom serializers, override the `serialize` method on the subclass,
 and ensure they also add the `class` field
 
+## Recipe Lifecycle
+
+### Stage 1a
+
+When Slimefun is enabled, all recipes in the resources folder will be
+moved to `plugins/Slimefun/recipes/` (unless a file with its name already exists).
+
+Addons should do the same. (We recommend saving to
+`plugins/Slimefun/recipes/<your-addon-name>/` but it's not required).
+
+To prevent unnecessary file operations, Slimefun/addons first send a list of
+filenames of recipes present in the resources folder to the recipe service,
+which then filters out all the files that already exist. Then each recipe can
+be read and copied over.
+
+### Stage 1b
+
+Also on enable, recipes defined in code should be registered. These two steps
+can be done in no particular order.
+
+### Stage 2
+
+On the first server tick, all recipes in the `plugins/Slimefun/recipes` folder
+are read and added to the `RecipeService`, removing all recipes with the
+same filename. This is why recipes should ideally be *defined* in JSON,
+to prevent unnecessary work.
+
+When loading JSON recipes, we also need to be able to tell the difference between
+a server owner changing a recipe, and a developer changing a recipe. To do this,
+we use a system called Recipe Overrides; it allows for updates to recipes from
+developers while also preserving custom recipes by server owners
+
+- Slimefun/addons should tell the recipe service it will apply a recipe
+  override on enable, **before** any JSON recipes are copied from the resources
+  folder
+- The recipe service checks all recipe overrides that have already run
+  (in the file `plugins/Slimefun/recipe-overrides`) and if it never received
+  that override before, it deletes the old files and all recipes inside them.
+  Then all recipes are loaded as before.
+
+### Stage 3
+
+While the server is running, recipes can be modified in code, saved to disk, or
+re-loaded from disk. New recipes can also be added, however not to any existing
+file (unless forced, which is not recommended)
+
+### Stage 4
+
+On server shutdown (or `/sf recipe save`), **all** recipes are saved to disk.
+This means any changes made while the server is running will be overwritten.
+Server owners should run `/sf recipe reload <file-name?>` to load new recipes
+dynamically from disk.
+
 ## Phases
 
 Each phase should be a separate PR
 
 - Phase 1 - Add the new API
-- Phase 2 - Migrate Slimefun toward the new API
+- Phase 2 - Migrate Slimefun items/multiblocks/machines toward the new API
+- Phase 3 - Update the Slimefun Guide to use the new API
 
 The entire process should be seamless for the end users, and
 backwards compatible with addons that haven't yet migrated

pom.xml

@@ -317,6 +317,7 @@
                     <include>biome-maps/*.json</include>
 
                     <include>languages/**/*.yml</include>
+                    <include>recipes/**/*.json</include>
                 </includes>
             </resource>
 

src/main/java/io/github/thebusybiscuit/slimefun4/api/SlimefunAddon.java

@@ -1,6 +1,18 @@
 package io.github.thebusybiscuit.slimefun4.api;
 
+import java.io.InputStream;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.nio.file.FileSystem;
+import java.nio.file.FileSystems;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Collections;
+import java.util.Set;
 import java.util.logging.Logger;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
 
 import javax.annotation.Nonnull;
 import javax.annotation.Nullable;
@@ -11,6 +23,8 @@
 import org.bukkit.plugin.java.JavaPlugin;
 
 import io.github.thebusybiscuit.slimefun4.api.items.SlimefunItem;
+import io.github.thebusybiscuit.slimefun4.core.services.RecipeService;
+import io.github.thebusybiscuit.slimefun4.implementation.Slimefun;
 
 /**
  * This is a very basic interface that will be used to identify
@@ -97,4 +111,72 @@ default boolean hasDependency(@Nonnull String dependency) {
         return description.getDepend().contains(dependency) || description.getSoftDepend().contains(dependency);
     }
 
+    /**
+     * @return A list of all recipes in the resources folder. Addons
+     * can override this to filter out certain recipes, if desired.
+     */
+    default Set<String> getResourceRecipeFilenames() {
+        URL resourceDir = getClass().getResource("/recipes");
+        if (resourceDir == null) {
+            return Collections.emptySet();
+        }
+        URI resourceUri;
+        try {
+            resourceUri = resourceDir.toURI();
+        } catch (URISyntaxException e) {
+            return Collections.emptySet();
+        }
+        if (!resourceUri.getScheme().equals("jar")) {
+            return Collections.emptySet();
+        }
+        try (FileSystem fs = FileSystems.newFileSystem(resourceUri, Collections.emptyMap())) {
+            Path recipeDir = fs.getPath("/recipes");
+            try (Stream<Path> files = Files.walk(recipeDir)) {
+                var names = files
+                    .filter(file -> file.toString().endsWith(".json"))
+                    .map(file -> {
+                        String filename = recipeDir.relativize(file).toString();
+                        return filename.substring(0, filename.length() - 5);
+                    })
+                    .collect(Collectors.toSet());
+                return names;
+            } catch (Exception e) {
+                return Collections.emptySet();
+            }
+        } catch (Exception e) {
+            return Collections.emptySet();
+        }
+    }
+
+    /**
+     * Copies all recipes in the recipes folder of the jar to
+     * <code>plugins/Slimefun/recipes/[subdirectory]</code>
+     * This should be done on enable. If you need to add
+     * any recipe overrides, those should be done before calling
+     * this method.
+     * @param subdirectory The subdirectory to copy files to
+     */
+    default void copyResourceRecipes(String subdirectory) {
+        Set<String> existingRecipes = Slimefun.getRecipeService().getAllRecipeFilenames();
+        Set<String> resourceNames = getResourceRecipeFilenames();
+        resourceNames.removeIf(existingRecipes::contains);
+        for (String name : resourceNames) {
+            try (InputStream source = getClass().getResourceAsStream("/recipes/" + name + ".json")) {
+                Files.copy(source, Path.of(RecipeService.SAVED_RECIPE_DIR, subdirectory, name + ".json"));
+            } catch (Exception e) {
+                getLogger().warning("Couldn't copy recipes in resource file '" + name + "': " + e.getLocalizedMessage());
+            }
+        }
+    }
+
+    /**
+     * Copies all recipes in the recipes folder of the jar to
+     * plugins/Slimefun/recipes. This should be done on enable.
+     * If you need to add any recipe overrides, those should
+     * be done before calling this method.
+     */
+    default void copyResourceRecipes() {
+        copyResourceRecipes("");
+    }
+
 }

src/main/java/io/github/thebusybiscuit/slimefun4/api/recipes/Recipe.java

@@ -66,7 +66,7 @@ public static Recipe fromItemStacks(String id, ItemStack[] inputs, ItemStack[] o
     public static Recipe fromItemStacks(ItemStack[] inputs, ItemStack[] outputs, RecipeType type, MatchProcedure match) {
         return new Recipe(
             Optional.empty(),
-            "other_recipes",
+            "other_recipes.json",
             RecipeInput.fromItemStacks(inputs, match),
             RecipeOutput.fromItemStacks(outputs),
             List.of(type),
@@ -193,11 +193,14 @@ public String toString() {
     public JsonElement serialize(JsonSerializationContext context) {
         JsonObject recipe = new JsonObject();
 
+        if (id.isPresent()) {
+            recipe.addProperty("id", id.get());
+        }
         if (!input.isEmpty()) {
-            recipe.add("input", context.serialize(input, AbstractRecipeInput.class));
+            recipe.add("input", input.serialize(context));
         }
         if (!output.isEmpty()) {
-            recipe.add("output", context.serialize(output, AbstractRecipeOutput.class));
+            recipe.add("output", output.serialize(context));
         }
         if (types.size() == 1) {
             recipe.addProperty("type", types.stream().findFirst().get().toString());