4.0 TileMap improvements needed

[WolfgangSenff]

Hey all,

Reduzio asked me to make a proposal/discussion for how I would update the TileMap editor in 4.0 to be simpler to use. I have some suggestions here for beginners to make it easier for them to use the new TileMap/Set editor, and then actually some suggestions for advanced users so that it's easier to use in the long run. Anything listed with (Me) or something similar is likely something I'll attempt if there's any support for it. Sorry, this is long!

TileMap 4.0 problems:

1. Lots of information is shown even when a TileMap is unusable due to no active TileSet

   • This is probably not a great one to start with, but I'm going to anyway as it's kind of the main entry point to TileSets. I feel that if a TileSet isn't actively set, it shouldn't show all the accoutrements related to the TileMap or Set at all, such as the TileMap button in the bottom dock.
   • Why are you able to edit layers without an active TileSet? I get that they're (loosely) orthogonal concepts and setting up a bunch of layers ahead of time might make sense to some, but it feels like too many entry points in the kitchen, if you catch my meaning.
   • I'm not exactly sure if there's anything that can be done about this, but having Y Sort Enabled as a property in both Node2D and in the Layers values is pretty confusing - which one are we supposed to enable? Or do we have to enable both? Same with Z Index I suppose, but with that it sort of makes sense, since one is for the TileMap entirely, the other is for each individual layer. But it's still pretty confusing for new users trying to figure out which one to use when, especially if they have little experience with TileMaps in the first place.
     

2. When adding a TileSet source, it says only Atlas and Scene Collection. I recognize Atlas is probably the more common name, but it makes it unclear (to me at least) that you can and should be adding images here. At the same time, there's no indication as to what your scene collection might contain, or what that even is. Is it a resource that I have to make in the editor, or do I have to create one in code and add scenes to it in code and set it manually in code? I'm confused because I just attempted to create one in the editor, and there's nothing exported on it that would help me to understand how it's supposed to be used.

3. Physics layers:

   • Looking at the physics layers led me to realize at least one part of why I find the new stuff in 4.0 to be very confusing: handling multiple different tilesets happens now in a single tilemap. If folks attempt to use it this way, rather than just having multiple, layered tilemaps, it's going to lead to a ton of problems. This is the definition of too many responsibilities. One potential solution would be to have a parent node that is something like TileMapContainer (bad name, but you get the drift) which has the meta properties for how the child tilemaps function w/r/t each other. So for example you define the YSort properties there. This would allow each tilemap to handle just its own properties and not have to worry about what happens when it's combined together with other tilemaps, making it easier to switch after the fact, if needed; at present, while you can drag layers around and stuff and that's pretty cool, it's fairly unintuitive imo.
   • I do not understand the purpose of the linear_velocity and angular_velocity values inside the physics layers in the editor itself. What if you have a platform that moves back and forth or up and down within the same space? Do I have to update those values, maybe with an animation player? If I have to update it with an animation player, I can't: it goes away since it's part of the bottom dock as well.
   • Painting physics shapes into any sufficiently complex tileset is at best a laborious job. I love the concept of painting them, but it's just not practical: you need to be able to draw a single physics shape across the entire tileset regardless of where the individual tiles are, and then have the individual tiles take on whatever physics shape is contained in their square when the drawing of the physics shape is done. For instance, with tilesets from Tilesetter, every single tile is likely to have a different collision shape, which means you can't paint more than one tile at a time anyway because you have to keep changing the shape of the brush. While there are probably some tilesets where you just want every shape to be the same, it has not been the case in my experience that this is the average usecase. If this is already possible to do, I have not yet figured out at all how to do it and it's extremely unintuitive if it is possible.
   • To facilitate making all the collision shapes, I wonder if it would be possible to have values that define the borders around shapes generally; then, you take the bitmask and calculate where the shapes need to go in order to meet the bitmask's requirements. This kind of shape autogenerator would be tremendously useful, as, often, the navigation shape is the sort of the set difference of the whole tile shape minus the collision shape(s), and so you'd be generating the navigation shape right along with the collision shape. These could then be manually edited in the editor itself and it wouldn't feel quite as bad, since all the work was setup for you in the first place. (Me - probably going to attempt a plugin with this.)
   • Minor quibble: There are so many properties on tiles now that it's impossible to tell at a glance what the values are for them. It would be nice to just have the collision shapes and navigation shapes show up as differently-colored polygons when a given tile is selected.
   • Minor quibble: if you create all the tiles in the tileset and then change the Texture Region Size, it has weird behavior and treats it as one giant tile. But very little is usable when that's the case. I thought maybe that was how to paint across multiple tiles at once, but it really didn't work.
   • I'm not sure what Use Texture Padding does, but at least in previous revisions, I know I had to have it turned off or things would not show up correctly.

4. Terrain sets:

   • Given the physics layer argument, there's a few things here that are separate, but the same note applies as well.
   • Terrains in the editor need to have the ability to set a terrain to No terrain simply by tapping a bit that already has a terrain equal to the one you're already attempting to place. The reason for this is simple: I should not have to travel all the way back to the Painting panel to set the terrain back to no terrain just to click on a single misclicked tile bit. I suggest either right-click or a second left-click when the terrain is the same as the one that's already placed in that bit. (Probably me.)
   • If you have multiple tilesets for a given tilemap, how often do you really want multiple, different autotile patterns embedded in the same tileset? I suppose it's possible I'm way off on this, but I just don't see it as necessary to have more than one. That makes being able to add multiple terrains just very confusing to me, especially because you can for some reason have multiple terrains in the same tile, but not in the same bit. I have no clue what that would mean or do.
     
   • The Terrain modes are extremely confusing. I guess eventually I'll get used to them, but they aren't intuitively-named at all. Indeed, I'm actually not entirely sure I know what they do now, except that "Match Corners and Sides" works for my ARPG.
   • When selecting colors for the Terrains, they show as full 255 alpha, but when placing them in the editor, they lose alpha somewhere along the way. I'm sure this is to make it possible to see the underlying tile, but it can lead to visual confusion and makes it especially hard for anyone with a vision impairment. I don't have a great solution for this, and the only real solution I can think of is to maybe only show the transparency when the mouse is hovered over the bit.
   • Having to click a square when painting to activate it, and then allowing painting only after it has been selected, is somewhat painful and represents an extra click. If you could remove bits with right click or a second click, this wouldn't ever cause any issues because you could just easily remove any accidental clicks as well. The old one worked this way and was much better in that sense: you never had to select a tile when placing bits.

5. Pretty confused about the animation properties. Do you select a set of animation frames as one "tile" in the actual atlas and then define a number of columns? What if it's broken up over multiple rows as well? Frames doesn't allow you to add any frames, even though the plus button is active.

6. When you put Signal as the type for a Custom Data Layer, what happens? Does it define a new signal for that tile? Lots of confusing stuff in this area. It’s also not well-documented (yet) how to get the relevant data out of that layer, but I’m sure that’ll come in time.

[Mickeon]

When selecting colors for the Terrains, they show as full 255 alpha, but when placing them in the editor, they lose alpha somewhere along the way. I'm sure this is to make it possible to see the underlying tile, but it can lead to visual confusion and makes it especially hard for anyone with a vision impairment.

It may be doable and decent looking to draw some sort of small figures around the terrain instead of covering it entirely.
Something like this?

[WolfgangSenff]

That is a very interesting idea. At a guess, I would think it's a healthy amount of work, but I do really like it conceptually.

[PoisonousGame]

This will cover up the underlying tiles

[PoisonousGame]

It is not a good experience when drawing terrain with a bit mask that cannot be simply canceled.

[groud]

This is probably not a great one to start with, but I'm going to anyway as it's kind of the main entry point to TileSets. I feel that if a TileSet isn't actively set, it shouldn't show all the accoutrements related to the TileMap or Set at all, such as the TileMap button in the bottom dock.

It kind of make sense, but I think this is quite inconsistent with how other node editors work. I don't think there is another situation where an editor will disappear because you did not set a given property of the node.

Why are you able to edit layers without an active TileSet? I get that they're (loosely) orthogonal concepts and setting up a bunch of layers ahead of time might make sense to some, but it feels like too many entry points in the kitchen, if you catch my meaning.

Hmm why not. But I am not really found of hiding things for the sake of it, but I guess they could be disabled if we decide to do that.
On the other hand, I dislike disabling things that could be edited if needed. It might be a bit far-fetched, but some people might just want to procedurally build a TileMap and assign the TileSet later on, in which case editing the layers without a TileSet might make sense.
Also, some people might just prefer set up the layers beforhands, so I am not really sure we need to forbid that to them as it is technically possible.

I'm not exactly sure if there's anything that can be done about this, but having Y Sort Enabled as a property in both Node2D and in the Layers values is pretty confusing - which one are we supposed to enable? Or do we have to enable both? Same with Z Index I suppose, but with that it sort of makes sense, since one is for the TileMap entirely, the other is for each individual layer. But it's still pretty confusing for new users trying to figure out which one to use when, especially if they have little experience with TileMaps in the first place.

This is kind of a technical requirement. We need to add a warning when you set a layer as Y-sorted and forgot to add Y-sort to the node itself.

When adding a TileSet source, it says only Atlas and Scene Collection. I recognize Atlas is probably the more common name, but it makes it unclear (to me at least) that you can and should be adding images here. At the same time, there's no indication as to what your scene collection might contain, or what that even is. Is it a resource that I have to make in the editor, or do I have to create one in code and add scenes to it in code and set it manually in code? I'm confused because I just attempted to create one in the editor, and there's nothing exported on it that would help me to understand how it's supposed to be used.

This is documentation material, but maybe the tooltip menus should explain what each do. Also, the empty TileSet source list could display a message to drop a texture there to create an atlas, if that helps.

Looking at the physics layers led me to realize at least one part of why I find the new stuff in 4.0 to be very confusing: handling multiple different tilesets happens now in a single tilemap. If folks attempt to use it this way, rather than just having multiple, layered tilemaps, it's going to lead to a ton of problems. This is the definition of too many responsibilities. One potential solution would be to have a parent node that is something like TileMapContainer (bad name, but you get the drift) which has the meta properties for how the child tilemaps function w/r/t each other. So for example you define the YSort properties there. This would allow each tilemap to handle just its own properties and not have to worry about what happens when it's combined together with other tilemaps, making it easier to switch after the fact, if needed; at present, while you can drag layers around and stuff and that's pretty cool, it's fairly unintuitive imo.

Sorry, I don't get that part. But top be clear this isn't really possible as it would likely end up in a situation more complex than the current one.
Also, the whole system of layers was implemented to avoid having to deal with several TileMaps on top of each other, for which a lot of people complained. So I am not sure we want to go back to such a situation again.

I do not understand the purpose of the linear_velocity and angular_velocity values inside the physics layers in the editor itself. What if you have a platform that moves back and forth or up and down within the same space? Do I have to update those values, maybe with an animation player? If I have to update it with an animation player, I can't: it goes away since it's part of the bottom dock as well.

This is for conveyor belts. If you give it a value, a body colliding with it will automatically move according to those values.

Painting physics shapes into any sufficiently complex tileset is at best a laborious job. I love the concept of painting them, but it's just not practical: you need to be able to draw a single physics shape across the entire tileset regardless of where the individual tiles are, and then have the individual tiles take on whatever physics shape is contained in their square when the drawing of the physics shape is done

You don't have to use the painting tool then, just select tiles one by one in the selection tool and edit the shaped in the inspector.
The painting mode has been requested for a while is a lot more efficient to batch set collisions on a ton of tiles. I guess in some situations it might not be the best, but seems like a majority like it a lot.

To facilitate making all the collision shapes, I wonder if it would be possible to have values that define the borders around shapes generally; then, you take the bitmask and calculate where the shapes need to go in order to meet the bitmask's requirements. This kind of shape autogenerator would be tremendously useful, as, often, the navigation shape is the sort of the set difference of the whole tile shape minus the collision shape(s), and so you'd be generating the navigation shape right along with the collision shape. These could then be manually edited in the editor itself and it wouldn't feel quite as bad, since all the work was setup for you in the first place. (Me - probably going to attempt a plugin with this.)

Seems like plugin material, yes. Though I would not be against providing some sort of a "shape palette" for the most common shapes you would use. That might help.

Minor quibble: There are so many properties on tiles now that it's impossible to tell at a glance what the values are for them. It would be nice to just have the collision shapes and navigation shapes show up as differently-colored polygons when a given tile is selected.

I don't really get it, when you select a tile in the tile inspector, you can select a given property, then it will display the polygon for the given property for all tiles in the atlas. I don't think we can do more than that without things becoming a visual mess.

Minor quibble: if you create all the tiles in the tileset and then change the Texture Region Size, it has weird behavior and treats it as one giant tile. But very little is usable when that's the case. I thought maybe that was how to paint across multiple tiles at once, but it really didn't work.

No, this is when your texture has tiles that are wider than the grid cell size. This is a must have for isometric tilesets, where all tile often have a vertical component which make them taller than the grid size.

I'm not sure what Use Texture Padding does, but at least in previous revisions, I know I had to have it turned off or things would not show up correctly.

It needs to be documented, but this makes it so the TileMap renderer internally creates duplicate of atlas texture so that tiles do not leak from one tile to another. It's generally to be turned on, at least for rectangular tiles, but may have a small performance impact.

Terrains in the editor need to have the ability to set a terrain to No terrain simply by tapping a bit that already has a terrain equal to the one you're already attempting to place. The reason for this is simple: I should not have to travel all the way back to the Painting panel to set the terrain back to no terrain just to click on a single misclicked tile bit. I suggest either right-click or a second left-click when the terrain is the same as the one that's already placed in that bit. (Probably me.)

Yes, I agree this is tedious. I think it would be possible to use right-click for that, now that right-click is disabled for scrolling (if I am not mistaking).

If you have multiple tilesets for a given tilemap, how often do you really want multiple, different autotile patterns embedded in the same tileset? I suppose it's possible I'm way off on this, but I just don't see it as necessary to have more than one. That makes being able to add multiple terrains just very confusing to me, especially because you can for some reason have multiple terrains in the same tile, but not in the same bit. I have no clue what that would mean or do.

I am not 100% sure about what you mean, but if you mean Terrain sets, yes, you need to be able to define several of them. Some tiles are made to work with each other while others are not. Typically, in a rectangular shape setup, you may want to have a path-style terrain (requiring the side-only matching mode) and normal 3x3 terrains. Mixing them in the same atlas should be possible.

The Terrain modes are extremely confusing. I guess eventually I'll get used to them, but they aren't intuitively-named at all. Indeed, I'm actually not entirely sure I know what they do now, except that "Match Corners and Sides" works for my ARPG.

This is documentation material.

When selecting colors for the Terrains, they show as full 255 alpha, but when placing them in the editor, they lose alpha somewhere along the way. I'm sure this is to make it possible to see the underlying tile, but it can lead to visual confusion and makes it especially hard for anyone with a vision impairment. I don't have a great solution for this, and the only real solution I can think of is to maybe only show the transparency when the mouse is hovered over the bit.

It would be impossible to set terrains correctly otherwise. I think we should add a transparency slider though, it should definitely help.

Having to click a square when painting to activate it, and then allowing painting only after it has been selected, is somewhat painful and represents an extra click. If you could remove bits with right click or a second click, this wouldn't ever cause any issues because you could just easily remove any accidental clicks as well. The old one worked this way and was much better in that sense: you never had to select a tile when placing bits.

The first click change the tile's terrain set, when the second change the tile's bit. I guess we could make the first click do both, but I think we also need a way to make a bunch of tiles belong to the given terrain set while not setting the bit. So I am not sure what is the best solution there.

Pretty confused about the animation properties. Do you select a set of animation frames as one "tile" in the actual atlas and then define a number of columns? What if it's broken up over multiple rows as well? Frames doesn't allow you to add any frames, even though the plus button is active.

I think we need a warning and some documentation there. Frames should be layered horizontally, and thus the tile to the right of the first one should be empty for you to be able to add a frame. I agree this should be made clearer. The column argument, if different from zero, defines when it should try to go down to a next line in the atlas.

When you put Signal as the type for a Custom Data Layer, what happens? Does it define a new signal for that tile? Lots of confusing stuff in this area. It’s also not well-documented (yet) how to get the relevant data out of that layer, but I’m sure that’ll come in time.

No, it just means you can now assign a signal to each tiles within this data layer. But I guess it does not make a lot of sense, unless in some weird use cases. I'd say most of the time you only would like to use the common int, float, etc... types, but you still have the possibility to assign something else.

[WolfgangSenff]

Alright, everything you said makes a lot of sense. I think the only thing I'm not sure of still (outside of a lot is documentation stuff, which would be very useful indeed!) is painting anything at all. If you still have to select a tile to paint on it, then how is painting any better than just drawing the individual shape? I think that's what's confusing me. Or can you select multiple tiles and paint on them all at once? I'm going to go try that now. :P

Regarding the conveyor belt bits, while that definitely needs to be documented as well (I'm willing to start working on this documentation when the beta comes out but will need your review(s) most likely), I'm wondering if that should be part of the tileset properties. It feels very unrelated.

[groud]

I think that's what's confusing me. Or can you select multiple tiles and paint on them all at once? I'm going to go try that now. :P

I am not sure what you mean, you don't have to select at all when in the painting mode. Here is a gif about how it works:

Peek.15-09-2022.15-18.mp4

You set a shape on the left, then paint it over the whole atlas where you need that shape.

[WolfgangSenff]

Oh! My bad. I must have been doing something wrong, or trying to set it in the wrong place. Seems to be working as intended now.

[SaracenOne]

I think aspects of the first point raised can be fixed with this PR. I fixed it a while back, but it seems to have regressed since then godotengine/godot#65370

[SaracenOne]

Other ideas for the TileMap editor when a tileset is not yet assigned: instead of just having a message telling you that a tileset is not assigned, we could replace it with a button to allow you to assign there too.

[Portponky]

I've been using the tilemap system in Godot 4 quite extensively. It has a lot of great new features, and overall it shows a lot of promise, but it lacks some of the features and ease of use that the 3.x tilemap has. Whilst I'm mostly complaining here, there are a ton of solid improvements, such as layers and custom data, that will be a huge help for me going forward.

Also this tool is just completely awesome:

As someone who primarily develops puzzle games, which are often grid or tile based, the tilemap is a vital tool. Most of my criticism here is for the terrain system, which serves as a replacement/upgrade for the autotiling system in 3.x.

When using the terrain system, there are a number of interactions where the system will alter or erase other tiles. For example, if you have two different terrain sets and attempt to draw them near each other, they will erase each other. I believe this is due to how can_modify_list is populated in TileMap::terrain_fill_connect - it fills in all neighbours regardless if they are of a matching set. It's also awkward that the terrain system matches blank tiles to fill constraints in any situation, rather than leaving tiles as they are. It generally doesn't feel great when the editor changes the type of tiles you haven't painted.

Initially I assumed the ignore_empty_terrains flag had something to do with this, but I have no idea what this flag does. In the code it adds constraints for empty terrains when this flag is false, but as the constraint matching uses a penalty system, avoiding these penalties seems to result in the same outcome as if the flag were true. It's also awkward/inconsistent that the editor draws tiles with this flag set to false, but the API defaults it to true. This could be tidied up, perhaps remove the flag or clarify what its intention is.

Behaviour of terrain is different depending on order of operations, in quite significant ways. As a small example, take this tileset:

I kept it 1d for simplicity. You can draw black lines and green lines, and there's a transition piece from black to green. If you draw a black line and then a green line, they do not connect:

But if you draw the black line second, they do:

The reason is because the constraint propagation handles tiles in the order that TileMap::terrain_fill_connect gets them. It lacks any backtracking you would see in a CSP, or analysis of the entire constraint set that you get in something like WFC.

I'm going to delve more into a core design issue which is a little trickier. First up, I'll quickly bring up Tiled's terrain editors, which I'm sure most reading this will be familiar with.

Tiled's terrain system is more polished and performs better than Godot 4's (not a criticism - Tiled is a mature product specializing in this exact feature). It does do some things quite differently. Namely, the terrains tiles are formulated just as connecters, there's no central bit or type. Similarly, when you draw on the map, you're drawing directly on the constraints, rather than in the tiles. For example, with corner connecting terrains, you draw the terrain types in the corners. It also shows a nice preview so you know what you're getting.

Tiled also doesn't connect tiles to empty space (is this what ignore_empty_terrains is supposed to do?) Terrains only interact within the drawn area.

Now, it might sound like I just want Godot to copy Tiled's terrain system, but this is not the case. In fact, I think Tiled is tailored more towards action level design (like 2D Zelda or maybe Metroid style), and I am much more interested in the general case. For example, a Tiled-like interface wouldn't be a good way to make a Wordle clone, but in Godot one could use a Tilemap to do that, regardless of if that's the best way.

When you assign terrain, you can assign a terrain type to the 'central bit', a feature which was added during the Godot 4 alpha. The way this works is a little odd. You don't have to assign a central bit, but if you don't it makes it tricky for the terrain algorithm to place that tile. This kind of feels like it's awkwardly between what Tiled is trying to do, and what Godot 3.x's autotiling is trying to do. It's definitely editing tiles, rather than constraints, but if you like the tile can be only constraints... but then the tile isn't utilized, because you don't paint constraints.

Lastly, there is no way (that I have found) to implement the equivalent of Godot 3.x's _is_tile_bound feature. This was a useful and relatively popular feature, even if it was a bit messy. There are numerous of youtube videos with thousands of views and dozens of comments explaining this feature. As a puzzle game developer, the lack of this feature is preventing me from easily moving to Godot 4.

There are other minor issues but I don't want this to sound too negative. I'm ony writing so much because the new tilemap is exciting to me and I want it to be the best.

So, instead of just nitpicking, I spent a long time thinking about this, and I have come up with a suggestion that I think would really strengthen the tilemap system, and clear up all the above issues.

• Remove 'ignore_empty_terrain' flag.
• Enforce central bit, both that it must be set, and placed tiles must adhere to it.
• Change the peering bits for each tile to be a mask of suitable connecting terrains types, rather than just one type.

The last point in particular will fix a lot of the behavioural quirks, because it will linearize the constraint satisfaction problem. Placing tiles will be a straightforward search for the best tile in any given location. It'll be order independent and much more performant. This will also act as a full replacement for the functionality of _is_tile_bound, as well as opening the door for simpler API such as get_terrain_type for tiles.

As an example, using the tileset above, letting the green tiles have a left connection which accepts both green and black makes it trivial for the tile placement algorithm to use the right connector.

I would like to hear your thoughts on this. If needed I can write a formal proposal. Thanks!

[TheYellowArchitect]

New godot user here, want to give some feedback. I think what is most needed atm is documentation, as there is no example page or anything for coding with tilemaps. For example, the most basic example is raycasting from your camera to a tile (related to #5538), yet it doesn't exist. I want to code a tilemap into existence (using tilemap as a grid visualization), via a tileset, but the API to TileMap is like a black box to me.

[peter-rocchio]

For the record I agree the the entire concept of tilemap layers doesn't work well.

A tilemap layer is essentially a duplicate of the functionality of a tilemap node but now with lots of extra nesting and very awkward UX. This violates the core philosophy of how Godot nodes are supposed to work with one simple responsibility per node and using multiple nodes in concert for composition.

The only benefit I can see is that there are fewer nodes in the hierarchy, but that isn't particularly important since you can just nest them and collapse the branch anyway, which is how Godot is designed to be used.

Furthermore, layers do not eliminate the need for multiple tilemaps anyway. You still need multiple tilemaps if you want to have tiles of different sizes, or use tilemaps with parallax for example.

[dfkeenan]

Some things I would like.

• I think it would be good to be able to set a (default?) layer for all tile types. I think tiles having a layer would minimize placing them in the wrong layer.

Scene tiles:

• The previews can be quite terrible. Not sure if the TileSet editor should use a custom EditorResourcePreviewGenerator or if this is just a general issue with how the editor generates preview images.
• Having a search filter could be handy.
• It would be great if there was some way you could edit properties of the Scenes being placed. Not sure how feasible this would be. One option I am looking into is adding a "Add tile to scene" option to the Tile properties you can check. So the editor adds a child node to the TileMap rather then placing it as a tile.

[Calinou]

Not sure if the TileSet editor should use a custom EditorResourcePreviewGenerator or if this is just a general issue with how the editor generates preview images.

• See Add ThumbnailCamera2D and ThumbnailCamera3D nodes to use for editor scene thumbnail generation #4759.

[dfkeenan]

• See Add ThumbnailCamera2D and ThumbnailCamera3D nodes to use for editor scene thumbnail generation #4759.

That could be interesting.

[dfkeenan]

I did manage to get checkbox into the Tile properties. But the object it is editing is a "proxy object". And doesn't seem to be an easy way to add metadata to the TileSet from there.