buffers_images.md

Buffers and images

Vulkan may use memory as one of two types of memory objects : buffers and images.

Images are used as simple textures that you place on top of the geometries, you may also use images to store results of rendering ( framebuffer images, swapchain images ) and many more.

Similarly you may use buffers as vertex buffers that send vertex data to vertex shaders, you may also use it as uniform buffers, storage buffers, etc.

In Pumex library each of these objects is implemented using pumex::MemoryBuffer and pumex::MemoryImage class - both classes derive from common pumex::MemoryObject class.

pumex::MemoryBuffer and pumex:Buffer

pumex::Buffer class is a convenient interface to pumex::MemoryBuffer.

It may represent a single structure defined by user :

std::shared_ptr<pumex::Buffer<MyStructure>> buffer;

or it may store a vector of objects defined by user :

std::shared_ptr<pumex::Buffer<std::vector<MyStructure>>> buffer;

To make structure defined by user fully visible to shaders - this structure must obey its alignment rules :

• for uniform buffers it may obey std140 rules
• for storage buffers it must obey std430 rules

Hint: the simplest way to have these requirements met is to use glm::vec4 and glm::mat4 variables only. See tutorial .

pumex::Buffer class has two constructors available :

• we use first constructor when data stored on GPU side may be different for each surface or device ( like for example when buffer stores camera parameters and these parameters are different for each surface ) :

Buffer::Buffer(std::shared_ptr<DeviceMemoryAllocator> allocator, VkBufferUsageFlags bufferUsage, PerObjectBehaviour perObjectBehaviour = pbPerDevice, SwapChainImageBehaviour swapChainImageBehaviour = swForEachImage, bool useSetDataMethods = true);

• second constructor is used when data must be the same on each device or surface ( for example when positions of rendered objects are independent from surface that renders it ) - in that case constructor ensures that pointer to data is stored internally in pumex::Buffer instance.

Buffer::Buffer(std::shared_ptr<T> data, std::shared_ptr<DeviceMemoryAllocator> allocator, VkBufferUsageFlags bufferUsage, PerObjectBehaviour perObjectBehaviour, SwapChainImageBehaviour swapChainImageBehaviour);

Constructor parameters :

• allocator - allocates GPU/CPU memory that Vulkan will use.
• bufferUsage - every created buffer must declare its usage ( will it be used as vertex buffer, index buffer, uniform buffer, etc)
• perObjectBehaviour - buffer may allocate its copies on each used device ( pumex::pbPerDevice ) or on each used surface ( pumex::pbPerSurface ).
• swapChainImageBehaviour - on each device/surface buffer will have only one copy ( pumex::swOnce ), or it will have as many copies as there is swapchain images ( pumex::swForEachImage )
• useSetDataMethods - buffer must declare if it will be sending data using setData() methods. Some buffers do not need to use setData(), because for example its data is generated by compute shaders. This variable is always true for buffers created using second constructor.
• data - data used by second constructor.

During command buffer building surface uses as many primary command buffers as there is swapchain images. If you use three swapchain images - surface creates three primary command buffers : one for each swapchain image. When you acquire a swapchain image - it means that corresponding command buffer is not in use and may be rebuilt if required - or it may just be submited to queue instantly when there's no need to rebuild.

Question is : in what situation there is a need to rebuild a primary command buffer?

For memory buffers the answer is simple: when buffer was removed and created again. When does that happen ? When you want to change the size of the buffer. You may do it only by destroying old buffer and creating a new one with bigger size.

If you have buffer created with pumex::swOnce flag - it means that all three primary command buffers use the same buffer and at least one of them is currently executed on GPU ( command buffer is in pending state ). You cannot remove a memory buffer which is used by command buffer in pending state.

Conclusion : **WHEN YOU KNOW THAT YOUR BUFFER WILL NOT CHANGE SIZE DURING ITS LIFETIME - USE pumex::swOnce FLAG DURING CREATION. ** OTHERWISE ALWAYS USE pumex::swForEachImage FLAG .

setData() and setBufferSize() methods declared in pumex::Buffer class have a group of limitations listed below :

• void setData(const T& dt) - sets data for all surfaces/devices. May be called only when buffer was created with second constructor
• void setData(Surface* surface, std::shared_ptr data) - sets data for specific surface. May only be called when
  • buffer was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerSurface
  • useSetDataMethods is equal to true
• void setData(Surface* surface, const T& data) - same as previous one
• void setData(Device* device, std::shared_ptr data) - sets data for specific device. May only be called when
  • buffer was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerDevice
  • useSetDataMethods is equal to true
• void setData(Device* device, const T& data) - same as previous one
• std::shared_ptr getData() - may be called only when object was created using second constructor
• void invalidateData() - may be called only when object was created using second constructor. After modyfying data pointed by buffer this method should be called to inform buffer that data needs validation.
• void setBufferSize(Surface* surface, size_t bufferSize) - sets buffer size for specific surface. May be called only when object was created using first constructor. This method works when perObjectBehaviour is pumex::pbPerSurface
• void setBufferSize(Device* device, size_t bufferSize) - sets data for specific device. May be called only when object was created using first constructor. This method works when perObjectBehaviour is pumex::pbPerDevice

When user calls setData(), or setBufferSize() methods - the actual operations that update the buffers on GPU side are not called immediately, but are postponed to node validation phase ( for vertex and index buffers ) or to descriptor validation phase - for uniform buffers, storage buffers, etc ( see: Main render loop ).

pumex::MemoryImage

pumex::MemoryImage class is similar to its buffer counterpart. Main difference is that it only accepts gli::texture objects as the ones storing image data. Also there are more methods to manipulate the data.

As with the pumex::Buffer class we have two similar constructors :

• first one is able to use different data on each device/surface ( in case of MemoryImage it not only means that pixels are different, but also that VkImage properties declared by pumex::ImageTraits may be different for each surface/device )

MemoryImage::MemoryImage(const ImageTraits& imageTraits, std::shared_ptr<DeviceMemoryAllocator> allocator, VkImageAspectFlags aspectMask, PerObjectBehaviour perObjectBehaviour = pbPerDevice, SwapChainImageBehaviour swapChainImageBehaviour = swForEachImage, bool sameTraitsPerObject = true, bool useSetImageMethods = true);

• second one has the same data on each device / surface

MemoryImage::MemoryImage(std::shared_ptr<gli::texture> texture, std::shared_ptr<DeviceMemoryAllocator> allocator, VkImageAspectFlags aspectMask = VK_IMAGE_ASPECT_COLOR_BIT, VkImageUsageFlags imageUsage = VK_IMAGE_USAGE_SAMPLED_BIT, PerObjectBehaviour perObjectBehaviour = pbPerDevice);

Constructor parameters :

• imageTraits - describes properties of a VkImage created on a device / surface, such as :
  • image usage ( VkImageUsageFlags )
  • image format ( VkFormat )
  • image extent ( VkExtent3D )
  • number of mipmap levels
  • number of array layers
  • number of samples per texel
  • initial image layout ( VkImageLayout )
  • image type ( VkImageType )
  • and others...
• allocator - allocates GPU/CPU memory that Vulkan will use.
• aspectMask - image aspect mask ( VkImageAspectFlags )
• perObjectBehaviour - image may allocate its copies on each used device ( pumex::pbPerDevice ) or on each used surface ( pumex::pbPerSurface ).
• swapChainImageBehaviour - on each device/surface image will have only one copy ( pumex::swOnce ), or it will have as many copies as there is swapchain images ( pumex::swForEachImage )
• sameTraitsPerObject - image declares if imageTraits will be the same for all VkImage copies on GPU
• useSetImageMethods - image must declare if it will be sending data using setImage() methods. Some images do not need to use setImage(), because for example its data is created during render pass on a GPU side. This variable is always true for images created using second constructor.
• texture - gli::texture object that will be the same for all VkImage copies on GPU side

Remarks concerning pumex::swOnce and pumex::swForEachImage meant for buffers also apply to images.

setImageTraits(), setImage() and clearImage() methods have following limitations :

• void setImageTraits(const ImageTraits& traits) - sets image traits for all surfaces/devices. May be called only when image was created by second constructor.
• void setImageTraits(Surface* surface, const ImageTraits& traits) - sets image traits for specific surface. May only be called when:
  • image was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerSurface
• void setImageTraits(Device* device, const ImageTraits& traits) - sets image traits for specific device. May only be called when:
  • image was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerDevice
• void setImage(Surface* surface, std::shared_ptrgli::texture tex) - sets image data for specific surface. May only be called when:
  • image was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerSurface
  • useSetImageMethods is equal to true
• void setImage(Device* device, std::shared_ptrgli::texture tex) - sets image data for specific device. May only be called when:
  • image was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerDevice
  • useSetImageMethods is equal to true
• void setImages(Surface* surface, std::vector<std::shared_ptr>& images) - sets images created outside MemoryImage as internal. This method is used to register swapchain images, as a objects belonging to MemoryImage. May only be called when:
  • image was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerSurface
  • sameTraitsPerObject is equal to false
• void setImages(Device* device, std::vector<std::shared_ptr>& images) - sets images created outside MemoryImage as internal. This method is used to register swapchain images, as a objects belonging to MemoryImage. May only be called when:
  • image was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerDevice
  • sameTraitsPerObject is equal to false
• void clearImages(const glm::vec4& clearValue, const ImageSubresourceRange& range) - clears image with provided value on for all surfaces/devices
• void clearImage(Surface* surface, const glm::vec4& clearValue, const ImageSubresourceRange& range) - clears image with provided value on a specific surface. May only be called when:
  • image was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerSurface
• void clearImage(Device* device, const glm::vec4& clearValue, const ImageSubresourceRange& range) - clears image with provided value on a specific surface. May only be called when:
  • image was created by first constructor
  • perObjectBehaviour is equal to pumex::pbPerDevice

Descriptor resources

Buffers and images are not used directly in Pumex ( except for vertex and index buffers ), but through descriptors. Descriptor requires pumex::Resource class descendant to know how to interpret data in buffer or image.

Currently Pumex library has following pumex::Resource descendants for buffers :

• pumex::UniformBuffer
• pumex::StorageBuffer

There are following pumex::Resource descendants available for pumex::MemoryImage :

• pumex::SampledImage
• pumex::StorageImage
• pumex::CombinedImageSampler
• pumex::InputAttachment

There is also one resource that is not associated directly with Images, but it's necessary for sampllng pumex::SampledImage in a shaders :

• pumex::Sampler

Examples

MemoryBuffers are connected to resource directly :

auto cameraBuffer = std::make_shared<pumex::Buffer<pumex::Camera>>(buffersAllocator, VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT, pumex::pbPerSurface, pumex::swOnce, true);

auto descriptorSet = std::make_shared<pumex::DescriptorSet>(descriptorPool, descriptorSetLayout);
descriptorSet->setDescriptor(0, std::make_shared<pumex::UniformBuffer>(cameraBuffer));

MemoryImages are connected to Resource through ImageView class :

auto volumeMemoryImage = std::make_shared<pumex::MemoryImage>(volumeImageTraits, volumeAllocator, VK_IMAGE_ASPECT_COLOR_BIT, pumex::pbPerSurface, pumex::swOnce);

auto volumeImageView = std::make_shared<pumex::ImageView>(volumeMemoryImage, volumeMemoryImage->getFullImageRange(), VK_IMAGE_VIEW_TYPE_3D);

auto descriptorSet = std::make_shared<pumex::DescriptorSet>(descriptorPool, descriptorSetLayout);
descriptorSet->setDescriptor(0, std::make_shared<pumex::SampledImage>(volumeImageView));

There exists another way to connect images and buffers - when images or buffers are associated to render graphs ( using viewer->getExternalMemoryObjects()->addMemoryObject() method before render graph is compiled ). In case of images pumex::RenderWorkflow creates pumex::ImageView objects for us :

auto volumeMemoryImage = std::make_shared<pumex::MemoryImage>(volumeImageTraits, volumeAllocator, VK_IMAGE_ASPECT_COLOR_BIT, pumex::pbPerSurface, pumex::swOnce);

pumex::ResourceDefinition image_3d(pumex::rmtImage, "image_3d");

viewer->getExternalMemoryObjects()->addMemoryObject("voxels", image_3d, volumeMemoryImage, VK_IMAGE_VIEW_TYPE_3D);

auto descriptorSet = std::make_shared<pumex::DescriptorSet>(descriptorPool, descriptorSetLayout);
descriptorSet->setDescriptor(0, std::make_shared<pumex::StorageImage>("voxels"));

Input attachments are created internally by pumex::RenderWorkflow. Each input attachment is associated to a render graph by default. To sample input attachment as an image you need to :

• provide sampler in pumex::InputAttachment constructor
• use provided external sampler in a shader.

First case looks like this :

auto iaSampler = std::make_shared<pumex::Sampler>(pumex::SamplerTraits());

auto descriptorSet = std::make_shared<pumex::DescriptorSet>(descriptorPool, descriptorSetLayout);
descriptorSet->setDescriptor(0, std::make_shared<pumex::InputAttachment>("albedo", iaSampler));

Second case :

auto descriptorSet = std::make_shared<pumex::DescriptorSet>(descriptorPool, descriptorSetLayout);
descriptorSet->setDescriptor(0, std::make_shared<pumex::InputAttachment>("albedo"));
descriptorSet->setDescriptor(1, std::make_shared<pumex::Sampler>(pumex::SamplerTraits()));