[Docs]flytekit plugin docs improvements in flytesnacks

[kumare3]

Describe the documentation update
Flytesnacks should have an updated documentation for how to author flytekit plugins

• SQL plugins
• flytekit only plugins - task template plugin; New doc
  • https://github.com/flyteorg/flyte/pull/986/files
• flytekit template plugins - Template and task plugins are the same. Custom container (task template/template only plugin), custom behavior (flytekit plugin); New page
  • https://www.youtube.com/watch?v=51hWdfVZHbs&t=33s
• flytekit shim plugins for backend plugins
  • https://www.youtube.com/watch?v=59w9HYOcJJ8&t=1282s
• Type plugins
  • https://www.youtube.com/watch?v=E_X3I5XF-_o
• Data persistence plugins
  • https://github.com/flyteorg/flytekit/tree/master/plugins/flytekit-data-fsspec

[EngHabu]

@samhita-alla is this planned for December? maybe as part of the doc-push?

[samhita-alla]

@samhita-alla is this planned for December? maybe as part of the doc-push?

Oh yeah. That sounds good.

[samhita-alla]

This issue is partly covered in #834.

@cosmicBboy, if taking up the above-mentioned issue, which ones would you likely be working on in the given list?

[cosmicBboy]

Hey @samhita-alla I think some of the problems articulated in #834 have been addressed in this section:
https://docs.flyte.org/projects/cookbook/en/latest/auto/core/extend_flyte/index.html

I think the bullet points above

SQL plugins
flytekit only plugins
flytekit template plugins
flytekit shim plugins for backend plugins
Type plugins
Data persistence plugins

Can be new pages under the "Extending Flyte" page in flytesnacks.

@kumare3 can you clarify the difference between "flytekit only plugins" and "flytekit template plugins?" Just because I think:

• this page already takes care of "Type plugins"
• this page takes care of "flytekit only plugins"

FYI there are these pages in the flytekit API reference:

• https://docs.flyte.org/projects/flytekit/en/latest/extend.html
• https://docs.flyte.org/projects/flytekit/en/latest/tasks.extend.html
• https://docs.flyte.org/projects/flytekit/en/latest/types.extend.html
• https://docs.flyte.org/projects/flytekit/en/latest/data.extend.html

But I think those should basically be a function/class API reference while the new pages under flytesnacks are for user education (and should link to appropriate functions/class in the API reference via intersphinx)

I'm going to close #834. Let me know if you need any help with any of these pages!

update

• we should also improve the docs on how to write backend plugins

[cosmicBboy]

fixes flyteorg/flytesnacks#744

[cosmicBboy]

Closing this issue, breaking up the remaining tasks into:

• [Docs] Guide on how to write Flytekit shim plugins for Backend plugins #2679
• [Docs] Guide on how to write SQL plugins #2680
• [Docs] Guide on how to write data persistence plugins #2681

[cosmicBboy]

Will close this issue when flyteorg/flytesnacks#744 is merged

[samhita-alla]

merged flyteorg/flytesnacks#744