This repository has been archived by the owner on Sep 20, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 129
Shotgrid: Add production beta of shotgrid integration #2921
Merged
Merged
Changes from all commits
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
0e96071
Add shotgrid module
victorbartel 00285ca
Fix hound complaints
victorbartel feae04f
Fix hound Exception complaints
victorbartel 521b562
Add fixes after first PR
victorbartel 68941da
Add fixes for hound
victorbartel 1214cb9
Remove an empty line
victorbartel d106ba4
Reformat code after hound barking
victorbartel 3c1ecfb
Drawn back to the obsolete version of python
victorbartel 3a3e552
Make the publish work under maya 2018
BenoitConnan 0c85093
Add memoize implementation for settings purposes
victorbartel 16dc433
Fix after hound notes
victorbartel 57386db
Fix after hound notes, part 2
victorbartel 60dd3df
Merge branch 'develop' into shotgrid_module_update
iLLiCiTiT 2f0bbee
Merge pull request #5 from pypeclub/shotgrid_module_update
ClementHector 3258273
Merge branch 'develop' into shotgrid_module
ClementHector 29e095e
Remove avalon DB path
ClementHector 26b6bba
update poetry lib
ClementHector 8a06a4a
remove unused import
ClementHector 0163eae
Remove unused code and add sugestions
ClementHector d49c2ba
reverted poetry.lock changes
iLLiCiTiT 3485529
Merge branch 'develop' into bugfix/rever_poetry_lock_changes
iLLiCiTiT b4c6cb4
remove unused import
ClementHector 151bf29
Remove unused code and add sugestions
ClementHector ad0aea0
remove unused method
ClementHector 50ec61f
use context to get shotgrid project id
ClementHector 47d2a61
update poetry lock
ClementHector 1fd8723
Merge pull request #6 from pypeclub/bugfix/rever_poetry_lock_changes
ClementHector ad3380b
remove unused import
ClementHector 0417fb1
poetry lock rebase
ClementHector e5fbdd1
added shotgun-api3 source files to poetry lock
iLLiCiTiT f85865a
Merge pull request #7 from pypeclub/bugfix/last_poetry_fix
ClementHector 144ef08
change default settings
ClementHector 83db3de
Merge branch 'shotgrid_module' of https://github.com/Ellipsanime/Open…
ClementHector File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -102,3 +102,6 @@ website/.docusaurus | |
|
||
.poetry/ | ||
.python-version | ||
.editorconfig | ||
.pre-commit-config.yaml | ||
mypy.ini |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
## Shotgrid Module | ||
|
||
### Pre-requisites | ||
|
||
Install and launch a [shotgrid leecher](https://github.com/Ellipsanime/shotgrid-leecher) server | ||
|
||
### Quickstart | ||
|
||
The goal of this tutorial is to synchronize an already existing shotgrid project with OpenPype. | ||
|
||
- Activate the shotgrid module in the **system settings** and inform the shotgrid leecher server API url | ||
|
||
- Create a new OpenPype project with the **project manager** | ||
|
||
- Inform the shotgrid authentication infos (url, script name, api key) and the shotgrid project ID related to this OpenPype project in the **project settings** | ||
|
||
- Use the batch interface (Tray > shotgrid > Launch batch), select your project and click "batch" | ||
|
||
- You can now access your shotgrid entities within the **avalon launcher** and publish informations to shotgrid with **pyblish** |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
from .shotgrid_module import ( | ||
ShotgridModule, | ||
) | ||
|
||
__all__ = ("ShotgridModule",) |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
MODULE_NAME = "shotgrid" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,125 @@ | ||
|
||
from urllib.parse import urlparse | ||
|
||
import shotgun_api3 | ||
from shotgun_api3.shotgun import AuthenticationFault | ||
|
||
from openpype.lib import OpenPypeSecureRegistry, OpenPypeSettingsRegistry | ||
from openpype.modules.shotgrid.lib.record import Credentials | ||
|
||
|
||
def _get_shotgrid_secure_key(hostname, key): | ||
"""Secure item key for entered hostname.""" | ||
return f"shotgrid/{hostname}/{key}" | ||
|
||
|
||
def _get_secure_value_and_registry( | ||
hostname, | ||
name, | ||
): | ||
key = _get_shotgrid_secure_key(hostname, name) | ||
registry = OpenPypeSecureRegistry(key) | ||
return registry.get_item(name, None), registry | ||
|
||
|
||
def get_shotgrid_hostname(shotgrid_url): | ||
|
||
if not shotgrid_url: | ||
raise Exception("Shotgrid url cannot be a null") | ||
valid_shotgrid_url = ( | ||
f"//{shotgrid_url}" if "//" not in shotgrid_url else shotgrid_url | ||
) | ||
return urlparse(valid_shotgrid_url).hostname | ||
|
||
|
||
# Credentials storing function (using keyring) | ||
|
||
|
||
def get_credentials(shotgrid_url): | ||
hostname = get_shotgrid_hostname(shotgrid_url) | ||
if not hostname: | ||
return None | ||
login_value, _ = _get_secure_value_and_registry( | ||
hostname, | ||
Credentials.login_key_prefix(), | ||
) | ||
password_value, _ = _get_secure_value_and_registry( | ||
hostname, | ||
Credentials.password_key_prefix(), | ||
) | ||
return Credentials(login_value, password_value) | ||
|
||
|
||
def save_credentials(login, password, shotgrid_url): | ||
hostname = get_shotgrid_hostname(shotgrid_url) | ||
_, login_registry = _get_secure_value_and_registry( | ||
hostname, | ||
Credentials.login_key_prefix(), | ||
) | ||
_, password_registry = _get_secure_value_and_registry( | ||
hostname, | ||
Credentials.password_key_prefix(), | ||
) | ||
clear_credentials(shotgrid_url) | ||
login_registry.set_item(Credentials.login_key_prefix(), login) | ||
password_registry.set_item(Credentials.password_key_prefix(), password) | ||
|
||
|
||
def clear_credentials(shotgrid_url): | ||
hostname = get_shotgrid_hostname(shotgrid_url) | ||
login_value, login_registry = _get_secure_value_and_registry( | ||
hostname, | ||
Credentials.login_key_prefix(), | ||
) | ||
password_value, password_registry = _get_secure_value_and_registry( | ||
hostname, | ||
Credentials.password_key_prefix(), | ||
) | ||
|
||
if login_value is not None: | ||
login_registry.delete_item(Credentials.login_key_prefix()) | ||
|
||
if password_value is not None: | ||
password_registry.delete_item(Credentials.password_key_prefix()) | ||
|
||
|
||
# Login storing function (using json) | ||
|
||
|
||
def get_local_login(): | ||
reg = OpenPypeSettingsRegistry() | ||
try: | ||
return str(reg.get_item("shotgrid_login")) | ||
except Exception: | ||
return None | ||
|
||
|
||
def save_local_login(login): | ||
reg = OpenPypeSettingsRegistry() | ||
reg.set_item("shotgrid_login", login) | ||
|
||
|
||
def clear_local_login(): | ||
reg = OpenPypeSettingsRegistry() | ||
reg.delete_item("shotgrid_login") | ||
|
||
|
||
def check_credentials( | ||
login, | ||
password, | ||
shotgrid_url, | ||
): | ||
|
||
if not shotgrid_url or not login or not password: | ||
return False | ||
try: | ||
session = shotgun_api3.Shotgun( | ||
shotgrid_url, | ||
login=login, | ||
password=password, | ||
) | ||
session.preferences_read() | ||
session.close() | ||
except AuthenticationFault: | ||
return False | ||
return True |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
|
||
class Credentials: | ||
login = None | ||
password = None | ||
|
||
def __init__(self, login, password) -> None: | ||
super().__init__() | ||
self.login = login | ||
self.password = password | ||
|
||
def is_empty(self): | ||
return not (self.login and self.password) | ||
|
||
@staticmethod | ||
def login_key_prefix(): | ||
return "login" | ||
|
||
@staticmethod | ||
def password_key_prefix(): | ||
return "password" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
from openpype.api import get_system_settings, get_project_settings | ||
from openpype.modules.shotgrid.lib.const import MODULE_NAME | ||
|
||
|
||
def get_shotgrid_project_settings(project): | ||
return get_project_settings(project).get(MODULE_NAME, {}) | ||
|
||
|
||
def get_shotgrid_settings(): | ||
return get_system_settings().get("modules", {}).get(MODULE_NAME, {}) | ||
|
||
|
||
def get_shotgrid_servers(): | ||
return get_shotgrid_settings().get("shotgrid_settings", {}) | ||
|
||
|
||
def get_leecher_backend_url(): | ||
return get_shotgrid_settings().get("leecher_backend_url") |
100 changes: 100 additions & 0 deletions
100
openpype/modules/shotgrid/plugins/publish/collect_shotgrid_entities.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
import os | ||
|
||
import pyblish.api | ||
from openpype.lib.mongo import OpenPypeMongoConnection | ||
|
||
|
||
class CollectShotgridEntities(pyblish.api.ContextPlugin): | ||
"""Collect shotgrid entities according to the current context""" | ||
|
||
order = pyblish.api.CollectorOrder + 0.499 | ||
label = "Shotgrid entities" | ||
|
||
def process(self, context): | ||
|
||
avalon_project = context.data.get("projectEntity") | ||
avalon_asset = context.data.get("assetEntity") | ||
avalon_task_name = os.getenv("AVALON_TASK") | ||
|
||
self.log.info(avalon_project) | ||
self.log.info(avalon_asset) | ||
|
||
sg_project = _get_shotgrid_project(context) | ||
sg_task = _get_shotgrid_task( | ||
avalon_project, | ||
avalon_asset, | ||
avalon_task_name | ||
) | ||
sg_entity = _get_shotgrid_entity(avalon_project, avalon_asset) | ||
|
||
if sg_project: | ||
context.data["shotgridProject"] = sg_project | ||
self.log.info( | ||
"Collected correspondig shotgrid project : {}".format( | ||
sg_project | ||
) | ||
) | ||
|
||
if sg_task: | ||
context.data["shotgridTask"] = sg_task | ||
self.log.info( | ||
"Collected correspondig shotgrid task : {}".format(sg_task) | ||
) | ||
|
||
if sg_entity: | ||
context.data["shotgridEntity"] = sg_entity | ||
self.log.info( | ||
"Collected correspondig shotgrid entity : {}".format(sg_entity) | ||
) | ||
|
||
def _find_existing_version(self, code, context): | ||
|
||
filters = [ | ||
["project", "is", context.data.get("shotgridProject")], | ||
["sg_task", "is", context.data.get("shotgridTask")], | ||
["entity", "is", context.data.get("shotgridEntity")], | ||
["code", "is", code], | ||
] | ||
|
||
sg = context.data.get("shotgridSession") | ||
return sg.find_one("Version", filters, []) | ||
|
||
|
||
def _get_shotgrid_collection(project): | ||
client = OpenPypeMongoConnection.get_mongo_client() | ||
return client.get_database("shotgrid_openpype").get_collection(project) | ||
|
||
|
||
def _get_shotgrid_project(context): | ||
shotgrid_project_id = context.data["project_settings"].get( | ||
"shotgrid_project_id") | ||
if shotgrid_project_id: | ||
return {"type": "Project", "id": shotgrid_project_id} | ||
return {} | ||
|
||
|
||
def _get_shotgrid_task(avalon_project, avalon_asset, avalon_task): | ||
sg_col = _get_shotgrid_collection(avalon_project["name"]) | ||
shotgrid_task_hierarchy_row = sg_col.find_one( | ||
{ | ||
"type": "Task", | ||
"_id": {"$regex": "^" + avalon_task + "_[0-9]*"}, | ||
"parent": {"$regex": ".*," + avalon_asset["name"] + ","}, | ||
} | ||
) | ||
if shotgrid_task_hierarchy_row: | ||
return {"type": "Task", "id": shotgrid_task_hierarchy_row["src_id"]} | ||
return {} | ||
|
||
|
||
def _get_shotgrid_entity(avalon_project, avalon_asset): | ||
sg_col = _get_shotgrid_collection(avalon_project["name"]) | ||
shotgrid_entity_hierarchy_row = sg_col.find_one( | ||
{"_id": avalon_asset["name"]} | ||
) | ||
if shotgrid_entity_hierarchy_row: | ||
return { | ||
"type": shotgrid_entity_hierarchy_row["type"], | ||
"id": shotgrid_entity_hierarchy_row["src_id"], | ||
} | ||
return {} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have few questions regarding this function.
At least it looks it's done that way. I maybe missed that part of shotgrid implementation?
_id
contains strings?Few comments:
Please try avoid using
avalon
in variables and also be more specific about the content (e.g.asset_doc
ortask_name
tells more about it's content), also pass project document just to use it's name can be simplified to pass just the name?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my memory, it seemed simpler ans more maintainable for @victorbartel to store shotgrid data in another database and store in avalon on the asset that corresponds to the id of this database
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would make sense in SQL database, not in mongo.
Now you have to query twice from 2 different databases to get data related to the same entity. To get shotgun entity based on asset name you have to query it from avalon database to get it's id, then based on the id query it's shotgrid id from shotgrid database and then query it from shotgrid server? Performance of that with cloud mongo wouldn't be great.
And
_id
key is notObjectId
but string, which is not requirement, but the ObjectId is used for indexing in mongo collections.