cosmogonies.net Blog of cosmogonies.net

feedback
18Apr/130

Shotgun (DAM) python object wrapper

shotgun

Asset Manager is a great way to fluidify your workflow.

As a DAM (Digital Asset Manager) has a fuzzy definition, I will quickly describe how I see Shotgun.
Shotgun does the boring thing you do not want to do when dealing with digital assets.
Shotgun types them, calling them Entity, record them and all kind of metadatas attached to them (string, float, date, entites, etc.. and lists), and call that Fields.
In facts, Shotgun is a web-page (internet or intranet), displaying in a very user-friendly way those datas (a lot more elaborated and cgi-oriented than a phpMyAdmin for example).
So basically, it is a back-end database (postGre), and a front-end browsing page.

What it does NOT, is storing files, or have any further connection with a filesystem but some string fields where you can put paths.
Now the new TANK extension of shotgun could do that, I will talk about it later.

So the method I present here is how deal very easily with shotgun entities as objects, manipulating very semantically with calls like

print myAsset.description, myScene.created_at, myScene.created_by

Using the analogy: Entity=Object, Field=Member of python class instances.

This is how I organized the package (ALK_ prefix stands for the studio I currently work, Alkymia):
ALK_Entity\ (Package name)
ALK_Entity\__init__.py (Main superclass to inherit from shotgun API)
ALK_Entity\Scenes.py (sub-classes definition, one representing the exact shotgun entity, the other to simply add your additional stuff. So the module name is the Entity name in plural.)
ALK_Entity\Shots.py
ALK_Entity\Assets.py
... #and so on

ShotgunWrapperUML

From this UML diagram on the left, you can see how the package hierarchy works.

With that in mind, you can graph easily a real network of all your assets.
You will obtain a dependency graph, of relationships as "needs" or "is a part of" that is really appreciated in object-oriented scripting. It is really powerful.

The dark side of it, as you may notice parsing my code, is that every entity is requested with all fields, which is not very performance.
Good usability versus good performance is often a very tricky balance to handle.
(Mistake already done: In my first code iteration, I had put the request of filed as entity into the constructor, not the attribute getter.
Result: any simple entity was bouncing all over dependencies and links, putting the whole f** database into RAM for one single asset^^.)

So, this is the final result looks like, and this is how I deal with our metadatas in the pipeline tools I made:

###
import ALK_Entity.Scenes

myScene = ALK_Entity.Scenes.ALK_Scene("S004_M001")
print type(myScene)  # <class 'ALK_Entity.Scenes.ALK_Scene'>
print myScene.id, myScene.code  # 794 S004_M001

print myScene.sg_sequence  # [<ALK_Entity.Sequences.ALK_Sequence object at 0x0000000038BADE80>]
print myScene.sg_sequence[0].code  # S004

mySceneById = ALK_Entity.Scenes.ALK_Scene(794)
print mySceneById.code  # S004_M001
###

If you are interested, here is an example of python scripts I wrote.
I shared them "as it is", and with quick mods to be more clear and subject-centric.
This modifications could have altered functionality, you have my apologies if so.
( In another blog post, I will describe how I setup my production versioning system linked to shotgun entities. )

""" File: ...\ALK_Entity\__init__.py  """

# -*- coding: utf-8 -*-
""" Contains all connections to the Asset Management (Shotgun API wrapper). """
import os
import shotgun_api3.shotgun
_PROJECT_ID = 00  # We only worked on one project so I made very few cases of multi-project instances.

class ShotgunEntity(object):
  """Representation in python object of an entity in shotgun."""

  def getShotgunHandle(self):
    """ Default method that needs to be overloaded in sub-classes """
    import shotgun_api3.shotgun
    SERVER_PATH = "http://your_shotgun_url"
    SCRIPT_NAME = 'Framework_Connector'
    SCRIPT_KEY = '##########################################'
    return shotgun_api3.shotgun.Shotgun(SERVER_PATH, SCRIPT_NAME, SCRIPT_KEY)

  def getEntityType(self):
    """ Default method that needs to be overloaded in sub-classes """
    return 'DefaultValue'

  def getNameFilter(self,_name):
    """ Default method that needs to be overloaded in sub-classes """
    #Must overload that, for example tasks has no 'code' but 'content'...
    return ['code','contains',_name]

  def getProjectFilter(self):
    """ Default method that needs to be overloaded in sub-classes """
    return ['project','is',{'type':'Project','id':_PROJECT_ID}]

  def updateFromDataBase(self,_sg_Id):
    """Looking for the shotgun Asset and retrieve its metadatas.
    Be aware that you can use shotgun Id or Name to find the Asset."""

    shotgunHandle = self.getShotgunHandle()
    filters =[]

    if self.getProjectFilter()!=None:
      filters.append(self.getProjectFilter())

    if type(_sg_Id) == str:
      filters.append(self.getNameFilter(_sg_Id))
    elif type(_sg_Id) == int:
      filters.append(['id','is',_sg_Id])
    else:
      pass
      #put here your throwing error system

    fields = self.__dict__.keys()
    #All fields, that does not fit well for high performance or big requests on huge database of course.
    Assetdata = shotgunHandle.find_one(self.getEntityType(), filters, fields)
    shotgunHandle.close()

    if not Assetdata:
      print("ERROR, no entry in shotgun DB for entity with id = "+str(_sg_Id)+" ("+str(self.getEntityType())+")  ",eVerboseLevel.kError)
      print("Filters were ="+str(filters),eVerboseLevel.kError)
      raise ALK_InputOutput.Debug.ShotGunEntryMissing("ERROR, no entry in shotgun DB for Asset !")
    else:
      for currentmember in fields:
        if not Assetdata.has_key(currentmember):
          print("ERROR, no entry in shotgun DB for member :"+currentmember,eVerboseLevel.kError)
        else:
          setattr(self, currentmember, Assetdata[currentmember] )

  def __getattribute__(self,name):
    """ When a member of the instance of the class is queried, we get its data. """

    myValue = object.__getattribute__(self, name)  # getting a member value given its name as a string.

    if(isinstance(myValue, list)):

      entityList=[]
      for currentSubObj in myValue:
        newEntity = self.__castFromDictToEntity(currentSubObj)
        entityList.append(newEntity)
      setattr(self, name, entityList)
      return entityList

    else:
      if(isinstance(myValue, dict)):
        newEntity = self.__castFromDictToEntity(myValue)
        setattr(self, name, newEntity)
        return newEntity
      else:
        return myValue

    return None

  def __castFromDictToEntity(self, _data):
    """ With the native python's shotgun wrapper, every entity is represented as a dict, this will cast them """

    #every shotgun dict has an id and a type (every id is unique based on its type, often show as entity name)
    if(isinstance(_data, dict)):
      if(_data.has_key('id') and _data.has_key('type')):

        myType = _data['type']
        myALK_Entity = _createEntityByName(myType, _data['id'])
        return myALK_Entity
    return _data

  def updateIntoDataBase(self):
    """" Publish all members from python RAM into shotgun Database."""
    #import shotgun_api3.shotgun

    shotgunHandle = self.getShotgunHandle()
    fields = self.__dict__.keys()
    memberDict = {}
    for currentmember in fields:
      memberDict[currentmember] = getattr(self, currentmember)

    AssetdataNew = shotgunHandle.update(self.getEntityType(), self.id, memberDict)
    shotgunHandle.close()

  def updateMemberIntoDataBase(self,_memberNameToUpdate):
    """" Publish given member from python RAM into shotgun Database."""

    shotgunHandle = self.getShotgunHandle()

    memberDict={}
    memberDict[_memberNameToUpdate] = getattr(self, _memberNameToUpdate)
    AssetdataNew = shotgunHandle.update(self.getEntityType(), self.id, memberDict)

    print("Asset Updated in shotgun :" + str(AssetdataNew))
    shotgunHandle.close()

  def _createEntityByName(_TypeName, _Id):
    """ Cast from a type as string into a ALK_Entity wrapped instance. """

    if _Id==None:
      return None

    if _TypeName == 'Asset':
      import ALK_Entity.Assets
      return ALK_Entity.Assets.ALK_Asset(_Id)

    if _TypeName=='Scene':
      import ALK_Entity.Scenes
      return ALK_Entity.Scenes.ALK_Scene(_Id)

    if _TypeName=='Version':
      import ALK_Entity.Versions
      return ALK_Entity.Versions.ALK_Version(_Id)

    #And so on for all your managed entities.
    printt("Un-managed shogun type = "+_TypeName)

And now this is an entity wrap of a scene:
""" File: ...\ALK_Entity\Scenes.py  """

# -*- coding: utf-8 -*-
""" Wrapper for Scenes into the Asset Manager Database. """
import os
import ALK_InputOutput.Debug
import ALK_Entity

_SERVER_PATH = "http://your_shotgun_url" # make sure to change this to https if your studio uses it.
_SCRIPT_NAME = 'ALK_Scene' #In your administration panel, be sure to create as many scripts as you have entities wrapper.
_SCRIPT_KEY = '############################'

class __Shotgun_Scene(ALK_Entity.ShotgunEntity):
    """Representation in python object of a Scene entity in shotgun.
    We assume that all members of this class is a field with a same name."""

    def __init__(self,_sg_Id):
        """Constructor of the class with defaultValues and next updated them from database."""

        self.addressings_cc = 'DefautValue'
        self.assets = 'DefautValue'
        self.code = 'DefautValue'
        self.created_at = 'DefautValue'
        self.created_by = 'DefautValue'
        self.description = 'DefautValue'
        self.id = 'DefautValue'
        self.image = 'DefautValue'
        self.notes = 'DefautValue'
        self.open_notes = 'DefautValue'
        self.open_notes_count = 'DefautValue'
        self.project = 'DefautValue'
        self.sg_scene_type = 'DefautValue'
        self.sg_status_list = 'DefautValue'
        self.shoot_days = 'DefautValue'
        self.shots = 'DefautValue'
        self.step_0 = 'DefautValue'
        self.tag_list = 'DefautValue'
        self.tasks = 'DefautValue'
        self.task_template = 'DefautValue'
        self.updated_at = 'DefautValue'
        self.updated_by = 'DefautValue'
        #Custom fields
        self.sg_cameras = 'DefautValue'
        self.sg_sequence = 'DefautValue'#Warning it is a list of dictionnaries

        self.updateFromDataBase(_sg_Id)

    def getShotgunHandle(self):
        import shotgun_api3.shotgun
        return shotgun_api3.shotgun.Shotgun(_SERVER_PATH, _SCRIPT_NAME, _SCRIPT_KEY)

    def getEntityType(self):
        return 'Scene'

    def getNameFilter(self,_name):
        return ['code','contains',_name]

class ALK_Scene(__Shotgun_Scene):
    """Custom class to implement metadatas of Scenes without shotgun synchronisation."""

    def __init__(self,_sg_Id):
        """ Constructor. """
        #First calling the inherited constructor:
        super(ALK_Scene,self).__init__(_sg_Id)

        #TODO verif if sequence.code is in scene.code

    def getFolder(self,_task):
        import ALK_Entity.Projects
        import ALK_Entity.Sequences

        if len(self.sg_sequence)>=1:
            mySeq = self.sg_sequence[0]
            #print mySeq.code
            #print _task.step.code
            if _task!=None:
                myFilePath = ALK_Entity.Projects.getZeProject().getProjectDirectory()+"\\Exploitation\\"+mySeq.code+"\\scenes\\"+_task.step.code+"\\"+_task.content
            #maybe replace myFilePath = ALK_Entity.Projects.getZeProject().getProjectDirectory()+"\\Exploitation\\"+mySeq.code+"     by ALK_Seq.getFolder()
                return myFilePath
            else:
                printt("No task associated to this scene !", eVerboseLevel.kError)
                raise ALK_InputOutput.Debug.ShotGunEntryMissing("ERROR, no entry in shotgun DB for this task !"+self.code)
        else:
            printt("No sequence associated to this scene !",eVerboseLevel.kError)

            #raise

    def getRange(self):
        import sys
        import ALK_Entity.Versions

        StartFrame = sys.maxint
        EndFrame = 0

        for (currentVersion,currentShot,currentCamera) in ALK_Entity.Versions.getAllValidated(self):
            print currentVersion.code , currentCamera.sg_short_name, "In=" ,currentVersion.sg_cut_in,"Out=",currentVersion.sg_cut_out

            if currentVersion.sg_cut_inEndFrame:
                EndFrame = currentVersion.sg_cut_out

        return (StartFrame, EndFrame)

def getSceneList():
    import shotgun_api3.shotgun
    myShotgunHandle = shotgun_api3.shotgun.Shotgun(_SERVER_PATH, _SCRIPT_NAME, _SCRIPT_KEY)

    allItems = myShotgunHandle.find('Scene', [['project','is',{'type':'Project','id':ALK_Entity._PROJECT_ID}]])
    result=[]
    for currentItem in allItems:
        newItem = ALK_Scene(currentItem['id'])
        result.append(newItem)
    return result

news
Comments (0) Trackbacks (0)

No comments yet.


Leave a comment

about

No trackbacks yet.

report
about