...

Modify Collibra Assets In Bulk

Rupal Sarkar

Aug. 11, 2022

Modify assets in Bulk

In this blog you will learn to create a workflow that helps to modify assets in Bulk.

 

  1. You start the workflow by setting up some process variables like the universally unique identifiers (UUIDs) for the attribute types. This also makes the variables available for later edits on the workflow definition page in Collibra Data Governance Center.

  2. You then write a script task retrieve all the assets under a domain and insert that into a HashMap.

  3. With the first user task, you ask start user to select assets from the checkbox provided. Here you use dynamic_checkbox variable to create a dynamic checkbox.

  4. Before moving forward, you write a script task to initialize variables - counter and selectedAsset. As you'll be modifying assets in a loop, counter decides when to stop the iteration. The other variable selectedAsset stores the asset's UUIDs.

  5. With the new user task, you ask the startUser to enter new attribute Values and store that into variables.

  6. You create a script task to implement the changes. If an attribute of the selected asset didn't have a value earlier, then you'll add new attribute value using AddAttributeRequest.Builder. Otherwise you simply change the attribute value using ChangeAttributeRequest.Builder.

  7. Finally while creating the sequence flow, you have to define a condition to repeat step 5 and step 6 in a loop. 

 

Add a pool and lane

  1. Add a pool and configure the pool properties:

    1. General:
      • Id: modify_assets_in_bulk_pool
        Name: Modify Assets In Bulk
    2. Process:
      • Id: modify_assets_in_bulk
      • Name: Modify Assets In Bulk
      • Namespace: http://www.collibra.com/apiv2

Add and configure the start event

  1. From the Start event section of the Palette, drag a StartEvent

  2. In the Properties view, select the Main config section. In the Initiator field, enter a name for the variable that defines the user who starts the workflow. We are using startUser in this example. The value of the variable will be the username of that user

  3. In the Properties view, select the Form section

  4. Add a variable to set the UUID of the Definition attribute type:

    1. Click New

    2. Enter the required information:
      • Id: uuid_asset_definition
      • Name: The UUID of the Definition attribute type
      • Type: string
      • Default: 00000000-0000-0000-0000-000000000202
      • Readable: False
      • Writeable: True
      • Required: True
  5. Add a variable to set the UUID of the Descriptive Example attribute type:
    1. Click New

    2. Enter the required information:
      • Id: uuid_asset_descriptive_example
      • Name: The UUID of the Descriptive Example attribute type
      • Type: string
      • Default: 00000000-0000-0000-0000-000000003115
      • Readable: False
      • Writeable: True
      • Required: True

Create a script to fetch all the assets under a particular domain

  1. From the Task section of the Palette, drag a ScriptTask to the lane.
  2. In the General section, enter a Name: Create list of assets.
  3. In the Main config section:
    • Select the groovy script language.
    • Add the script to set up the initial variables
import java.util.HashMap
import java.util.Map
import com.collibra.dgc.core.api.model.instance.Asset
import com.collibra.dgc.core.api.dto.instance.asset.FindAssetsRequest
import com.collibra.dgc.core.api.model.instance.Domain
import com.collibra.dgc.core.api.model.instance.Community

def buildAndRegister = {
    Domain currentDomain = domainApi.getDomain(item.getId());
    UUID initialCommunityId = currentDomain.getCommunity().getId();
    Map<String, String> dynamicMapOfValues = new HashMap<String, String>();
    def lst = []

    fromAssets = assetApi.findAssets(FindAssetsRequest.builder()
        .domainId(currentDomain.getId())
        .build())
        .getResults();
    for(asset in fromAssets) {
        String assetDisplayName = (asset.getDisplayName().toString());
        lst.add(asset.getId())
        dynamicMapOfValues.put(asset.getId().toString(), "<em><b>"+assetDisplayName+"</b></em>");
    }
    execution.setVariable("dynamicMapOfValues", dynamicMapOfValues.sort { it.value.toLowerCase() });
    execution.setVariable('proposedTerms', lst)
}
buildAndRegister();
  1. Connect the start event to the script task with a SequenceFlow connection.

Create a user task to select assets

The user task ask the start user to select assets from the checkbox provided.

  1. From the Task section of the Palette drag a UserTask to the lane.
  2. Connect the previous script task to the user task with a SequenceFlow connection.
  3. Select the user task.
  4. In the Properties view, select the Main Config section and enter the Candidate user:
    user(${startUser}).
  5. In the Properties view, select the Form section and add the following elements to the dialog box:
    1. A drop-down list of available assets:
      • Id: dynamic_checkbox
      • Name: <b>Choose assets:</b>
      • Type: dynamicCheckbox
      • Readable: True
      • Writeable: True
      • Required: True
      • Form Values:
        • Id: dynamicValuesExpression
        • Name: ${dynamicMapOfValues}
    2. A button to submit:
      • Id: next
      • Name: Submit
      • Type: button
      • Readable: True
      • Writeable: True
      • Required: False

Create a script task to initialize variables

  1. From the Task section of the Palette drag a ScriptTask to the lane after the asset selection user task.
  2. In the General section, enter a Name: Initializing Variables
  3. In the Main config section:
    • Select the groovy script language
    • Add the script to initialize variables:
List selectedAsset= Arrays.asList(dynamic_checkbox.split("\\s*,\\s*"));
counter = 0
execution.setVariable('selectedAsset', selectedAsset)
execution.setVariable('counter', counter)
  1. Connect the asset selection user task to the script task with a SequenceFlow connection

Configure a user task to get attribute values from start user

With the new user task, you ask the startUser to enter new attribute Values and store that into variables..

  1. From the Task section of the Palette drag a UserTask to the lane.
  2. Connect the previous script task to the user task with a SequenceFlow connection.
  3. Select the user task.
  4. In the Properties view, select the Main Config section and enter the Candidate user:
    user(${startUser}).
  5. In the Properties view, select the Form section and add the following elements to the dialog box:
    1. A variable to store asset's definition:
      • Id: varAssetDefinition
      • Name: Enter Asset Definition
      • Type: textarea
      • Readable: True
      • Writeable: True
      • Required: False
    2. A variable to store asset's descriptive example:
      • Id: varAssetDescriptiveExample
      • Name: Enter Asset's Descriptive Example
      • Type: textarea
      • Readable: True
      • Writeable: True
      • Required: False

Configure a script task to add/change attribute values

  1. From the Task section of the Palette drag a ScriptTask to the lane after the get user input user task.
  2. In the General section, enter a Name: Changing/Adding Attributes
  3. In the Main config section:
    • Select the groovy script language
    • Add the script to initialize variables:
import com.collibra.dgc.core.api.dto.instance.attribute.FindAttributesRequest
import com.collibra.dgc.core.api.dto.instance.attribute.ChangeAttributeRequest
import com.collibra.dgc.core.api.dto.instance.attribute.AddAttributeRequest

def attributeValue = [:]
noOfAssets = selectedAsset.size()
counter = execution.getVariable('counter')
assetId = selectedAsset[counter]

fromAttributes = attributeApi.findAttributes(FindAttributesRequest.builder().assetId(string2Uuid(assetId)).build()).getResults()
for(attribute in fromAttributes) {
	attributeValue.put(attribute.getType().getName().toString(),attribute)
}

// Implementing Modification
if (attributeValue['Definition'] == null){
	attributeApi.addAttribute(AddAttributeRequest.builder().assetId(string2Uuid(assetId)).typeId(string2Uuid(uuid_asset_definition)).value(varAssetDefinition.toString()).build())
}
else if (attributeValue['Definition'].getValueAsString() != varAssetDefinition){
	ChangeAttributeRequest changeAttributeRequest = ChangeAttributeRequest.builder().id(attributeValue['Definition'].getId()).value(varAssetDefinition.toString()).build()
	attributeApi.changeAttribute(changeAttributeRequest)
}

if (attributeValue['Descriptive Example'] == null){
	attributeApi.addAttribute(AddAttributeRequest.builder().assetId(string2Uuid(assetId)).typeId(string2Uuid(uuid_asset_descriptive_example)).value(varAssetDescriptiveExample.toString()).build())
}
else if (attributeValue['Descriptive Example'].getValueAsString() != varAssetDescriptiveExample){
	ChangeAttributeRequest changeAttributeRequest = ChangeAttributeRequest.builder().id(attributeValue['Descriptive Example'].getId()).value(varAssetDescriptiveExample.toString()).build()
	attributeApi.changeAttribute(changeAttributeRequest)
}

execution.setVariable('noOfAssets', noOfAssets);
execution.setVariable('counter',counter+1);
execution.setVariable('varAssetDefinition','');
execution.setVariable('varAssetDescriptiveExample','');
  1. Connect the Get User Input user task to the script task with a SequenceFlow connection

Route the workflow based on whether no of assets modified is equal to the no of assets selected or not

  1. From the Gateway section of the Palette, drag an ExclusiveGateway to the lane after the add/change attribute values script task
  2. Connect the script task to the exclusive gateway with a SequenceFlow connection
  3. From the End event section of the Palette, drag an EndEvent to the lane after the exclusive gateway
  4. Connect the exclusive gateway to the EndEvent with a SequenceFlow connection
  5. Select the sequence flow
  6. In the Properties view, select the General section and enter a Name: Satisfied
  7. In the Properties view, select the Main config tab and enter the Condition${counter>=noOfAssets}
  8. Connect the exclusive gateway to the Get User Input user task with a SequenceFlow connection.
  9. Select the sequence flow
  10. In the Properties view, select the General section and enter a Name: Not Satisfied
  11. In the Properties view, select the Main config tab and enter the Condition: ${counter<noOfAssets}

Deploy your workflow

  1. Sign in to Collibra DGC as a user with the Sysadmin global role or a global role that has at least the Workflow Administration global permission
  2. In the main menu, click Settings
  3. In the tab pane, click Workflows → Definitions
  4. Click Upload a file and locate the BPMN file you have just created inside the Eclipse workspace folder