Monitoring HCM Data Load jobs in OIC

To start an HCM Data Load job from OIC you need to use the Oracle HCM Cloud Adaptor. This is a pretty sophisticated endpoint that exposes a lot of HCM-related functionality, including:

  • Query, Create, Update or Delete Information
  • Extract Bulk Data using HCM Extracts
  • Subscribe to Updates (via ATOM feed)
  • Send Files to HCM Cloud

Using OIC’s built-in mapping functionality you can create a HDL format data file (I will cover this in a separate blog), which you then upload to Universal Content Management using the last of the above options.

Once the data file has been uploaded, you will then need to create another Activity using the HCM Cloud adaptor. This time you need to select the Query, Create, Update or Delete Information option. This options exposes Business Objects, Services and Business (REST) Resources – we’re interested in the Services option.

The Service we need to use is the HCMDataLoader service which provides a number of handy Operations:

  • importAndLoadData
  • getDataSetStatus
  • getDfltObjAttrHints
  • getEntityList
  • getServiceLastUpdateTime

We are only going to be using the first two Operations, the first to start the HCM Data Load process and the second to monitor it. When you execute the importAndLoadData Operation, the response is not very useful. All the operation does is put the Import and Load into a request queue. To find out how the job actually went, you need to use the getDataSetStatus Operation. Ideally this should be embedded in a While loop that will wait and loop until the Job is complete.

As is good practice, the loop is constrained by a maximum iterations constant, so that it can’t run on to infinity. You can see that after the getHdlProcessStatus is fired there is another scope Activity. The getDataSetStatus operation returns an XML fragment as a single string. I have found that the easiest way to consume this properly is to write it out to a temporary file on the OIC server, and then read it back as “proper” XML:

The writeHdlProcessStatus Activity is a Stage File (Write File) that uses the “Opaque Schema” XSD:

<?xml version = '1.0' encoding = 'UTF-8'?>  
<schema targetNamespace="http://xmlns.oracle.com/pcbpel/adapter/opaque/" xmlns="http://www.w3.org/2001/XMLSchema">
  <element name="opaqueElement" type="base64Binary" />
</schema>

The file is then read back in the readHdlProcessStatus Activity using the following XSD:

<?xml version="1.0" encoding="utf-8"?>
<!-- Created with Liquid Studio 2019 (https://www.liquid-technologies.com) -->
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="DATA_SET_STATUS">
    <xs:complexType>
      <xs:sequence>
        <xs:element minOccurs="0" name="RESULT_COUNT" type="xs:unsignedByte" />
        <xs:element minOccurs="0" name="DATA_SET">
          <xs:complexType>
            <xs:sequence>
              <xs:element minOccurs="0" name="DATA_SET_NAME" type="xs:string" />
              <xs:element minOccurs="0" name="CONTENT_ID" type="xs:string" />
              <xs:element minOccurs="0" name="CREATION_DATE" type="xs:string" />
              <xs:element minOccurs="0" name="STATUS" type="xs:string" />
              <xs:element minOccurs="0" name="PROCESS">
                <xs:complexType>
                  <xs:sequence>
                    <xs:element minOccurs="0" name="PROCESS_ID" type="xs:unsignedShort" />
                    <xs:element minOccurs="0" name="SUBMISSION_DATE" type="xs:string" />
                  </xs:sequence>
                </xs:complexType>
              </xs:element>
              <xs:element minOccurs="0" name="IMPORT">
                <xs:complexType>
                  <xs:sequence>
                    <xs:element minOccurs="0" name="STATUS" type="xs:string" />
                    <xs:element minOccurs="0" name="PERCENTAGE_COMPLETE" type="xs:unsignedByte" />
                    <xs:element minOccurs="0" name="DATA_LINE_COUNTS">
                      <xs:complexType>
                        <xs:sequence>
                          <xs:element minOccurs="0" name="TOTAL" type="xs:unsignedByte" />
                          <xs:element minOccurs="0" name="SUCCESS" type="xs:unsignedByte" />
                          <xs:element minOccurs="0" name="FAILED" type="xs:unsignedByte" />
                          <xs:element minOccurs="0" name="UNPROCESSED" type="xs:unsignedByte" />
                        </xs:sequence>
                      </xs:complexType>
                    </xs:element>
                  </xs:sequence>
                </xs:complexType>
              </xs:element>
              <xs:element minOccurs="0" name="LOAD">
                <xs:complexType>
                  <xs:sequence>
                    <xs:element minOccurs="0" name="STATUS" type="xs:string" />
                    <xs:element minOccurs="0" name="PERCENTAGE_COMPLETE" type="xs:unsignedByte" />
                    <xs:element minOccurs="0" name="OBJECT_COUNT">
                      <xs:complexType>
                        <xs:sequence>
                          <xs:element minOccurs="0" name="TOTAL" type="xs:unsignedByte" />
                          <xs:element minOccurs="0" name="SUCCESS" type="xs:unsignedByte" />
                          <xs:element minOccurs="0" name="FAILED" type="xs:unsignedByte" />
                          <xs:element minOccurs="0" name="UNPROCESSED" type="xs:unsignedByte" />
                        </xs:sequence>
                      </xs:complexType>
                    </xs:element>
                  </xs:sequence>
                </xs:complexType>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

There are three Statuses that we are interested in:

  • /DATA_SET_STATUS/DATA_SET/STATUS
  • /DATA_SET_STATUS/DATA_SET/IMPORT/STATUS
  • /DATA_SET_STATUS/DATA_SET/LOAD/STATUS

The first gives us the status of the overall Job; the second gives us the status of the Import stage; and the final one give us the status of the Load stage. The While loop mentioned earlier can use the first of these statuses to determine whether to continue looping – if the Data Set Status is not COMPLETED or not ERROR, we need to keep looping round.