Join this group to access featured content and participate in discussions related to DMX.

149 Members
Join Us!

Optional Data Sources

I have a need to extract some data from a database and "optionally" some data from a file if it exists.  Since the database is always up and running... no issues with this source.  However, since the file is NOT always present the DMX task fails when not found in the source directory on the server.  Anyone have any ideas how to prevent the task from failing when a file is not found?

You need to be a member of Syncsort Community to add comments!

Join Syncsort Community

Email me when people reply –


  • Brad,

    You could have a custom or extended task that always does a 'touch file' command on your source as a step right before the task runs.  An empty file will be created and the task will read 0 records or if the file already exists, it's last updated metadata will be refreshed.  This may be the easiest way to keep the jobs running. 

    Hope it helps!


    • Thx Gary... that's exactly what I did.  I created a custom task that "streams" the data from the file to standard output and used a pipe as the source input in the next task.  If the file does not exist I simply throw the error to /dev/null


      usage () {
        echo "Usage: $0 -a [App]"
        echo "   use -h option for help"
        echo "   -a (eg. 123, abc, etc.)"

      while getopts "a:h" option
              case $option in
                      a)      _App=$OPTARG;;
                      h)      usage
                              exit 0;;
                      \?)     echo "ERROR: Unknown parameter"
                              exit 1;;
            :) echo "ERROR: Option -$OPTARG requires arguement"
               exit 1;;

      if [[ ${_App} = "123" ]] then
              tail -n +2 ${DMXDataDirectory}/OIW/Test/123.dat 2>/dev/null
      elif [[ ${_App} = "abc" ]] then
              tail -n +2 ${DMXDataDirectory}/OIW/Test/abc.dat 2>/dev/null
              echo "Invalid App: ${_App}"
              exit 1

      exit 0

This reply was deleted.

To access Syncsort Knowledge Base, visit:

My Support