Join this group if you use the Hadoop version of DMX.

78 Members
Join Us!

Mapping column Names

Hi, I have a huge file with multiple columns occurs within a occurs nested.. there are around close to 900 columns.. I have successfully transferred the file to Hdfs but I find it really difficult to individual map each field to column name in the GUI.. is there a simple way of doing this mapping please let me know.

You need to be a member of Syncsort Community to add comments!

Join Syncsort Community

Email me when people reply –

Replies

  • Syed,

    If you are just ingesting and migrating the data along with the EBCDIC to ASCII/UTF-8 encoding conversion, you can avoid mapping each and every individual field by just mapping the layout level item.  This will become a composite field in DMX and each field will be converted individually according to its data type and data format.  You can do the same for hierarchy qualifies or group level names, i.e. map some fields along with a group name, and all fields in the group will be converted as individual fields. 

    The only time you need to map each field is to create different target layouts or apply functions to fields.  In this case an alternative way to make the mapping slightly easier, would be to create a layout with field names corresponding to the named values and or original field names and map to that layout.  When you bring up the reformat dialog and choose map to layout, the mappings should be auto-populated based on the matching names.  This may not work for array fields though, in which you would have to map the individual elements where you would like them.

    Hope it helps!

    Gary

    • Thanks Gary, I do have arrays and I am trying to map the layout to column names but the problem is I have lot many occurs and the field comes close to 900 , in the GUI it asks me to map the individual field names and name the column names as well. I was wondering if there is a way to tell DMx to pick the layout field names as column names rather than me specifying it also if writing a dtl script would be more efficient for doing this?
    • So you are trying to change the source layout (copybook) to an alternative layout that has a subset of fields and potentially different field names - is that correct?  Are you able to share your copybook and what you want the final layout to look like?

      DTL is a possibility, but would like to make sure I completely understand the requirement.  Thanks!

      Gary

    • Ok, say for example I have the below Copybook
      01 Student-details
      10 name.
      10 subject occurs 100
      15 subject1
      15 subject2
      ..
      ..
      ..
      15 subject 100.

      I also have created a hive table with 101( student name and subjects 1 to 100)column names.. but the column names differ from the names in the copybook.. now I can map the source file layout i.e. copybook to the hive column names one by one under column mapping s by hitting the green arrow for every field in the Cobol layout..as the layout field name and column field names are not exactly the same name they don't get automatically mapped..however it becomes time consuming task if there are 1000 fields that need to be mapped manually.. I don't want to hit that green arrow 1000 times. Is there a way to map files with column s automatically when their names are different.. hope I am clear...
    • Yes - understood.  Not sure I have any silver bullets for this one. 

      As far as I know, this would require you to click through the mappings since there is no good way for DMX to know which field should map to which. 

      You could certainly automate the generation of DTL and then import that for a potentially quicker implementation, but unless you do this often, the time to create a script or program to generate this may offset the time it would take for you to click through it.

      Gary

This reply was deleted.

To access Syncsort Knowledge Base, visit:

My Support