Monday, September 26, 2011

SORT - Eliminate all duplicate records

//************** TO ELIMINATE ALL DUPLICATES IN A FILE  ************
//STEP010  EXEC PGM=ICETOOL 
//TOOLMSG  DD SYSOUT=* 
//DFSMSG   DD SYSOUT=* 
//SYSPRINT DD DUMMY                                                  
//IN       DD DSN=XXXXXXX.TSTS.INPUT,DISP=SHR     
//OUT      DD DSN=XXXXXXX.TSTS.INPUT1,DISP=OLD    
//TOOLIN   DD *                                   
 SELECT FROM(IN) TO(OUT) ON(1,10,CH) NODUPS  
/*


The NODUPS option has to be used a little carefully, since it removes ALL the duplicate records, unlike the SUM FIELDS=NONE option in simple SORT, which retains ONE of the duplicate records.

eg) If the input is:

AAAAAAAAAA
BBBBBBBBBB
BBBBBBBBBB
BBBBBBBBBB
CCCCCCCCCC
CCCCCCCCCC
DDDDDDDDDD



The NODUPS option of ICETOOL returns:

AAAAAAAAAA
DDDDDDDDDD


whereas the SUM FIELDS=NONE option on simple SORT will return:

AAAAAAAAAA
BBBBBBBBBB
CCCCCCCCCC
DDDDDDDDDD


So, depending on your requirement, you may use either of them.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.