COBOL : how to invoke internal/external sorts

Suppose you are required to come up with the following application:
1. Pre-processing: Delete all input records which have zero in a particular field (call it the OMIT field). The OMIT field is columns 30-34 of the records. 

2. Sorting: Sort the remaining records in descending order by a particular character format field (call it the KEY field). The KEY field is in columns 5-24 of the records. 

3. Post-processing: Write one output record with each key. 

You can do it in three different ways:
A. External SORT with control statements
B. Internal SORT - COBOL program with INPUT/OUTPUT PROCEDUREs
C. Internal SORT - COBOL program with (external) SORT control statements


Method A. External SORT with control statements
   When DFSORT is invoked directly, control statements can be specified using the data set defined by SYSIN.
 
//SYSIN DD *
 OMIT COND=(30,5,CH,EQ,C'ZZZZZ')
 SORT FIELDS=(5,20,CH,D)
 SUM FIELDS=NONE
/*
Operation
a. The system calls DFSORT directly. 
b. DFSORT reads the input data set and deletes each record with a zero OMIT field as requested by the OMIT statement. 
c. DFSORT sorts the remaining records as requested by the SORT control statement. 
d. DFSORT writes one record with each key to the output data set as requested by the SUM statement. 


Method B. Internal SORT - COBOL program with INPUT/OUTPUT PROCEDUREs

This method uses a SORT statement INPUT PROCEDURE for pre-processing and a SORT statement OUTPUT PROCEDURE for post-processing. 

NOFASTSRT is in effect for input and output processing due to the use of INPUT and OUTPUT PROCEDUREs. 

An INPUT PROCEDURE or OUTPUT PROCEDURE can add, delete, alter, edit, or otherwise modify the records. The INPUT PROCEDURE and OUTPUT PROCEDURE are actually special forms of E15 and E35 exits which are called by DFSORT during its processing, but controlled by the COBOL calling program. The INPUT PROCEDURE is responsible for supplying all of the input records to be sorted to DFSORT, while the OUTPUT PROCEDURE is responsible for disposing of all the records after they are sorted. 

Cobol calling program

  *-----------------------------------------------------------------
  *  METHOD 1: COBOL INPUT AND OUTPUT PROCEDURES.
  *-----------------------------------------------------------------
  *  1. PRE-PROCESS:
  *     THE PROGRAM USES A SORT INPUT PROCEDURE TO DELETE RECORDS
  *     WITH A ZZZZZ OMIT FIELD BEFORE SORTING. THE OMIT FIELD IS
  *     IN COLUMNS 30-34.
  *  2. SORT
  *     THE PROGRAM CALLS DFSORT TO SORT THE RECORDS IN DESCENDING
  *     ORDER. THE KEY IS IN COLUMNS 5-24.
  *  3. POST-PROCESS:
  *     THE PROGRAM USES A SORT OUTPUT PROCEDURE TO WRITE ONE
  *     RECORD WITH EACH KEY AFTER SORTING.
  *
  *  INPUT/OUTPUT: READS INDS AND WRITES OUTDS.
  *                DFSORT PASSES RECORDS TO THE PROCEDURES.
  *-----------------------------------------------------------------
   ID DIVISION.
     PROGRAM-ID. CASE2.
   ENVIRONMENT DIVISION.
     INPUT-OUTPUT SECTION.
     FILE-CONTROL.
          SELECT INDS    ASSIGN TO INDS.
          SELECT OUTDS   ASSIGN TO OUTDS.
          SELECT SORT-FILE  ASSIGN TO SORTFILE.
   DATA DIVISION.
     FILE SECTION.
     FD INDS RECORD CONTAINS 160 CHARACTERS
        LABEL RECORD STANDARD BLOCK 27840
        DATA RECORDS ARE INDS-RECORD.
     01 INDS-RECORD.
      05 FILLER        PIC X(4).
      05 INDS-KEY      PIC X(20).
      05 FILLER        PIC X(5).
      05 INDS-OMIT     PIC X(5).
      05 FILLER        PIC X(126).
     FD OUTDS RECORD CONTAINS 160 CHARACTERS
       LABEL RECORD STANDARD BLOCK 27840
       DATA RECORDS ARE OUTDS-RECORD.
    01 OUTDS-RECORD.
     05 FILLER          PIC X(160).
    SD SORT-FILE RECORD CONTAINS 160 CHARACTERS
       DATA RECORD SORT-RECORD.
    01 SORT-RECORD.
     05 FILLER        PIC X(4).
     05 SORT-KEY      PIC X(20).
     05 FILLER        PIC X(136).
    WORKING-STORAGE SECTION.
    01 FLAGS.
     05 INDS-EOF     PIC X VALUE SPACE.
        88 SFLAG     VALUE "Y".
     05 TEMP-EOF     PIC X VALUE SPACE.
        88 TFLAG     VALUE "Y".
    01 PSTART     PIC 9(1) VALUE 0.
    01 SAVE-KEY   PIC X(20).
     01 TEMP-RECORD.
      05 FILLER        PIC X(4).
      05 TEMP-KEY      PIC X(20).
      05 FILLER        PIC X(136).
     PROCEDURE DIVISION.
     MASTER SECTION.
  *-----------------------------------------------------------------
  *     CALL DFSORT TO SORT THE RECORDS IN DESCENDING ORDER.
  *-----------------------------------------------------------------
         SORT SORT-FILE
          ON DESCENDING KEY SORT-KEY
          INPUT PROCEDURE INPUT-PROC
          OUTPUT PROCEDURE OUTPUT-PROC.
         IF SORT-RETURN > 0
            DISPLAY "SORT FAILED".
         STOP RUN.
  *-----------------------------------------------------------------
  *     SORT INPUT PROCEDURE:
  *       READ INDS.
  *       DELETE ALL RECORDS WITH A 'ZZZZZ' OMIT FIELD.
  *       SEND ALL OTHER RECORDS TO DFSORT FOR SORTING.
  *-----------------------------------------------------------------
     INPUT-PROC SECTION.
        OPEN INPUT INDS
        READ INDS AT END SET SFLAG TO TRUE
        END-READ
        PERFORM UNTIL SFLAG
          IF INDS-OMIT NOT = "ZZZZZ"
            RELEASE SORT-RECORD FROM INDS-RECORD
          END-IF
          READ INDS AT END SET SFLAG TO TRUE
          END-READ
        END-PERFORM.
        CLOSE INDS.
  *-----------------------------------------------------------------
  *     SORT OUTPUT PROCEDURE:
  *       RECEIVE RECORDS FROM DFSORT INTO TEMP.
  *       WRITE ONE RECORD WITH EACH KEY TO OUTDS.
  *-----------------------------------------------------------------
      OUTPUT-PROC SECTION.
         OPEN OUTPUT OUTDS
         RETURN SORT-FILE INTO TEMP-RECORD AT END SET TFLAG TO TRUE
         END-RETURN
         PERFORM UNTIL TFLAG
           IF PSTART = 0
   *-----------------------------------------------------------------
   *     FIRST RECORD - SAVE KEY AND WRITE RECORD TO OUTDS.
   *-----------------------------------------------------------------
             MOVE TEMP-KEY TO SAVE-KEY
             WRITE OUTDS-RECORD FROM TEMP-RECORD
             MOVE 1 TO PSTART
           ELSE
             IF TEMP-KEY NOT = SAVE-KEY
  *-----------------------------------------------------------------
  *     RECORD HAS NEW KEY - SAVE KEY AND WRITE RECORD TO OUTDS.
  *-----------------------------------------------------------------
              MOVE TEMP-KEY TO SAVE-KEY
              WRITE OUTDS-RECORD FROM TEMP-RECORD
            END-IF
          END-IF
          RETURN SORT-FILE INTO TEMP-RECORD
            AT END SET TFLAG TO TRUE
          END-RETURN
        END-PERFORM.
        CLOSE OUTDS.
Operation
a. The SORT statement results in a call to DFSORT with a parameter list that contains a SORT control statement and other information. 
b. DFSORT treats the INPUT PROCEDURE as an E15 user exit which must supply all the input records. 
c. DFSORT calls the INPUT PROCEDURE once for each input record. The INPUT PROCEDURE reads the input data set and uses RELEASE to pass each record with a non-zero OMIT field to DFSORT. 
d. DFSORT sorts the records passed to it by the INPUT PROCEDURE as requested by the SORT statement passed to it by the calling program. 
e. DFSORT treats the OUTPUT PROCEDURE as an E35 user exit which must dispose of all the output records. DFSORT calls the OUTPUT PROCEDURE once for each sorted record. The OUTPUT PROCEDURE uses RETURN to obtain the records passed from DFSORT and writes one record with each key to the output data set. 


Method C. Internal SORT - COBOL program with (external) SORT control statements 

In this method FASTSRT is used for input and output processing. 

DFSORT provides a powerful set of collating and editing functions available though the use of control statements and options. Collating and editing functions can be used to replace program code. They are designed to adapt to the run-time characteristics of an application in order to provide significant performance benefits. 

DFSORT's most significant collating and editing functions are: 
SORT or MERGE: enable you to override the SORT or MERGE control statement generated by the compiler. 

INCLUDE or OMIT: enable you to include or delete records whose fields meet certain criteria. 

INREC and OUTREC: enable you to delete and rearrange fields in your records, and to insert blanks, zeros, and constants in records. 

SUM: enables you to sum fields in records with equal keys and to keep only one record with each key. 

OUTFIL: enables you to perform a wide variety of tasks (for example, subsets, editing, reports, and conversion) for multiple output data sets. 

SKIPREC and STOPAFT: enable you to delete records at the beginning or end of your data set. 

When a COBOL calling program is used, INCLUDE, OMIT, INREC, OUTREC, SUM, and OUTFIL can be specified in the data set defined by ddname SORTCNTL. SKIPREC and STOPAFT can be specified on an OPTION statement in the data set defined by ddname SORTCNTL. 

Cobol calling program

  *-----------------------------------------------------------------
  *  METHOD 2: COBOL MAIN PROGRAM.
  *-----------------------------------------------------------------
  *  1. PRE-PROCESS:
  *     A DFSORT OMIT CONTROL STATEMENT DELETES RECORDS WITH A
  *     'ZZZZZ' OMIT FIELD BEFORE SORTING. THE OMIT FIELD IS IN
  *     IN COLUMNS 30-34.
  *  2. SORT
  *     THE PROGRAM CALLS DFSORT TO SORT THE RECORDS IN DESCENDING
  *     ORDER. THE KEY IS IN COLUMNS 5-24.
  *  3. POST-PROCESS:
  *     A DFSORT SUM CONTROL STATEMENT WRITES ONE RECORD WITH
  *     EACH KEY AFTER SORTING.
  *
  *  INPUT/OUTPUT: DFSORT READS SORTIN AND WRITES SORTOUT.
  *-----------------------------------------------------------------
   ID DIVISION.
    PROGRAM-ID. CASE3.
   ENVIRONMENT DIVISION.
    INPUT-OUTPUT SECTION.
      FILE-CONTROL.
         SELECT SORTIN  ASSIGN TO SORTIN.
         SELECT SORTOUT ASSIGN TO SORTOUT.
         SELECT SORT-FILE  ASSIGN TO SORTFILE.
   DATA DIVISION.
     FILE SECTION.
      FD SORTIN RECORD CONTAINS 160 CHARACTERS
         LABEL RECORD STANDARD BLOCK 27840
         DATA RECORDS ARE SORTIN-RECORD.
      01 SORTIN-RECORD.
        05 FILLER          PIC X(160).
      FD SORTOUT RECORD CONTAINS 160 CHARACTERS
         LABEL RECORD STANDARD BLOCK 27840
         DATA RECORDS ARE SORTOUT-RECORD.
      01 SORTOUT-RECORD.
        05 FILLER          PIC X(160).
      SD SORT-FILE RECORD CONTAINS 160 CHARACTERS
         DATA RECORD SORT-RECORD.
       01 SORT-RECORD.
        05 FILLER        PIC X(4).
        05 SORT-KEY      PIC X(20).
        05 FILLER        PIC X(136).
      WORKING-STORAGE SECTION.
    PROCEDURE DIVISION.
      MASTER SECTION.
  *-----------------------------------------------------------------
  *     CALL DFSORT TO SORT THE RECORDS IN DESCENDING ORDER.
  *-----------------------------------------------------------------
        SORT SORT-FILE
          ON DESCENDING KEY SORT-KEY
          USING SORTIN
          GIVING SORTOUT.
        IF SORT-RETURN > 0
          DISPLAY "SORT FAILED".
        STOP RUN.
Control statements

//SORTCNTL DD *
 OMIT COND=(30,5,CH,EQ,C'ZZZZZ')
 SUM FIELDS=NONE
/*
Operation
a. The SORT statement results in a call to DFSORT with a parameter list that contains a SORT control statement and other information. 
b. DFSORT reads the input data set and deletes each record with a zero OMIT field as requested by the OMIT statement. 
c. DFSORT sorts the remaining records as requested by the SORT statement passed from the calling program. 
d. DFSORT writes one record with each key to the output data set as requested by the SUM statement. 

Performance
Since Method C uses USING and GIVING rather than INPUT and OUTPUT PROCEDUREs, FASTSRT is in effect. In addition, by eliminating the overhead related to passing each record between DFSORT and the user exits, and enabling DFSORT to use its highly optimized editing code, 

Method C achieves significant performance gains in CPU time, elapsed time, and EXCPs compared to Method B. The use of DFSORT editing functions rather than COBOL code reduces considerably the effort 
required to perform the application. Source code for pre-processing and post-processing is eliminated along with the time to compile and debug the code. 

In addition, future changes to the editing functions performed by the application can be made by simply changing the control statements. Access to source code or recompiles are not necessary. 

Performance for method C is comparable to that of method A, and is significantly better than that for method B.