What happens when one job creates a GDG (+1) while another job tries to use the current generation (0)?

 When a job starts, the initiator places locks on all datasets referenced by the job:

->Exclusive lock is taken if any step references the dataset with DISP=NEW, MOD, or OLD.

->Shared lock is taken if all references use DISP=SHR.

The lock remains in place until the last step that references the dataset completes.

For GDGs, locking is applied at the GDG base level:

->If any step creates a new generation +1 +2, + 3 etc, the GDG base receives an exclusive lock.

->If all referenced generations use DISP=SHR, the GDG base receives a shared lock.

This lock is released only after the last step referencing that GDG completes, ensuring data integrity.

Example

Consider TESTJOBA with 20 steps:

->Step 5 creates GDG(+1).

->Step 10 reads GDG(0).

Because the job creates a new generation, TESTJOBA acquires an exclusive lock on the GDG base before execution begins. The lock remains until Step 10 completes (the last step referencing that GDG).

If TESTJOBB is submitted while TESTJOBA is running and its first step tries to read GDG(0) from the same GDG, it cannot obtain a share lock on the Base GDG. As a result, TESTJOBB waits in dataset contention and starts only after TESTJOBA releases the GDG lock at the end of Step 10.

When a Job Step Dynamically Allocates a Dataset/GDG Generation with DISP=NEW, MOD, or OLD

When a dataset or GDG generation is dynamically allocated, the system first checks whether the job already holds a lock on the dataset or base GDG.

If the job already holds a lock:

->A shared lock indicates that another step later in the same job references the dataset/GDG generation with DISP=SHR.

->The system attempts to promote the shared lock to an exclusive lock.

->If no other job holds a lock on the dataset/base GDG, the promotion succeeds immediately.

->If another job holds a lock, the current job waits until that lock is released, then acquires the exclusive lock.

->The lock is retained until the last job step that references the dataset/GDG generation, at which point it is released.

If the job does not already hold a lock:

->This indicates that no other step in the job references the dataset/GDG generation.

->If no other job holds a lock on the dataset/base GDG, the job acquires an exclusive lock and releases it at the end of the step.

->If another job holds a lock, the current job waits until the lock is released, then acquires the exclusive lock and proceeds.

 

When a Job Step Dynamically Allocates a Dataset/GDG Generation with DISP=SHR

When a dataset or GDG generation is dynamically allocated, the system first checks whether the job already holds a lock on the dataset or base GDG.

If the job already holds a lock:

->If the existing lock is either a shared lock or an exclusive lock, no additional lock processing is required for the current step.

->The existing lock remains in effect until it is released according to normal lock management rules.

If the job does not already hold a lock:

->This indicates that no other step in the job references the dataset/GDG generation.

->If no other job holds an "exclusive lock" on the dataset/base GDG, the job acquires a "share lock" and releases it at the end of the step.

->If another job holds a "exclusive lock", the current job step waits until the lock is released, then acquires the "share lock" and releases it at the end of the step.

 

Understanding "Normal Disposition" and "Abnormal Disposition" in the JCL DISP Parameter

 When learning the JCL DISP parameter, beginners often get confused about the terms Normal Disposition and Abnormal Disposition.
 
Every job step that is executed generates "return code". Famous return codes are 0,4,8,12,16..
 
If the job step does NOT abend, "normal dispostion" comes into effect. If the job step generates return code such as 8/12/16(any return code for that matter) still that is considered "normal disposition"
 
IF the job step abends with Sxxx (such as S0C4, SOC7, SB37, SE37, S322 etc) , Uxxx(user abend), "abnormal disposition" comes into effect.
 
 
This can be illustrated with following example
 
//STEP1   EXEC PGM=IDCAMS                                       
//SYSPRINT DD SYSOUT=*                                          
//DD1      DD DSN=USERID.STEP1.DSN1,                           
//            DISP=(NEW,CATLG,DELETE),                          
//            LRECL=80,RECFM=FB,                                
//            SPACE=(TRK,(1,1),RLSE)                            
//DD2      DD DSN=USERID.STEP1.DSN2,                           
//            DISP=(NEW,CATLG,DELETE),                          
//            LRECL=80,RECFM=FB,                                
//            SPACE=(TRK,(1,1),RLSE)                            
//SYSIN    DD *                                                 
 SET MAXCC=16                                                   
//*                                                             
//STEP2   EXEC PGM=SORT                                         
//SORTIN   DD DUMMY,LRECL=80,RECFM=FB                           
//SORTOUT  DD DUMMY,LRECL=80,RECFM=FB                           
//SYSOUT   DD SYSOUT=*                                          
//DD1      DD DSN=USERID.STEP1.DSN2,                           
//          DISP=(OLD,CATLG,DELETE)                             
//SYSIN    DD *                                                 
 INTENTIONALLY FORCING ABEND BY NOT GIVING PROPER CONTROL CARD  
//*                                                             
 
The below JOB Log messages indicates that even thoug STEP1 genarated return code 16, still USERID.STEP1.DSN1, USERID.STEP1.DSN2 datasets are cataloged.
 
IEF142I USERIDS STEP1 - STEP WAS EXECUTED - COND CODE 0016                  
IEF285I   USERID.USERIDS.JOB04476.D0000103.?         SYSOUT                
IGD104I USERID.STEP1.DSN1                           RETAINED,  DDNAME=DD1   
IGD104I USERID.STEP1.DSN2                           RETAINED,  DDNAME=DD2   
 
The below JOB Log messages indicates that since STEP2 abended with abend code U016, USERID.STEP1.DSN2 was deleted as coded in the DISP parameter.
 
IEF472I USERIDS STEP2 - COMPLETION CODE - SYSTEM=000 USER=0016 REASON=00000000    
IEF285I   USERID.USERIDS.JOB04476.D0000104.?         SYSOUT                      
IGD105I USERID.STEP1.DSN2                           DELETED,   DDNAME=DD1   

What happens when you use DUMMY parameter for a dataset

When the DUMMY parameter is specified for a dataset, no disk or tape resources are allocated to that dataset, and no I/O operations are performed against it.

During batch job testing, I frequently use the DUMMY parameter for output datasets that would otherwise contain millions of records and are not required for validation. By eliminating the creation of these unnecessary output files, the job avoids the associated I/O overhead, resulting in shorter execution times of the job

The following example illustrates the behaviour of DUMMY parameter.

Sample job with the DUMMY parameter specified for the output file

//STEP1 EXEC PGM=SMF30ASM    
//STEPLIB  DD DSN=USERID.LOAD,DISP=SHR        
//SYSPRINT DD SYSOUT=*                                          
//SMF30IN DD DISP=SHR,DSN=USERID.WEEKLY.SMF30 
//OUT     DD DUMMY,                   
//        DISP=(NEW,CATLG,DELETE),                          
//        SPACE=(CYL,(50,50,)),
//        DCB=(LRECL=140,RECFM=FB)  

                               
The "EXCP Statistics" section of the JESYSMSG in the job log does not contain an entry for DDNAME OUT. This indicates that no I/O operations were performed against DDNAME OUT.

EXCP Statistics
===============
DDNAME   CC# Unit EXCP Count
STEPLIB      914F         15
SMF30IN   +2 9087      48001
SMF30IN      9186      48001
SMF30IN   +3 9284      23996
SMF30IN   +1 9080      26856
SMF30IN   +1 9187      48001

The above job with the output file defined without the DUMMY parameter
======================================================================
//STEP1 EXEC PGM=SMF30ASM                                
//STEPLIB  DD DSN=USERID.LOAD,DISP=SHR       
//SYSPRINT DD SYSOUT=*                                          
//SMF30IN DD DISP=SHR,DSN=USERID.WEEKLY.SMF30 
//OUT     DD DSN=USERID.SMF30.ASM.OUT,           
//        DISP=(NEW,CATLG,DELETE),                          
//        SPACE=(CYL,(50,50,))              
//        DCB=(LRECL=140,RECFM=FB)  

                           
The "EXCP Statistics" section of the JESYSMSG in the job log shows the number of I/O operations performed against DDNAME OUT.    

EXCP Statistics
===============
DDNAME   CC# Unit EXCP Count
STEPLIB      914F         15
SMF30IN   +1 9187      48001
SMF30IN   +2 9087      48001
SMF30IN      9186      48001
SMF30IN   +3 9284      23996
SMF30IN   +1 9080      26856
OUT          9187      12898
 

Mainframe Batch Window Optimization: Lessons from a 40,000-Jobs batch cycle

 Several years ago, I worked with a colleague on a Mainframe Batch Window Reduction initiative for a customer. The batch processing window ran from 7:00 PM to 10:30 PM, during which approximately 40,000 batch jobs were executed. The batch cycle contained around 10 critical processing paths.

Interestingly, nearly 99% of the 40,000 jobs completed within one minute, while only a handful of jobs had execution times ranging from 5 to 10 minutes. Most of the jobs were running every 1 minute.

We applied following techniques to reduce the batch window. As a resul we introduced about 45 minutes of slack time in 5 critical paths

->Removed unwanted job dependencies

->Worked with upstream applications to get input files early

->Moved non-critical jobs out of the critical paths where feasible.

->Preponed the time triggered jobs

->Converted sequential DB2 unload steps to parallel jobs

->Replaced IDCAMS steps with SORT where feasible.

->Replaced DSNTIAUL unload steps with BMC UNLOAD where feasible.

->Replaced BMC UNLOAD step by image copy where feasible.

Understanding COND=(0,NE) in JCL

 The COND parameter is often one of the most confusing JCL concepts for beginners because its logic works in a somewhat counterintuitive way.

When you code COND=(0,NE) on a job step, the condition is evaluated against the return codes of all previous steps.

If any previous step returns a code other than 0, the condition (0,NE) evaluates to true, and the current step is bypassed (FLUSHED).

If all previous steps return 0, the condition evaluates to false, and the current step executes normally.

 In simple terms, COND=(0,NE) means "Execute this step only if all preceding steps completed successfully with RC=0."

Example

//STEP1    EXEC PGM=IDCAMS

//SYSPRINT DD SYSOUT=

//SYSIN    DD

  SET MAXCC=4

//*

//STEP2    EXEC PGM=IEFBR14

//*

//STEP3    EXEC PGM=IEFBR14,COND=(0,NE)

//*

Execution Results : 

 STEP3 is Flushed.

STEP1 ended with RC=04.

Since 04 is not equal to 0, the condition COND=(0,NE) becomes true.

As a result, STEP3 is skipped (FLUSHED).

 Even though STEP2 completed with RC=00, JCL evaluates the condition against all preceding steps, not just the immediately prior step. Because STEP1 returned a non-zero code, STEP3 does not execute.

Key takeaway: COND=(0,NE) is commonly used to ensure a step runs only when all earlier steps have completed successfully with a return code of zero.

 

How a Mainframe COBOL program misused MQ queue

I came across a Mainframe COBOL program reading messages one by one from MQ eueue, and does some processing and writes the output data to another MQ queue.  It is a very simple program.

The read was destructive MQGET and MQPUT was done using MQPMO_NO_SYNCPOINT so messages were written to the output MQ queue immediately. 

When I checked with application SMEs Why this program was designed this way instead of using plain sequential files.

The answer: restartability.  If this program abends, you simply restart the program, no clean up is needed.

This kind of approach can significantly increase Mainframe CPU utilization and place additional load on the MQ subsystem.

Uncovering the Hidden Cause: A Db2 -911 Error That Silently Broke Data Consistency

Back in 2009, I was working as a Mainframe Consultant for a leading healthcare insurance company. The client’s system architecture was heavily based on CICS, with around 18 production regions handling healthcare claim adjudication. Each region was responsible for processing claims from specific parts of the United States and ran multiple background tasks simultaneously.

At that time, claim data was primarily stored in VSAM KSDS files, with each CICS region maintaining its own dataset. These records were quite large—around 18 KB each—and contained both the original claim data and the derived or calculated information generated during processing.

Around 2006–2007, the client has introduced Db2 into the claim processing flow. The monolithic VSAM structure was normalized into approximately 12 Db2 tables. These tables were split similarly into two logical groups—one for original claim details and another for derived data. Unlike VSAM, the Db2 tables were shared across all CICS regions, rather than being region-specific.

However, to minimize risk, the organization continued to treat VSAM as the primary data source, with Db2 acting as a shadow repository.

The claim processing flow remained largely unchanged: the system would read the VSAM record at the start, cache it in memory, process it across multiple programs, and finally write the updated data back to VSAM. With Db2 integration, additional SELECT, INSERT, and UPDATE statements were introduced throughout the processing steps to keep Db2 in sync with the cached data.

After deploying these changes to production, application SMEs observed inconsistencies—certain claims had missing or incomplete data in Db2 tables. Interestingly, the issue was not consistent; different tables were affected for different claims. Instead of investigating the root cause, a reconciliation batch job was implemented. This job compared VSAM records against Db2 and corrected discrepancies by inserting or updating missing data.

In 2009, I was assigned to debug a production issue related to claim processing. I set up an Xpediter session and carefully traced the execution of a claim through the system.

During debugging, I encountered a recurring Db2 SQL error: SQLCODE -911, which indicates a rollback due to a deadlock or timeout. I continued the session and noticed that, after processing, certain Db2 tables were missing data for the same claim.

This was a critical observation. The -911 error triggers a rollback, which means any previous INSERT or UPDATE operations in that logical unit of work are undone.

Further analysis revealed that the application had retry logic for handling SQLCODE -911. Whenever this error occurred, the program would retry the same SQL statement up to five times. If any retry succeeded, processing would continue as if nothing had happened.

The problem?

This retry logic was implemented for SELECT statements.

As a result:

A -911 error would roll back prior updates toDb2 tables.

A subsequent successful retry (on a SELECT) allowed processing to continue.

This led to partial or missing data in Db2 tables.

This flaw had gone unnoticed for nearly two years after Db2 integration.

When I presented my findings to the application SMEs, there was initial skepticism, partly because I was relatively new to the team. However, I substantiated the findings using official Db2 documentation explaining SQLCODE -911.

Eventually, the team acknowledged the root cause. The resolution involved significant changes to remove inappropriate retry logic and ensure proper handling of transactional failures.

This experience reinforced an important lesson:

Retry mechanisms must be carefully designed—especially in transactional systems—otherwise, they can silently introduce data inconsistencies.