Showing posts with label Contention. Show all posts
Showing posts with label Contention. Show all posts

Saturday, July 25, 2026

𝗔 𝗦𝗶𝗺𝗽𝗹𝗲 𝗧𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲 𝘁𝗼 𝗘𝗻𝘀𝘂𝗿𝗲 𝗢𝗻𝗹𝘆 𝗢𝗻𝗲 𝗝𝗼𝗯 𝗥𝘂𝗻𝘀 𝗮𝘁 𝗮 𝗧𝗶𝗺𝗲 𝗶𝗻 𝗮 𝗚𝗿𝗼𝘂𝗽 𝗼𝗳 𝗣𝗮𝗿𝗮𝗹𝗹𝗲𝗹 𝗝𝗼𝗯𝘀

While reviewing our batch processing cycle, I noticed that several jobs contained the following DD statement in their final job step:

//ENQUEUE DD DSN=DUMMY.ENQUEUE.DSN,DISP=OLD

Interestingly, the dataset referenced by this DD statement was completely empty and was not accessed or processed by any program within the job. This raised the question: Why was this dataset included in multiple jobs?

After further analysis, I discovered that this was being used as a simple serialization mechanism. Although these jobs could potentially be scheduled to run in parallel, they were intentionally prevented from doing so to avoid database contention issues.
 
How It Works

The key lies in the DISP=OLD parameter. When a job is selected for execution, z/OS allocates all datasets required by a job step before the step begins execution. Because the dataset DUMMY.ENQUEUE.DSN is requested with DISP=OLD, the system reserves it for exclusive use.

As a result:

  • The first job that acquires the dataset proceeds normally.
  • Any other job that also requests the same dataset with DISP=OLD must wait until the dataset is released.
  • Since the DD statement is present in the last step of each job, the dataset remains allocated until that step completes, effectively ensuring that only one job from the group runs at a time.
This creates a simple enqueue mechanism using dataset allocation.
 
Alternative Approaches
 
Mainframe Job schedulers provide built-in facilities for managing job dependencies, resource constraints, and mutual exclusion requirements. These features are usually more flexible and easier to maintain than relying on dataset allocation techniques.
 
However, the application developers chose to implement serialization using DISP=OLD on a dummy dataset. This may have been due to historical reasons, or perhaps a lack of awareness of the scheduler's resource management capabilities.

Sunday, July 19, 2026

How We Eliminated DB2 table Contention Across 50 Batch Jobs Using ESP Renewable Resources

During our month-end batch processing cycle, we had nearly 50 batch jobs that updated the same row in a DB2 table.

Individually, these jobs completed within seconds. However, when multiple jobs ran concurrently, they frequently encountered table contention because they were attempting to update the same row simultaneously. As a result, some jobs would abend.
 
Every month-end cycle, we typically experienced 3 to 5 job abends due to this contention. Since the month-end processing window was already tight, every abend introduced delays in the month end cycle and increased operational effort for reruns.
 
The Initial Approach: Mutual Exclusion with NOTWITH
 
To prevent these jobs from running at the same time, we initially considered modifying the schedule so that each job was mutually exclusive with every other job.
 
ESP provides the NOTWITH statement for defining mutually exclusive jobs. This seemed like a viable solution, but implementation quickly became cumbersome.
 
For example:
 
JOB JOB1
   RUN WORKDAYS
   REL JOBXXX
   NOTWITH(JOB2, JOB3, JOB4, ... JOB50)
ENDJOB
 
JOB JOB2
    RUN WORKDAYS
    REL JOBYYY
    NOTWITH(JOB1, JOB3, JOB4, ... JOB50)
ENDJOB
 
With nearly 50 jobs involved, each job needed to reference the other 49 jobs in its NOTWITH list. Maintaining such a configuration would have been difficult, error-prone, and far from elegant.
 
A Better Solution: ESP Renewable Resource
 
Instead of managing a large network of mutual exclusions, we leveraged ESP's RESOURCE functionality.
 
The RESOURCE statement allows jobs to request resources before submission. We defined a virtual renewable resource called MY_TABLE with a maximum count of 1.
 
This effectively represented our DB2 table as a shared resource that only one job could access at a time.
 
Each job was updated as follows:
 
JOB JOB1
    RUN WORKDAYS
    REL JOBXXX
    RESOURCE (1,MY_TABLE)
ENDJOB
 
JOB JOB2
    RUN WORKDAYS
    REL JOBYYY
    RESOURCE (1,MY_TABLE)
ENDJOB
 
...
 
JOB JOB50
    RUN WORKDAYS
    REL JOBZZZ
    RESOURCE (1,MY_TABLE)
ENDJOB
 
How It Works
 
When ESP submits a job that requires a renewable resource, it temporarily allocates the requested resource from the available resource pool.
 
Since MY_TABLE was defined with a count of 1, only one job could obtain the resource at any given time.

  • If the resource is available, the job starts immediately.
  • If the resource is already in use, ESP holds the job until the resource becomes available.
  • Once the job completes, regardless of whether it ends successfully or abends, the resource is automatically returned to the pool.
 
In essence, each job borrows the resource for the duration of its execution, ensuring exclusive access to the shared table while preventing contention.
 
The Results
 
This approach delivered several benefits:

  • Eliminated table update contention between batch jobs.
  • Removed the need for complex and difficult-to-maintain NOTWITH definitions.
  • Simplified scheduling logic significantly.
  • Reduced month-end job abends caused by concurrent updates.