A batch job was executed to consolidate data from the most
recent 13 generations of a GDG, representing approximately 13 months
of historical information.
The process consists of 13 sequential steps, with
each step appending its output records to a single output dataset. The dataset was allocated on a single DASD volume.
Dataset attributes:
- RECFM: FB
- LRECL: 21
- BLKSIZE: 27,993
The number of records appended by each processing step is
shown below. STEP100 created the dataset, and all subsequent steps appended records to the same dataset using DISP=MOD.
|
Step
|
Records Written
|
|
STEP100
|
37,923,095
|
|
STEP110
|
44,967,733
|
|
STEP120
|
57,202,964
|
|
STEP130
|
44,714,138
|
|
STEP140
|
43,599,256
|
|
STEP150
|
43,838,386
|
|
STEP160
|
45,313,346
|
|
STEP170
|
46,349,138
|
|
STEP180
|
45,525,353
|
|
STEP190
|
46,162,534
|
|
STEP200
|
46,518,402
|
|
STEP210
|
45,979,524
|
|
STEP220
|
46,381,066
|
Total records written from STEP100 to STEP220 : 594,474,935
But the Final dataset had only 70,325,745 records.
Dataset Track Limitation
The output dataset was allocated as a basic-format
sequential dataset, which is limited to a maximum of 65,535 tracks per
volume.
But z/OS 3.2 allowed dataset to grow beyond 65,535 tracks per volume. This is a defect.
When the dataset grows beyond this limit:
- The system can no longer
correctly address the entire dataset.
- It effectively treats
the dataset as if it contains only the records corresponding to the
overflowed tracks at the beginning of the file.
- Existing records are
overwritten as additional data is appended.
- Consequently, part of
the previously accumulated data is lost.
First Overflow Event (After STEP130)
The number of tracks is calculated as: (Number of Records × LRECL) / (2 × Block Size)
|
Step
|
Records Written
|
Running Record Count
|
Calculated Tracks
|
|
STEP100
|
37,923,095
|
37,923,095
|
14,225
|
|
STEP110
|
44,967,733
|
82,890,828
|
31,092
|
|
STEP120
|
57,202,964
|
140,093,792
|
52,549
|
|
STEP130
|
44,714,138
|
184,807,930
|
69,321
|
At the completion of STEP130, the dataset reaches 69,321
tracks, exceeding the maximum addressable limit of 65,535 tracks.
Overflow tracks: 69,321 - 65,535 = 3,785 tracks
System now thinks the dataset has 3785 tracks instead of 69,331 tracks.
3785 tracks equates to first 10,090,810 records written by STEP100.
At this point, the system begins reusing existing track addresses, causing subsequent writes to overwrite data already present in the dataset
The process then continues appending additional data:
|
Step
|
Records
Written
|
Running
Record Count
|
Calculated
Tracks
|
|
STEP100
|
10,090,810
|
|
3,785
|
|
STEP140
|
43,599,256
|
53,690,066
|
20,139
|
|
STEP150
|
43,838,386
|
97,528,452
|
36,582
|
|
STEP160
|
45,313,346
|
142,841,798
|
53,579
|
|
STEP170
|
46,349,138
|
189,190,936
|
70,965
|
At the completion of STEP170, the dataset again exceeds the
track limit.
Overflow: 70,965 − 65,535 = 5,430 tracks
5,430 tracks corresponds to approximately 14,475,047 records, consisting of:
- First 10,090,810 records written from STEP100
- First 4,384,237
records written from STEP140
Subsequent write operations reuse track addresses that have already been used, causing newly written records to overwrite previously stored data.
Processing then continues with the next set of steps:
|
Step
|
Records
Written
|
Running
Record Count
|
Calculated
Tracks
|
|
STEP100
|
10,090,810
|
|
3,785
|
|
STEP140
|
4,384,237
|
|
1,645
|
|
STEP180
|
45,525,353
|
60,000,400
|
22,506
|
|
STEP190
|
46,162,534
|
106,162,934
|
39,821
|
|
STEP200
|
46,518,402
|
152,681,336
|
57,270
|
|
STEP210
|
45,979,524
|
198,660,860
|
74,517
|
At the completion of STEP210, the dataset exceeds the
addressable limit for a third time.
Overflow tracks: 74,517 - 65,535 = 8,982 tracks
8,982 tracks corresponds to approximately 23,944,679 records, consisting of:
- First 10,090,810 records written from STEP100
- First 4,384,237 records written from STEP140
- First 9,469,632 records written from STEP180
The final processing step STEP220 then appends an additional
46,381,066 records.
Final Dataset Contents
After all overflow events and the completion of the final
processing step, the dataset contained records originating from only a subset
of the processing steps.
|
Step Contribution Retained
in Final Dataset
|
Record Count
|
|
STEP100
|
10,090,810
|
|
STEP140
|
4,384,237
|
|
STEP180
|
9,469,632
|
|
STEP220
|
46,381,066
|
|
Total Records in Final Dataset
|
70,325,745
|
Conclusion
The multiple overflow events caused entire sections of data
written by several intermediate processing steps to be lost. As a
result, the final dataset did not contain a continuous history of all records
written during the job execution.
The detailed issue outlined above was successfully resolved after implementing the z/OS 3.2 maintenance update (RSU2604/PUT2601).