When a Simple Action Turned into a Major TSO Logon Problem

 One fine day, several Mainframe users found themselves unable to log in to their TSO sessions. The issue appeared in different forms:

->For some users, the TSO logon JCL failed with JCL errors during the login process.

->For others, the TSO logon job was being submitted with another user's LOGON PROC, causing the job to fail.

This issue did not affect all Mainframe users; only a subset of users experienced these login problems.

The TSO LOGON PROCLIB contains a dedicated LOGON PROC for each Mainframe user.

Prior to the start of the issue, a member of the Mainframe Infrastructure team compressed the TSO LOGON PROCLIB using CA PDSMAN. This action may have been taken because the PROCLIB was nearing capacity while new user entries were being added.

Following the compression activity, some users began experiencing the TSO logon issues described above.

The question is: How could compressing the TSO LOGON PROCLIB using CA PDSMAN result in these TSO logon failures ?

Further investigation revealed that the installation was using CA PMO within a Sysplex environment consisting of both PROD and DEV LPARs.

One of CA PMO's functions is to cache the directory entries of frequently accessed PDS datasets, enabling faster retrieval of PDS members when they are subsequently  accessed.  

As a result, the directory information for the TSO LOGON PROCLIB was cached in both the DEV and PROD LPARs.

The CA PDSMAN compression job was executed on the DEV LPAR. CA PMO on the DEV system correctly detected the compression activity and refreshed its cache with the updated directory entries for the TSO LOGON PROCLIB.

However, the PROD LPAR was not aware that the PROCLIB had been compressed. Consequently, the CA PMO cache on the PROD LPAR continued to reference the pre-compression directory entries.

Because PDS compression reorganizes members and updates directory information, the stale CA PMO cache on the PROD LPAR no longer matched the actual state of the PROCLIB. This mismatch caused TSO logon processing to retrieve incorrect or invalid PROC directory information, resulting in symptoms such as:

->JCL errors during TSO logon.

->TSO logon jobs being submitted with the wrong user's LOGON PROC.

In summary, the root cause of the issue was a stale CA PMO directory cache on the PROD LPAR following the CA PDSMAN compression of the TSO LOGON PROCLIB in the DEV LPAR. To fix the issue, CA PMO cache in PROD LPAR was refreshed.