Recovering from a missing or corrupted Symphony file

About this task

You cannot start an end-to-end server on which a Symphony file is missing, unless the Translator.chk file is also missing. To prevent this problem from occurring, rename the Symphony file (mv command) instead of deleting it (rm command). In this way, if you need to restart the end-to-end server before a new Symphony file is created, you can restore the old Symphony file by renaming it back to its original name.

If you try to start an end-to-end server on which a Symphony file is missing and the Translator.chk file is present, the following messages are stored in the EQQMLOG: 
EQQPT67E A Dump was taken for Problem Determination purpose
EQQPT35E Unable to Open Symphony and/or events files, 
Reason: AWSBIO019E File "/WRKDIR/Symphony could not be opened"
The error was: "EDC5129I No such file or directory"
EQQPT40I Output Translator thread is shutting down
Also, the following message is stored in the JESMSGLG of the end-to-end server: 
DUMPID=001 REQUESTED BY JOB (jobname)
DUMP TITLE=DUMP TAKEN BY MODULE TWSPlanManagerClass.C AT PLACEHOLDER NUMBER 01
If you try to submit a CP batch job (REPLAN, EXTEND, or SYMPHONY RENEW) to create a new Symphony file, the batch job fails and the following messages are issued: 
EQQ3091E FAILED SYNCHRONIZATION WITH THE END-TO-END DISTRIBUTED ENVIRONMENT
EQQ3088E THE SYMPHONY FILE HAS NOT BEEN CREATED
Depending on whether the Symphony file was renamed or deleted, perform the following recovery procedure:
Recovering from a Symphony file that was renamed
  1. Stop the end-to-end server.
  2. Rename the Symphony file back to its original name.
  3. Restart the end-to-end server.
  4. Run the CP batch job, if you want to update the Symphony file.
Recovering from a Symphony file that was deleted
  1. Check for any records that were not processed in the EQQTWSIN and EQQTWSOU files by performing the following steps:
    1. From the IBM Z Workload Scheduler main menu, deactivate the job submission on fault-tolerant agents.
    2. From SEQQSAMP, run the EQQCHKEV utility.
      Look at the resulting output. If you see the messages ALL DATASET EVENTS HAVE BEEN PROCESSED related to EQQTWSIN and EQQTWSOU, all the records were processed: 
      EQQ3161I DDNAME         = EQQTWSIN 
EQQ3161I RECORDS NUMBER = nnnnnnnn
EQQ3161I STATUS IS      = ALL DATA SET EVENTS HAVE BEEN PROCESSED
EQQ3161I DDNAME         = EQQTWSOU                    
EQQ3161I RECORDS NUMBER = nnnnnnnn  
EQQ3161I STATUS IS      = ALL DATASET EVENTS HAVE BEEN PROCESSED
  2. If all the records were processed, perform the following recovery procedure:
    1. Stop the end-to-end server.
    2. Rename or delete the Translator.chk file.
    3. Restart the end-to-end server.
    4. If required, from the IBM Z Workload Scheduler main menu activate the job submission on fault-tolerant agents.
    5. Run a Symphony renew to create the Symphony file.
    Note: By following this procedure, messages about discarding some events might be stored in the controller EQQMLOG. These events, if any, are those left in the USS *.msg files.
  3. If there were some records not processed, perform the following recovery procedure:
    1. Stop the controller and end-to-end server tasks.
    2. Delete the WRKDIR by issuing the command rm -r /WRKDIR
    3. Run the EQQPCS05 file to create a new WRKDIR.
    4. Delete or rename and reallocate the EQQTWSIN and EQQTWSOU files, by using the JCL taken from sample job EQQPCS06.
    5. Delete and re-create the EQQSCPDS file, by using the sample taken from job EQQPCS01.
    6. Start the controller and end-to-end server started tasks (EQQTWSIN and EQQTWSOU files will be formatted by the controller started task).
    7. If required, from the IBM Z Workload Scheduler main menu, activate the job submission on fault-tolerant agents.
    8. Run a CP REPLAN to create a new Symphony file