not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

Site / TIMI OARA Originator Claudia Costache

EVOLIUM SAS

OMC-R Troubleshooting Guide Release B10

System Sub-system Document Category

: : :

ALCATEL 900/BSS OMC TROUBLE SHOOTING GUIDE

PREDISTRIBUTION: O&M B10 DPM VAL OMC : G. Tudor : S. Chalon : IST, L3Support

ABSTRACT This document provides the operational workarounds, which can be applied by the Alcatel teams on OMC B10 from the first version.

Approvals Name App. G. Tudor Sub-System DPM S. Chalon VAL DPM

Name App.

C. Seby Quality Manager

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 1/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

REVIEW
Ed. 01 Proposal 01 Ed. 02 Proposal 01 Ed. 03 Proposal 01 12/10/2006 27/03/2007 20/06/2007 TD/MRC/OMT/2006.709 TD/MRC/OMT/2007.148 TD/MRC/OMT/2007.319

HISTORY
Ed. 01 Proposal 01 Ed. 01 Released Ed.02 Proposal 01 12/10/2006 24/11/2006 27/03/2007 First proposal Released Issued for review. This new version takes into account the following FRs: 3BKA32FBR205998: When restarting all the processes from DSM, the omcalarms group fails to start after a power cut off 3BKA32FBR184212 (3BKA32FBR159927): RNUSM Tree Browser displays wrong number of cells in Configuration view during Cold Start RNIM 3BKA32FBR209049 : Background color changes when Mozilla Window is opened 3BKA32FBR203031: During osserver crash, one bssim database has been corrupted. How to remove and recreate the RNIM database: was updated with step 0) B). Use the described procedure in case of export SC action failure. How to remove BSSIM database without losing data: was updated with MX BSS case, 6.) and 9.) How to remove and recreate manually BSSIM database: was updated with this

procedure is applied for MX and G2 BSC.

It is not possible to do an export from SEC or the SEC import does not work because of LDAP database corruption

Ed. 02 Released

16/04/2007

Released

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 2/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

Ed.03 Proposal 01

20/06/2007

Issued for review. This new version takes into account the following FRs: Ch. 11 MPM/NPOe was added Problem OBSAL - After CD3 installation Loss of GPRS counters was removed

Ed. 03 Released

02/07/2007

Released

TABLE OF CONTENT

1. 2. 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 3. 3.1 3.2 3.3 3.4

SCOPE ...................................................................................................................................................... 7 BASIC PROCEDURES.............................................................................................................................. 8 What to do in case of message "NML not available or RNIM not responding"..................................... 8 What to do in case of BAD PERFORMANCE OF OMCR V490 WITH SE 3510 FC ARRAY............... 8 How to remove BSSIM database without losing data ......................................................................... 10 How to remove and recreate manually BSSIM database ................................................................... 14 How to stop a systematically crash of RNIM or crash of RNIMPRC................................................... 15 How to remove and recreate the RNIM database .............................................................................. 16 Problem occurred on a BSC iN reserved Mutual Exclusion ............................................................... 20 How to proceed when a BSC stays abnormally busy ......................................................................... 21 Impossible to launch RNUSM: "the user is unknown to the security manager".................................. 22 Cases of blocked BSSIM/RNIM initialization process........................................................................ 23 How to manage the hanging alarms ................................................................................................... 23 the omcalarms group fails to start after a power cut off ..................................................................... 24 OMC3 ADMINISTRATION ...................................................................................................................... 25 How to UnLOCK A LOCKED AXADMIN USER.................................................................................. 25 Cases of blocked OMC initialization process..................................................................................... 26 The IconBox starts with NO FAD........................................................................................................ 27 How to reduce the size of txn.log........................................................................................................ 29

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 3/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.5 3.6 3.7 3.8 3.9

ObjectStore doesn't start after a reboot or an OMC start ................................................................... 30 Objecstore is freezed.......................................................................................................................... 31 Impossible to open SEC for a user different of axadmin .................................................................... 32 DSMUSM is empty because DSMIM is not running ........................................................................... 33 DEC ALPHA `Num Lock` selected: Only the mouse is working. ........................................................ 34 ERROR: Print file failed! When user note field not correctly formatted .............................................. 36 Solaris 10: problem of colors ............................................................................................................. 37 "The DT messaging system could not be starting, axadmin iconbox cannot be launched.", ............. 38 It's not possible to give a name to a PRC when Verr Num (Num lock) key is activated..................... 39 Legato backup cannot be re-started in case of failure of "save_pre_legato" script............................ 40 Launch of CRAFT terminal from OMC3 causes several Mozzila error messages ............................. 41 Ports blocked due to repetitive incorrectly terminated actions............................................................ 42 RNUSM freeze on HMI during normal usage ..................................................................................... 42 No error message if the NavServer is down ....................................................................................... 43 STOP/START SEC service ................................................................................................................ 43 False message appears when OMC is started from OSM. ................................................................ 44 Remained actions pending in the BSSUSM follow-up ........................................................................ 45 No more free virtual memory (98% used) on HMI due to lss process ................................................ 45 Full partition due to generation of lss core file under "/"...................................................................... 46 ObjectStore cache manager could not be started .............................................................................. 46 Not possible to navigate between USMs from Iconbox ...................................................................... 47 Problems with CDROM management on V880 and 280R ................................................................. 47 V490 Panic CPU following find command .......................................................................................... 48 OMC not usable after delete PRC ...................................................................................................... 48 Security error when declaring a BSS/MFS from OAMUSM................................................................ 49 During osserver crash, one bssim database has been corrupted. ..................................................... 50 OMC/BSC COMMUNICATION ............................................................................................................... 51 Communication to all BSS lost-Nothing happens on the field whereas an action was triggered........ 51 OMC3 X25 configuration problem ...................................................................................................... 52 How to recover an X25 link when invalid licence number was applied............................................... 53 RADIO...................................................................................................................................................... 55 The "wrong BCCH/BSIC" error appears during a check PRC ............................................................ 55 A Cell goes misaligned after an audit ................................................................................................. 56 The Cell/BSC goes misaligned after an activation or modification of the SC ..................................... 57 The Alignment Status of a Cell stays inConfig.................................................................................... 59

3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22 3.23 3.24 3.25 3.26 3.27 3.28 3.29 3.30 4. 4.1 4.2 4.3 5. 5.1 5.2 5.3 5.4

5.5 Processing Error - ERROR - bssim-5513 : For the cell (lac xxx, ci yyy),there is a reselection found on BSC but not created on omc, with BCCH frequency bbb................................................................................. 60 5.6 5.7 5.8 5.9 WARNING bssim-5529 "For the cell (lac xxx,ci yyy), there is a reselection...................................... 61 ERROR - rnim-1005 : Stubbed activation mode: PRC activation will not be downloaded to any NE. 63 Wrong license number: crash rnimsc ................................................................................................. 64 Duplication of handovers after Radio upload (RNIM case)................................................................. 65

5.10 ERROR RNIM 238 "Warning: the uploaded cell (LAC/CI) cannot be created due to LAC+CI conflict. (To upload successfully the new cell, the existing cell must bemodified using change LAC-CI)." ......................... 67 Ed.03Released CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 4/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.11 5.12 5.13 6. 6.1 6.2 7.

HOW TO CREATE A BACKUP PRC IF THE BSC TO BE COPIED ARE RESERVED..................... 69 How to correct inconsistencies for more than 5000 adjacencies........................................................ 69 Misaligned sector after X.25 interruption during PRC-Activation........................................................ 70 MLU ......................................................................................................................................................... 72 Several BSS impacted by a MLU session should not be impacted .................................................... 72 Error-messages After activation of MLU............................................................................................. 72 OMC3/OMC3 MIGRATION ..................................................................................................................... 74 How to manually activate a SC during a migration ............................................................................. 74 How to run installMig.pl again if something goes wrong during the first installMig.pl or MF-start....... 75 B10/B10 OMC-R software upgrade: after CD3 install, Master reboot no possible by "init 6" command76 After a BSS B9->B10 Migration, some actions remained pending in the BSSUSM ........................... 76 MFS cannot be declared in DCN ........................................................................................................ 77 RNUSM Tree Browser displays wrong number of cells in Configuration view during Cold Start RNIM77 Problem OBSAL - After CD3 installation ............................................................................................ 78 PERFORMANCE MANAGEMENT.......................................................................................................... 80 Not enough disk space on /db/backup to perform full backup............................................................ 80 SUPERVISION ........................................................................................................................................ 81 All the alarms are automatically purged.............................................................................................. 81 Export BSS from BSSusm doesn't work............................................................................................. 82 Alarm filter not loadable ...................................................................................................................... 83 X-LARGE SPECIFIC ............................................................................................................................. 84 The procedure of X-Large OMC-R backup doesn`t work ................................................................... 84 With a Xlarge, impossible to perform a backup with external and internal tape driver....................... 85 MPM/NPOE ........................................................................................................................................... 86 APPENDIX I: SEEK PROBLEMS OF " SLOWNESS OMC "............................................................... 87 Analyse of the problem: ...................................................................................................................... 87 Analyse of the actions in progress: ..................................................................................................... 87 check if RNIM is not in automatic situation of restarting:.................................................................... 87 Seek of the ' orphans processes'........................................................................................................ 87 Control of the number of sessions / users on the HMI / HOST: ......................................................... 88 Detection of the input of the processes bssim / rnim in flow control:.................................................. 88 Check if the level of the traces ' + ' is activated: ................................................................................. 88 Check if Objecstore is freezed............................................................................................................ 88 APPENDIX II: OMC3 TOOLS................................................................................................................ 89 How to save Cell and BTS names/GPRS configuration ..................................................................... 89

7.1 7.2 7.3 7.4 7.5 7.6 7.7 8. 8.1 9. 9.1 9.2 9.3 10. 10.1 10.2 11. 12. 12.1 12.2 12.3 12.4 12.5 12.6 12.7 12.8 13. 13.1

14. APPENDIX III : EXPLANATION ABOUT NEW SUBLISTS/FILTERS INSTALLATION (RESTRICTED TO ADMINISTRATOR USE OR ALCATEL SUPPORT) ........................................................................................ 91 15. 15.1 15.2 16. APPENDIX IV : HOW TO HAVE A COMPLETE INFORMATION ABOUT AN ALARM ...................... 93 Historical alarms information .............................................................................................................. 93 Current alarms information ................................................................................................................. 94 APPENDIX V : OMC TRACES .............................................................................................................. 95

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 5/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

16.1 16.2 16.3 16.4 16.5 16.6

BSSCOMM traces .............................................................................................................................. 96 How to look after a flow of IMSI traces? ............................................................................................. 98 informations needed to analyse a problem of a cell in GSM misaligned state : ................................. 98 Import/Export traces ........................................................................................................................... 99 <j> = PRC number same as used for rnimpr traceObjectore traces .................................................. 99 How to get ASCII file information about the EML bssim HW audit ..................................................... 99

RELATED DOCUMENTS:

OMC3 B9 Troubleshooting Guide 3BK 29007 JAAA PWZZA MXBSC TROUBLESHOOTING 3BK112061070PWZZA BSC DOCUMENT PLAN RELEASE B10 BK102510008DCZZA MFS TROUBLESHOOTING GUIDE B9 RELEASE 3BK29042JAAAPWZZA OMC3 FORM FOR KEY GENERATOR (KEYGEN) - B10 3BK29301LAAARJZZA

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 6/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

1. SCOPE
The aim of this document is to provide hints and workarounds to operator using the B10 OMC-R; it is issued from the OMC B10 Trouble Shooting Guide This document is restricted for ALCATEL intern usage, notably for ALCATEL personnel providing onsite support at customer premises. The chapters BASIC PROCEDURES and the OMC-R ADMNISTRATION are the stable basic first aids, the other chapters are supposed to be updated each time new problems will occur.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 7/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2. BASIC PROCEDURES

2.1 WHAT TO DO IN CASE OF MESSAGE "NML NOT AVAILABLE OR RNIM NOT RESPONDING"
2.1.1 Reference FR

None

2.1.2 Problem description

DCN, RNUSM or an action in BSSUSM was launched, and there is in the DCN, BSSUM or RNUSM follow up, a warning message indicating NML not available or RNIM not responding

2.1.3 Corrective action

If you have error messages indicating a problem of communication with RNIM or NML when you launch DCN or RNUSM or an action which asks the reservation status of bss and no logical action which could slow down the RNIM process, you can Stop/Start RNIM in DSMUSM.

2.1.4 Problem solved

Not Applicable

2.2 WHAT TO DO IN CASE OF BAD PERFORMANCE OF OMCR V490 WITH SE 3510 FC ARRAY
2.2.1 Reference FR
None

2.2.2 Problem description

The issue The problem is that the diskarray works more slower without write-cache;(cache in write-through mode); normally, the cache is activated (write-back mode). In some circumstances, the cache could be auto disabled, and there are some triggers which can be activated to do that - meaning if one trigger is enabled, it can disable the cache in some conditions (overheat, one power source fail, connected ups fails or send some messages; battery board unit is charging or failed an so on).

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 8/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.2.3 Corrective action

If you have error messages indicating a problem of communication with RNIM or NML when you launch DCN or RNUSM or an action which asks the reservation status of bss and no logical action which could slow down the RNIM process, you can Stop/Start RNIM in DSMUSM. The principle is to simulate a move BSC for the corrupted BSS, as following: 1) As root user start the sccli mode:

root@masterHost sccli

2) Run the following commands in sccli mode: sccli> set auto controller disabled sccli> set auto battery disabled sccli> set auto ups-ac disabled sccli> set auto power disabled sccli> set auto fan-failure disabled sccli> set cache write-back

sccli> set ld ld0 write write-back sccli> set ld ld1 write write-back sccli> set ld ld2 write write-back sccli> set ld ld3 write write-back

sccli> abort media ld0 sccli> abort media ld1 sccli> abort media ld2 sccli> abort media ld3

3) The following checks must be done:

sccli> sh media sccli> sh parity sccli> sh ld

root@masterHost iostat -xnczpm 3 root@masterHost rwbypid.d ??? root@masterHost ps -ef

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 9/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.2.4 Problem solved

Not Applicable

2.3 HOW TO REMOVE BSSIM DATABASE WITHOUT LOSING DATA

WARNING: This method should be applied by ALCATEL TD or under ALCATEL TD direct control. ALCATEL-TD preferred

BE CARREFULL: if you do not use the PRC backup, all the pure GPRS radio parameters will be lost (replaced by the default value)!

2.3.1 Reference FR
None

2.3.2 Problem description

The BSC database has been corrupted and the administrator must remove the BSSIM database to recover. The problem is that after removing the BSS, the BTS, cells names and ATER GPRS configuration are lost and must be reentered manually.

2.3.3 Corrective action

NOTICE: In all the future cases where is mentioned to copy Network Elements (BSCs/Cells) in a PRC, and these element cannot be copied because are reserved please see also in this document the points 2.7 How to proceed when a BSC stays abnormally busy (cells aligningUp, aligningDown, misaligned ) to correct the alignment status.

The principle is to simulate a move BSC for the corrupted BSS, as following: 1) Create a PRC (BSCinitial) including all the cells and the MFS for that BSS and export it to idle export directory number x (where x is an integer value). That will serve as a backup of the original configuration 2) Copy the exported files to empty import directory y (where y is an integer value): axadmin@masterHost cd /alcatel/var/share/AFTR/ACIE axadmin@masterHost cp ACIE_NLexport_Dirx/* ACIE_NLimport_Diry

From the IconBox, launch the Command Mode by using the axadmin user on the master

If BSSIM is crashing at each restart it won't be possible to use this script, see Appendix III: OMC3 How to save Cell and BTS names/GPRSconfiguration chapter

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 10/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

COMMAND_MODE>omcdo -start EML 3) Type the following command: COMMAND_MODE>omcdo -cmd 'BSS_moveBSC(bssid,"bssid")'

Where bssid is the id of the BSS whose names should be kept For example, if the BSC is the BSC 27:

COMMAND_MODE>omcdo -cmd 'BSS_moveBSC(27,"27")' 4) Stop the EML command mode, and close the command window COMMAND_MODE> omcdo -stop EML COMMAND_MODE> exit

5) Verify in /alcatel/var/share/AFTR/MISC, that a file bssid.apd.bsc has been created. If not, look in the param.cfg file of Bssim the value of MoveBscExchangePath, where this file should be created. For example:
axadmin@masterHost cd /alcatel/var/share/AFTR/MISC axadmin@masterHost ls 27.apd.bsc This file will be taken into account during next discover. Note: if the error message "errorCode 1507: SS Migration: cannot retrieve MasterFile name. Cannot receive activateIntermediateReport until SW audit completed!" appears don't worry, it is a normal message that means that no SW Replacement session is running. 6) Note the following parameters from the DCN window:

For G2 BSS: <BSCID> (the BSCID bssid should be greater or equal to BSC_LOWER_ID in the /alcatel/omc3/rn/im/conf/param.cfg. If it not the case, then update this file and stop/start rnimsc), <BSCLABEL>, <HOSTNAME>, <RELEASE>, <NODE_ID>, <GENERATION>, <TSAP1>, <SSAP1>, <NSAP1>, <TSAP2>, <SSAP2>, <NSAP2>, <PREFEREDLINK>, <SECONDARY LINK> For MX BSS: <BSCID> (the BSCID bssid should be greater or equal to BSC_LOWER_ID in the /alcatel/omc3/rn/im/conf/param.cfg. If it not the case, then update this file and stop/start rnimsc), <BSCLABEL>, <HOSTNAME>, <RELEASE>, <NODE_ID>, <GENERATION>, <TSAP1>, <SSAP1>, <IP ADDRESS>

7) Stop the BSS measurements 8) Force Remove the BSS (in DCN). If MFSUSM is open, it enters in LIMITED MODE! It is better to closed it, on each workstation. 9) Re-declare BSS: From the IconBox, launch the Command Mode by using the axadmin user: For G2 BSS:

COMMAND_MODE>omcdo -start OAM Ed.03Released CONFIDENTIAL

tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 11/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

COMMAND_MODE>omcdo -cmd 'OMC_declareBSSWithTsapSsapBscId( <BSCID>, "<BSC LABEL>","<HOSTNAME>","<RELEASE>",<NODE ID>,"<TSAP1>","<SSAP1>","<NSAP1>","<TSAP2>","<SSAP2>","<NSAP2>",<PREFERED LINK>,<SECONDARY LINK>,None,"no")'

Example1: for the "bss_13" on the host "jenlain" with 2 links: COMMAND_MODE> omcdo -cmd 'OMC_declareBSSWithTsapSsapBscId(13,"bss_13","jenlain","B10",0,"4254","4253","366963","4254","425 3"," 366950 ",0,1,None,"no")' COMMAND_MODE>omcdo -stop OAM COMMAND_MODE>exit

Example2: for the "bss_13" on the host "jenlain" with 1 link: COMMAND_MODE> omcdo -cmd 'OMC_declareBSSWithTsapSsapBscId(13,"bss_13","jenlain","B10",0,"4254","4253","366963","","","",0,1,N one,"no")' COMMAND_MODE>omcdo -stop OAM COMMAND_MODE>exit

For MX BSS:

COMMAND_MODE>omcdo -start OAM COMMAND_MODE> omcdo -cmd 'OMC_declareBSSWithTsapSsapBscId( <BSCID>, "<BSC LABEL>","<HOSTNAME>","<RELEASE>","<GENERATION",<NODE ID>,"<TSAP1>","<SSAP1>","<IP address>",None,None,None,0,0,None,"no")'

Example: for the "bss_4" on the host "salvie" with IP address 172.25.128.132: COMMAND_MODE> omcdo -cmd 'OMC_declareBSSWithTsapSsapBscId(4,"bss_test","salvie","B10","Evolution",0,"1111","2222","172.25.12 8.132",None,None,None,0,0,None,"no")' COMMAND_MODE>omcdo -stop OAM COMMAND_MODE>exit

10) If needed, restart BSSUSM for that BSC and open the Discover window (Audit menu). Wait until the Status is Reporting . 11) Create a new PRC (BSCfinal) including the MFS and all cells for the BSS that will serve as a reference for the new configuration and export it to idle export directory number x. x may be the same number like the one from step 1. 12) Run the script to merge the BSCinitial and BSCfinal PRCs (mfsId must be replaced with the numerical value for the MFS Id, y with the number from step 2): axadmin@host /alcatel/omc3/ei/script/moveBscInterOmc.pl x y mfsId

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 12/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

Only if the SMSCB is activated, in order to avoid disabling the SMSCB Create the following file call for example f.txt which contains only the 2 following lines without blank spaces but with tab spaces: rnlAlcatelBSC->CBC_PASSWORD rnlAlcatelBSC->CBC_OPERATOR where -> means tabulation Launch the following command from the /alcatel/var/share/AFTR/ACIE/ directory /alcatel/omc3/ei/script/processCsv.pl f.txt /alcatel/var/share/AFTR/ACIE/ACIE_NLimport_Diry

Endif

13) Import the result (Population/Import menu) from directory y into the BSCfinal PRC. 14) Activate the PRC (GPRS is locked on the BSC as a result). 15) perform a Resynchronize GPRS on the BSC, then MFSUSM can be opened again in normal mode. 16) In order to allow GPRS traffic on this BSC, trigger on BSC "Unlock_GPRS" from RNUSM . 17) Start the BSS measurements which has been stopped in 6) 18) Trigger the RNUSM query adjacencies inconsistencies

WARNING: the PRCs previously created that contain cells of the removed BSC and/or incoming or outcoming adjacencies are no longer usable, except applying the script moveBscInterOmc (See point 8 of this paragraph)

WARNING: If the re-declared BSC belonged to a specific OAD, this information is lost and operators using this OAD cannot access to the BSC anymore, so the BSCid has to be add again in the OAD via SECusm

2.3.4 Correction
Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 13/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.4 HOW TO DATABASE

REMOVE

AND

RECREATE

MANUALLY

BSSIM

2.4.1 Reference FR
None

2.4.2 Problem description

The procedure to remove and recreate BSSIM database is used to solve specific problems: - not valide alarm at OMC - Abis TP locked Note that this procedure does not modify the RNIM database part, which is done by using the "how to remove BSSIM database without loosing any data" procedure.

2.4.3 Corrective action Note: This procedure is applied for MX and G2 BSC.
Prerequisite: Cells of the concerning BSC must be aligned or misaligned. No on-going action at equipment level for that BSC, and not at all at radio level.

Method: Launch remove_db.pl script

The script must be launched on the master host. The script receives as input a list of BSC Id (ex : 1, 5, 8), or a range of BSC Ids (1-20).

root@host$ cd /alcatel/omc3/bss/im/10/script root@host$ ./remove_db.pl <bssId>

WARNING: the PRCs previously created that contain cells of the removed BSC and/or incoming or outcoming adjacencies are no longer usable, except applying the script moveBscInterOmc as described in the "How to remove BSSIM database without losing data chapter).

2.4.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 14/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.5 HOW TO STOP A SYSTEMATICALLY CRASH OF RNIM OR CRASH OF RNIMPRC

2.5.1 Reference FR
None

2.5.2 Problem description

Symptom: the RNIM crashes systematically at the restart or there is a crash of RNIMPRC for example during a copy of cells in PRC (example there is some target HO unsolved). The RNIMSC database is probably corrupted.

2.5.3 Corrective action

1) Go to /alcatel/base/rn/im/data/ and type command axadmin@host echo > FIXDATABASE Verify that the file FIXDATABASE is created.

2) Start the RNIM process. If RNIM crashes again, you must delete the database of RNIM see the following section How to remove and recreate the RNIM database. If RNIM is restarted usually do: axadmin@host cat /alcatel/var/maintenance/trace/FIXDATABASE.report In the FIXDATABASE.report you will see the troubles encountered and the way the rnim fixed them (e.g. Duplicate cells with the same LAC/CI found. Corrective actions: to correct the situations we will delete one cell). This is only informative and is not requested an action from you.

2.5.4 Correction
Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 15/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.6 HOW TO REMOVE AND RECREATE THE RNIM DATABASE

ALCATEL-TD preferred

WARNING: This action should be done by ALCATEL TD or under ALCATEL TD direct control.

2.6.1 Reference FR

None.

2.6.2 Problem description

The rnim database can become corrupted and the administrator should then remove the RNIM database. Such action is done as a "last resort". The sections which implies the rnusm window (e.g. create a backup PRC) can be perform only if the rnim process is stable. If for any reasons it is necessary to remove a nameserver database: remove also the rnimsc + EVERY PRC database. WARNING: all the external cells not involved in the adjacencies will be lost; all the internal cells where an upload has not been performed before will be misaligned, and with parameters RACH_BUSY_THRES, THR_CCCH_LOAD, CCCH_LOAD_MEAS_PERIOD empty. The workaround for the second problem is to perform an Upload radio parameters from RNUSM.

2.6.3 Corrective action

The remove action must follow a specific procedure to save the BSS and to allow a resynchronization from rnim to the databases of bssim. Save NAMESERVER and RNIM traces for further analysis. To do so, proceed as follows: Use the message retrieval tool from the Log Viewer (Full Log) and perform a "Logs Merge on demand".

axadmin@host$ cd ~

(so you go to your home directory) (for example)

axadmin@host$ mkdir trace_dir axadmin@host$ cd trace_dir

axadmin@host$ cp /alcatel/var/maintenance/trace/rnim* . axadmin@host$ cp /alcatel/var/maintenance/trace/nameserver* . axadmin@host$ cp /alcatel/var/maintenance/log/* . axadmin@host$ cp R /alcatel/var/maintenance/log/SAVEDLOG/ . axadmin@host$ cp /alcatel/var/maintenance/hist/rnim* . axadmin@host$ cp /alcatel/var/maintenance/hist/nameserver* . Introduce a DAT tape in the streamer of the host in order to backup the logs retrieved above. axadmin@host$ tar -cv *.* First of all, create a backup PRC and copy in it all MFS/BSC/Cells (internal and external) 0) A) Export all the SC in the directory number x, or

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 16/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

B) If the export SC action cannot be performed or you want to recover the RNIM database from a previous export, follow the next procedure: cd /alcatel/var/maintenance/hist choose the latest SC export (e.g. export_0_AutomaticExport_1.tar.old.yymmdd.hhmmss.gz) cd /var/tmp mkdir export_yymmdd cp p export_0_AutomaticExport_1.tar.old.yymmdd.hhmmss.gz /var/tmp/export_yymmdd cd /var/tmp/export_yymmdd gzip cd export_0_AutomaticExport_1.tar.old.yymmdd.hhmmss.gz | tar xvf Choose a empty export directory from /alcatel/var/shareAFTR/ACIE/ACIE_NLexport_Dirx and copy from /var/tmp/export_yymmdd directory all 14 *.csv files to export directory (x) cp *.csv /alcatel/var/share/AFTR/ACIE/ACIE_NLexport_Dirx

Run the script buildAlignStatus (in the directory /alcatel/omc3/migration/script/).

axadmin@host$ ./ buildAlignStatus.pl d x where x is the export directory number 1) Under /alcatel/omc3/rn/im/conf create 3 files called omc3_list_of_bss, omc3_list_of_mfs and the empty file omc3_local_activate. 2) Edit the file omc3_list_of_bss and complete it as follows: One line for each bss. Each line following the model bssId -> bssName -> bssRelease. where -> means tabulation, bssId is the id of the BSS under DCN (id present in the left part of the window different from the Node Id) bss1name is the name given by the operator under DCN, bssRelease is the mib version of the BSC (B8 or B9). The file is built in the increasing order of bssId. As an example: 1 -> bss1name -> B8 2 -> bss2name -> B9

3) Edit the file omc3_list_of_mfs and complete it as follows: one line for each bss. Each line following the model mfsId -> mfsName. where -> means tabulation, mfsid is the id of the mfs under DCN (MFS Identifier, different from the Node Identifier). mfsname is the name given by the operator under DCN. The file is built in the increasing order of mfsId. As an example: 15 -> mfs15name

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 17/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

22 -> mfs22name Check that all the mfsId are in the range [MFS_LOWER_ID, MFS_UPPER_ID] (these 2 values are defined in /alcatel/omc3/rn/im/conf/param.cfg)

4) Under DSM, stop the RNIM process. If you choose to remove also nameserver database, stop also the nameserver process End If

5) Go to /alcatel/base/rn/im/data/ and type command If you choose to remove only the rnim database, then

axadmin@host$ /opt/ODI/OS6.3/ostore/bin/osrm rnim.db1

Verify that the file rnim.db1 has disappeared. Else (you have chosen to remove the rnim database and the nameserver database) axadmin@host$ /opt/ODI/OS6.3/ostore/bin/osrm rnim.db* Verify that all the rnim.db* have disappeared. axadmin@host$ cd /alcatel/base/nameserver/data/ axadmin@host$ /opt/ODI/OS6.3/ostore/bin/osrm nameserver.db

End If 6) Under DSM, If you have chosen to remove also nameserver database, start the nameserver process End If

Start the RNIM process and then launch RNUSM through the ICONBOX. The resynchronization happens then (the launch is slower than usually) and the radio configuration is uploaded. WARNING: the information mfsi is associated with bssj is lost during the resynchronization so: Move the exported files to empty import directory y (where y is an integer value): axadmin@host$ cd /alcatel/var/share/AFTR/ACIE axadmin@host$ cp ACIE_NLexport_Dirx/* ACIE_NLimport_Diry/ Under /alcatel/var/maintenance/trace/ create the file called neIdMapping Edit this file and complete it as follows: BSC -> bssId -> bssId The file is built in the increasing order of bssId. As an example: BSC -> 1 -> 1 BSC -> 100-> 100

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 18/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

where -> means tabulation Export again the SC that will serve as a reference for the new configuration and export it to idle export directory number z. Run the script to merge the BSC/MFS associations, GPRS values... stored in ACIE_NLimport_Diry and the new PRC value: axadmin@host$ /alcatel/omc3/ei/script/mergeRadioInXLconcentration.pl z y /alcatel/var/maintenance/trace/neIdMapping The External Cell will not be created in the PRC If for one BSC the SMSCB is activated (in order to avoid disabling the SMSCB) Create the following third file call for example f.txt which contains only the 2 following lines without blank characters but tab characters: rnlAlcatelBSC->CBC_PASSWORD rnlAlcatelBSC->CBC_OPERATOR where -> means tabulation Launch the following command /alcatel/omc3/ei/script/processCsv.pl f.txt /alcatel/var/share/AFTR/ACIE/ACIE_NLimport_Diry End of case of SMSCB activated

7) Run the script mergeGPRS (in the directory /alcatel/omc3/ei/script/). axadmin@host$ ./ mergeGPRS.pl y where y is the final import directory number 8) Create and open backup PRC Import the result (Population/Import menu) from directory y into the final backup PRC. 9) then activate the PRC and resynchronize the MFS(s).10) remove files omc3_list_of_bss, omc3_list_of_mfs and omc3_local_activate from /alcatel/omc3/rn/im/conf 11.) launch as root the cleanUp script (in directory /alcatel/omc3migration/script) root@host# cd /alcatel/omc3/migration/script/ root@host# ./cleanUp.pl

2.6.4 Problem solved

Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 19/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.7 PROBLEM OCCURRED ON A BSC IN RESERVED MUTUAL EXCLUSION

2.7.1 Reference FR
None

2.7.2 Problem description

Symptom: the discover is refused with an error message (this problem can also happen for others actions). Analysis: The customer wants to do an action on a BSC but the reservation Status of this BSC stays busy even if nothing happens.

2.7.3 Corrective action

The workaround depends of what is displayed in BSSUSM follow up when trying to trigger a discover; following cases have been noticed: 1) "BSC is already reserved by RNL. Please try again later.: In this case, wait 1 minute, then just try to trigger the discover once again. It should be unlocked the second time. If not please report to the "3)" and apply the described workaround. 2) "BSC is already reserved by [message]." where message is like "discover on going, sw activate on going" : In this case, if your SW replacement status is idle, just STOP/START BSSIM. It will be unlocked. 3) "SC is already reserved by RNL. Please try again later.: In such a case there are three possibilities: - remove the BSS, which will cause a loss of Supervision Outage during approximately 15 minutes (to do, please report to "How remove BSSIM database without losing data " chapter). - remove the RNIM database, which will cause a partial loss of the GPRS Configuration and a loss of the Radio Supervision for up to 1 hour depending of the amount of cells in the OMC(to do, please report to "How remove and recreate the RNIM database" chapter). -

2.7.4 Correction
Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 20/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.8 HOW TO PROCEED WHEN A BSC STAYS ABNORMALLY BUSY

2.8.1 Reference FR

None

2.8.2 Problem description

Symptom: In the RNUSM window, one or several BSC stay busy: the reservationStatus of the BSC(s) stays at the state busy. Analysis: Depends of the alignmentStatus of the BSC(s), in the same RNSUM window.

2.8.3 Corrective action

There are three cases, depending of the alignmentStatus of the BSC: 1) If it is aligningDown:

stop/start the rnim process in the DSM window (be sure before stopping the process that no other BSS are working, i.e. the reservationStatus should be Idle). The alignmentStatus should change after the defenceAligneDownTimeout defined in /alcatel/omc3/rn/im/conf/param.cfg (default value = 1hour) If it doesn't succeed, stop/start the impacted bssim process in the DSM window. If it doesn't succeed, please report How to remove BSSIM database without losing data section to remove the BSSIM database.
2) If it is aligningUp:

check the time-out logicalparm_display has expired (default value 3 minutes - See Appendix 1), stop/start the impacted bssim process in the DSM window.

If it doesn't succeed, please report to How to remove BSSIM database without losing data sections to remove the BSSIM database.
3) If it is aligned or misaligned: remove the BSSIM database: to do, please report to How to remove BSSIM database without losing data section.

2.8.4 Correction
Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 21/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.9 IMPOSSIBLE TO LAUNCH RNUSM: "THE USER IS UNKNOWN TO THE SECURITY MANAGER"

2.9.1 Reference FR
None

2.9.2 Problem description

Symptom: the Iconbox seems ok, but RNUSM does not start. The following message is displayed: "the user is unknown to the security manager".

2.9.3 Workaround
1) Edit the $LANG variable ("echo $LANG"). If there is something else than "C" or <blank>, then there is a problem of language configuration. The OMC application automatically replaces all the string by blank. - If the host was just installed, it is because the language parameter was badly set. In this case reinstall with special attention to the language. Reinstall beginning with CD 1. All is explained in the method. - If the host worked before, but does not work anymore the problem comes from the login panel. On this panel there is a menu option/language. C_____Posix must be selected. If not, select it and log on again. 2) If none of the above workarounds are operational, add two variables in all param.cfg of the host (rnusm,bssusm,DCN,rnim bssim(7 and 8), bsscomm ) : CO_USE_UTF8 CO_USE_I18N. This variable is boolean and must be set to False. For instance:

axadmin@host cd /alcatel/omc3/bss/im/9/conf axadmin@host vi param.cfg In the file, this parameter will be added at the end of the COMMON VARIABLES /COMMON VARIABLES o (to add a line and to be in insert mode) CO_USE_UTF8 CO_USE_I18N -> "TRUE" Where -> means tabulation

2.9.4 Correction
Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 22/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.10 CASES OF BLOCKED BSSIM/RNIM INITIALIZATION PROCESS

2.10.1 Reference FR

None

2.10.2 Problem description

The BSSIM/RNIM initialization process stays blocked because there is a problem with Objectstore. Symptom: After a start of the OMC3 application, via DSM, we launch RNUSM (or DCN) but RNUSM (or DCN) can t contact RNIM. In the files /alcatel/var/maintenance/trace/rnim.err and /alcatel/var/maintenance/trace/bssim.err it can be seen the reference to the Objectstore problem.

2.10.3 Corrective action

1) Stop the part application via DSMUSM 2) Stop Objectstore in the part Platform 3) Do the command as axadmin #axadmin@host$ ps ef | grep ODI If you find a process, log as root and kill 9 this process 4) Restart Objectstore via DSMUSM 5) Restart the part Application vi a DSMUSM

2.10.4 Problem solved

Not applicable.

2.11 HOW TO MANAGE THE HANGING ALARMS

2.11.1 Reference FR
None

2.11.2 Problem description

The alarms from OMC-R are not cleared.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 23/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

2.11.3 Corrective action

From BSSUSM window perform Alarm/State Audit and from AS Current Alarms icon choose Option/Synchronization/Alarm Sources

2.11.4 Problem solved

Not applicable.

2.12 THE OMCALARMS GROUP FAILS TO START AFTER A POWER CUT OFF
2.12.1 Reference FR
3BKA32FBR205998

2.12.2 Problem description

When restarting all the processes from DSM, the omcalarms group fails to start after a power cut off

2.12.3 Corrective action

1)remove /alcatel/omc3/mstopo/data/db.properties 2)launch as axadmin the script:

axadmin@host$/alcatel/omc3/mstopo/script/createDB.sh /usr/local/etc/openfusion.env /alcatel/omc3/mstopo/conf/param.cfg /alcatel/omc3/mstopo/conf/create_topo.sql Now the db.properties file was generated and the start of omcalarms can be done.

2.12.4 Problem solved

Not applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 24/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3. OMC3 ADMINISTRATION

3.1 HOW TO UNLOCK A LOCKED AXADMIN USER

3.1.1 Reference FR

None

3.1.2 Problem description

This problem occurs in the 2 following cases: After three attempts to log with a wrong password Aiming of an outdated account (in such a case, the user is not seen "locked").

3.1.3 Corrective action

A user axadmin can be unlocked by the following command: Log in as root on the station. # /alcatel/omc3/bin/ unlockaccount -l axadmin

WARNING: this action should be done only on Master

3.1.4 Problem solved
Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 25/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.2 CASES OF BLOCKED OMC INITIALIZATION PROCESS

3.2.1 Reference FR

None

3.2.2 Problem description

The OMC initialization process stays blocked on the script /etc/rc2.d/S72inetsvc. Symptom: in boot sequence, the master station is not able to resolve its own address from its hostname, thus the boot sequence is blocked.

3.2.3 Corrective action

Check the /etc/ nsswitch.conf file. axadmin@host$vi nsswitch.conf In nsswitch.conf a line "hosts: nis files" must be present, in this line invert order of resolutions, as following: replace the line "hosts: nis files [dns] " by the line "hosts: files nis [dns] . In case the customer has introduced DNS, at the same line, complete the DNS configuration by adding dns in nsswitch.conf - Reboot the OMC (logged as root): # init 6

3.2.4 Problem solved

Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 26/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.3 THE ICONBOX STARTS WITH NO FAD

3.3.1 Reference FR
None.

3.3.2 Problem description

Symptom: When the Iconbox is launched, it contains only a grayed ICON called NO FAD Declared.

3.3.3 Workaround
Try the following workarounds, in that order. 1) If the problem occurs just after an OMC reboot, it can be that the OMC has not finished its restart procedure. Close the Iconbox, wait a few minutes, and retry. 2) There are 2 cases: - If the user whose iconbox is empty is axadmin: launch the script restore-sec.pl (in the directory /alcatel/omc3/sec/script/).

axadmin@host$ cd /alcatel/omc3/sec/script/ axadmin@host$ ./restore-sec.pl This script will give back to axadmin its own profile under SEC (axadmin_special_profile).It will also recreate the user axadmin, if it has been destroyed (by a delete user).

If the user whose iconbox is empty is not axadmin: Check (as axadmin) in SECUSM, that the user has a profile assigned. You can see this under the Operator tab in SECUSM window.Click on the concrete operator, on the right side, you can see the profiles of this operator

WARNING: If only the profile of axadmin has been destroyed (and not the user), some error messages will appear (it tries to re-create the user). It doesn't matter. The specified identifier already exist If there is no way to launch the script as axadmin (no window available) log in as a new user. Then: User1@host$ su axadmin axadmin@host$ cd /alcatel/omc3/sec/script/ axadmin@host$./restore-sec.pl

3) Check osmd is running (under UNIX) axadmin@host ps -aef | grep osmd Check if lss is running (in DSMUSM). (process must be green) Check sec service is running Su

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 27/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

#svcs | grep sec7 You should see: online date-of-start svc:/network/ldap/sec7:default Note: The lss and osmd must be running on the HMI, the slave and the master. The sec server must be running on the master only. 4) Check the name of the user with Unix command "whoami" or "id". (We encountered some case where "axadmin" was seen as "#axadmin).

If this is the case and if you are on a HMI check if the "wrong" user has an entry to the file /etc/passwd. If so delete the concerned line (but on the HMI only) as root.

# vi /etc/passwd

3.3.4 Correction
Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 28/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.4 HOW TO REDUCE THE SIZE OF TXN.LOG

3.4.1 Reference FR

None

3.4.2 Problem description

Symptom: /alcatel/var is saturated (95% of the disk space is full) To see the load of your fileset, type the command df k

3.4.3 Corrective action

1) You stop the part Application in DSMUSM 2) As root, launch the command : /opt/ODI/OS6.3/ostore/bin/ossvrchkpt <host> 3) Stop Objectstore in the part Platform in DSMUSM 4) As root, launch the command : # /opt/ODI/OS6.3/ostore/lib/osserver i p /etc/opt/ODI/OS6.3/server_parameters 5) Start Objectsore via DSMUSM 6) Restart the part Application via DSMUSM

3.4.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 29/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.5 OBJECTSTORE DOESN'T START AFTER A REBOOT OR AN OMC START

3.5.1 Reference FR

None

3.5.2 Problem description

Symptom: After a reboot of the station or after a Stop/Start of the OMC3, Objectstore cannot restart (the process stays red in DSMUSM). Analysis: Those are problems with the cache Manager.

3.5.3 Corrective action

Type the following commands (as axadmin) on the master (is the OMC is XLARGE this command can be triggered, on the agent stations): axadmin@host$ cd /opt/ODI/OS6.3/ostore/bin/ axadmin@host$ ./oscmrf

3.5.4 Problem solved

Not applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 30/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.6 OBJECSTORE IS FREEZED

3.6.1 Reference FR

Not applicable

3.6.2 Problem description

Symptom: Bssim and rnim are blocked, impossible to create new PRC This kind of problem could occur just after or during an OMC backup or just after a crash disk.

3.6.3 Corrective action

You can trigger the following command:

axadmin@host$ ps -ealf | grep store

8 S root 1613 1516 0 41 20 6214a8e0 /opt/ODI/OS6.3/ostore/lib/oscmgr6 daemon 10 8 S root 1277 1 /opt/ODI/OS6.3/ostore/lib/osserver 0 41 20 6214b660

1735 ef30de84 49810 ef30de84

Aug 31 ? Aug 31 ?

1:01 30:33

When there is the 'T' on the second colon instead of S, you have to kill the process osserver with the option CONT

root@host$ kill - CONT <osserver pid>

3.6.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 31/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.7 IMPOSSIBLE TO OPEN SEC FOR A USER DIFFERENT OF AXADMIN

3.7.1 Reference FR

None

3.7.2 Problem description

Symptom: a user which just has been created with the right on SEC is not able, in remote login, to open the Access Rights Definition (SEC) window in the ICONBOX Analysis: after a user creation files .dtprofile and .profile are only created at the first CDE login.

3.7.3 Corrective action

Both files .dtprofile and .profile are to be copied from /alcatel/var/home/axadmin to the new user home directory for using this user on ssh as soon as the normal CDE login has never been performed.

3.7.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 32/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.8 DSMUSM IS EMPTY BECAUSE DSMIM IS NOT RUNNING

3.8.1 Reference FR
None

3.8.2 Problem description

Symptom: DSMUSM stays empty.

Analysis: dsmim is not running on the master.

3.8.3 Corrective action

If dsmim is not running on the master: -look in the trace (dsmim.out) why it does not start. axadmin@host$ cd /alcatel/var/maintenance/trace axadmin@host$ vi dsmim.out Note: dsmim should be automatically started by the inittab. Probably there is a syntax problem in the DSM configuration files. The reason is explained in the trace - check if some definitions are not duplicated in hosts.cfg or imsecurity.cfg. axadmin@host$ cd /alcatel/var/share/dsm/ axadmin@host$ vi hosts.cfg axadmin@host$ vi imsecurity.cfg. - Once the files are ok, log in as root # init q The init q launches the inittab, which launch dsmim.

3.8.4 Problem solved

Not applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 33/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.9 DEC ALPHA `NUM LOCK` SELECTED: ONLY THE MOUSE IS WORKING.
3.9.1 Reference FR

None

3.9.2 Problem description

DEC ALPHA SPECIFIC PROBLEM : When setting the `Num Lock` function on the keyboard (to have access to numbers) the keyboard doesnt work in most of the OMC Window. Only the mouse is working. For example : 1)In SECUSM window : impossible to set the "OAD elementary" name and the "profile" name during the "elementary OAD" and "profile" creation.

2)In DCN window : impossible to set the name of the BSS (or MFS) during the 'BSS creation' (or MFS creation). All the characters typed on the keyboard are not displayed in the textField when setting the name.

3) In BSS view window : impossible to set the name of the ODMC campaign and the start/end date and the start/end time.

3.9.3 Corrective action

unset the `Num Lock`.

3.9.4 Problem solved

Not applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 34/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this document, use and communication of its contents

5.) The ldap user axadmin is made available to the secserver.

root@host# su axadmin c /alcatel/omc3/sec/script/how_to_create_a_sec_operator.pl axadmin axadmin

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 35/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.10 ERROR: PRINT FILE FAILED! WHEN USER NOTE FIELD NOT CORRECTLY FORMATTED
3.10.1 Reference FR
None

3.10.2 Problem description

From the AS current USM, More information window, on the "output" button and print menu, the operator is able to request the print of an alarm in HTML format. The message "Print file failed!" is displayed with the following characters: Characters @ (without character \ before) is present in the user note field (except at the end of this field) Characters \ (without character \ before) is present at the end of the User Note field There is the following specific behavior with the 2 following characters: If the characters $ is present, the next word will be not present in the html file If the characters \ is present, the next character will be not present in the html file.

3.10.3 Corrective action

Remove or move the special characters @ \ $ in user note field, in order to get correct html file: If you need to print these 3 characters do the following actions in the User Note field: Replace @ by \@ Replace $ by \$ Replace \ by \\

3.10.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 36/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.11 SOLARIS 10: PROBLEM OF COLORS

3.11.1 Reference FR
3BKA32FBR209049

3.11.2 Problem description

If you have problems with the colors displayed on your monitor (the icon in DCN is black, the color of the icon inside BSSUSM is lost, and the border of RNUSMSC is grayed...) Background color changes when Mozilla Window is opened

3.11.3 Corrective action

A possible solution is to switch your workstation graphics mode to a color depth deeper than 8 bits (such as 16 bits, 24 bits, or 32 bits, depending on the video memory of your workstation). 1) Execute the Solaris command on the target workstation (HMI, Master,) /usr/sbin/m64config -prconf This command prints a list of the possible card resolutions, possible monitor resolutions, possible depths, current resolution setting, and current depth. 2) Find another resolution setting and depth setting that are possible on your monitor and then using the parameters -res <resolution> to change the monitor resolution and refresh rate -depth <depth> to change the color depth (for example to 16 bit, 24 bit, ...), execute the command ->> /usr/sbin/m64config For example, to change to 1152x900 resolution with 76Hz refresh rate and 24 bit color depth, the command would be: /usr/sbin/m64config -res 1152x900x76 -depth 24 You must restart the X11 server to activate the changes. If you use the CDE dtlogin window, logging out then logging in again activates the changes. and

3.11.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 37/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.12 "THE DT MESSAGING SYSTEM COULD NOT BE STARTING, AXADMIN ICONBOX CANNOT BE LAUNCHED.",
3.12.1 Reference FR

None

3.12.2 Problem description

After an OMC reboot, we try to log in OMC3 with axadmin user. This loggin failed with the following message: "The DT messaging system could not be started To correct the problem: 1 Choose [OK] to return to the login screen 2 Select Failsafe Session from the login screen's option menu and log in 3 Check to see that the hostname is correct in these locations /etc/src.sh /etc/hosts /usr/adm/inetd.sec " .

3.12.3 Corrective action

Log in root and launch the following command: rm /alcatel/var/home/axadmin/.TTauthority

3.12.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 38/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.13 IT'S NOT POSSIBLE TO GIVE A NAME TO A PRC WHEN VERR NUM (NUM LOCK) KEY IS ACTIVATED.
3.13.1 Reference FR

None

3.13.2 Problem description

It's not possible to give a name to a PRC when Verr Num (Num lock) key is activated. The problem its reproducible only on specific environments: NCD X-Terminals, Terminals with french keyboards, etc.

X-Windows defines the concept of modifiers. Modifiers are special keys on a keyboard that can be "combined" with standard keys (examples are Shift, Caps Lock, Num Lock, ALT, ...).

Five different categories of modifiers are defined, named mod1, mod2, ..., mod5. The OMC uses ILOG Views which considers that this categories must be : mod1 = Meta Modifier mod2 = AltG Modifier mod3 = Num Modifier mod4 = Alt Modifier But, on some environment (especially on NCD X-Terminals) this standard definition of categories is not implemented ! In fact, mod3 contains Alt modifiers and mod4 contains Num modifier !

3.13.3 Corrective action

WARNING: After applying the corrective action described here its possible to have another problem: when Verr Num (Num lock) key is activated the windows can not be moved and the right click on the mouse will not work.

So, to fix the PRC name problem, you only have to reconfigure correctly your keyboard using the dedicated X11 utility called xmodmap. Here are the commands necessary to swap mod 3 and mod4 : axadmin@host xmodmap -e "clear mod3" axadmin@host xmodmap -e "clear mod4" axadmin@host xmodmap -e "add mod3 = Num_Lock" axadmin@host xmodmap -e "add mod4 = Meta_L Meta_R"

3.13.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 39/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.14 LEGATO BACKUP CANNOT BE RE-STARTED IN CASE OF FAILURE OF "SAVE_PRE_LEGATO" SCRIPT

3.14.1 Reference FR

None

3.14.2 Problem description

LRG B9 old HW (1 Master and 2 Agents): - Backup is launched from Legato Networker - save_pre_legato script fails on Master during snapshots - Stop Backup on Legato Networker - Re-launch the Backup from Legato Networker ==> Legato Backup cannot be restarted

3.14.3 Corrective action

a) When the Legato Networker Backup cannot be started on XLRG old/new HW : the status of group Control is "Running" but nothing is saved on Media Device. It is recommended to abort the Backup on Legato Networker Server and to launch as root the cleanup script "/usr/bin/clean_legato.sh" on OMC-R Master to retrieve a normal behavior. or b) Master and Agents ==> Kill "save_pre_legato.sh" Launch "save_post_legato.sh" rm /var/tmp/pre_save_log.* rm /var/tmp/post_save_log.*

Master ==> rm /alcatel/var/share/traces/sshlog.* rm /alcatel/var/share/traces/rshlog.* rm -rf /alcatel/var/share/traces/legato rm -rf /alcatel/var/share/traces/<station1>

3.14.4 Problem solved

A notice has been added in Legato method, that if the script hangs to press Ctrl-C and start clean_legato.sh script.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 40/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.15 LAUNCH OF CRAFT TERMINAL FROM OMC3 CAUSES SEVERAL MOZZILA ERROR MESSAGES
3.15.1 Reference FR

None

3.15.2 Problem description

Pressing the "Navigate to IMT" button from the MFSUSM (before Mozzila window appears), the following error pops up: "Mozzila has detected a /alcatel/var/home/axadmin/.mozzila /lock file. ...."

This can be due to the fact another Mozzila window is already open. In this case it is a normal behavior. If no other Mozzila window is already open, delete the /alcatel/var/home/axadmin/. mozzila /lock file.

If above error has been confirmed with the "Ok" button, Mozzila can open two windows with the following messages: 1. The certificate issuer for this server is not recognized by Mozzila . .... Mozzila refuses to connect to this server...."

2."Java reported the following error on startup: java.lang.securityException: system classes were not signed"

Those errors can be due to the http server configuration, but the Craft terminal can normally be used.

3.15.3 Corrective action

Confirm all the error messages.

3.15.4 Problem solved

Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 41/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.16 PORTS BLOCKED TERMINATED ACTIONS

DUE

TO

REPETITIVE

INCORRECTLY

3.16.1 Reference FR
None

3.16.2 Problem description

Symptom: Ports are blocked, impossible to perform other actions This kind of problem could occur due to repetitive incorrectly terminated actions.

3.16.3 Corrective action

The workaround is to kill at least one process blocking the port using the following commands:

axadmin@host$ ps -ef | grep cmd

root@host$ kill -9 PID <PID of the cmdsession process>

3.16.4 Problem solved

Not applicable

3.17 RNUSM FREEZE ON HMI DURING NORMAL USAGE

3.17.1 Reference FR
None

3.17.2 Problem description

Symptom:

During normal usage sporadically the CPU load oscillates between 80-100% and sometimes reaches 100% The epim process was in FlowControl, which is normal taking into account the huge amount of information exchanged between processes just after migration As a consequence all the operations are slow down, RNUSM freeze for few minutes, no action possible meanwhile.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 42/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

After a while, 1-2 min it goes down again to 5-10% and the RNUSM windows can be normally used.

3.17.3 Corrective action

WARNING: before applying the workaround, please be sure that no radio modification is on going If any RNUSM window is opened during applying this workaround a normal warning message is raised: Communication problem with rnimsc. Close all windows immediately After pushing the OK button the RNUSM window will be closed.

The workaround is to stop/start from DSM the epim process.

You will need to reopen RNUSM window

3.17.4 Problem solved

Not applicable

3.18 NO ERROR MESSAGE IF THE NAVSERVER IS DOWN

3.18.1 Reference FR
None

3.18.2 Problem description

When a OMC application is opened from Icon-Box (Muse) and the Navserver is down no error message is displayed to the user.

3.18.3 Corrective action

The workaround is to restart Iconbox and then the application (to restart NavServer process from DSMUSM window)

3.18.4 Problem solved

Not applicable

3.19 STOP/START SEC SERVICE

3.19.1 Reference FR
None

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 43/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.19.2 Problem description

The sec server process is now managed as a service; it will not be visible in DSMUSM anymore.

3.19.3 Corrective action

As root is necessary to launch following scripts: STOP SEC service svcadm disables /network/ldap/sec7 START SEC service svcadm enable s /network/ldap/sec7

3.19.4 Problem solved

Not applicable

3.20 FALSE MESSAGE APPEARS WHEN OMC IS STARTED FROM OSM.

3.20.1 Reference FR
None

3.20.2 Problem description

When OMC is started from OSM the following message is displayed: "Starting OMC3 ... The security daemon could not be started within expected time Starting processes via DSM Please open the DSM window and WAIT until all processes became green!"

3.20.3 Corrective action

None. Because the sec server process is now managed as a service ignore the message: The security daemon could not be started within expected time

3.20.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 44/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.21 REMAINED ACTIONS PENDING IN THE BSSUSM FOLLOW-UP

3.21.1 Reference FR

none

3.21.2 Problem description

After BSS B8->B9 Migration, the BSS Discover is frozen in the BSSUSM follow-upwindow. Several actions remain at <100%. In the BSSUSM views, everything seems normal, meaning that all objects can be seen and opened.

3.21.3 Corrective action

Perform a new BSS HW Audit.

3.21.4 Problem solved

Not applicable

3.22 NO MORE FREE VIRTUAL MEMORY (98% USED) ON HMI DUE TO LSS PROCESS
3.22.1 Reference FR
None

3.22.2 Problem description

The memory on the HMI was running full. We received an email (axadmin mailbox) with the following information: " Scanned Page Limit At Fri Oct 14 09:56:39 CEST 2005, on bnsnsr64: The current used virtual memory (98%) has reached the warning threshold (90%). You are advised to stop and restart your applications on this station. "

3.22.3 Corrective action

stop and start lss process on the HMI host from DSM application.

3.22.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 45/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.23 FULL PARTITION DUE TO GENERATION OF LSS CORE FILE UNDER "/"
3.23.1 Reference FR
None

3.23.2 Problem description

On the HMI, lss was crashing but the core file was not put on the "normal" location (/alcatel/var/maintenance/core) but under /opt/almap/lss/data (so under the "/" partition). The root partition is small in terms of available space (i.e. on the E450 machine - only 160,5 Mb available).

So after the generation of the ls core file (160Mb core file) the "/" partition was full.

3.23.3 Corrective action

Delete the core file under /opt/almap/lss/data Restart the lss process from DSM

3.23.4 Problem solved

Not applicable

3.24 OBJECTSTORE CACHE MANAGER COULD NOT BE STARTED

3.24.1 Reference FR
None

3.24.2 Problem description

After the OMC has been powered off/on, it was not possible to restart the OMC properly. ObjectStore cache manager (oscmgr) could not be started because its default port (51031) was in use.

3.24.3 Corrective action

1- stop the ObjectStore (from DSM); 2- stop the nav process from DSM; 3- launch the command: ps -ef | grep -i nav; 4- kill all the nav processes resulted (nav_1, etc) from the command above (and assure that ps -ef | grep -i nav results nothing); 5- launch Object Store and wait in DSM until all is initialized;

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 46/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

6- launch again nav processes; 7- launch rnimnameserver and then rnim;

3.24.4 Problem solved

Not applicable

3.25 NOT POSSIBLE TO NAVIGATE BETWEEN USMS FROM ICONBOX

3.25.1 Reference FR
None

3.25.2 Problem description

If the iconbox is stoped with kill -9 command or with a crash then the lock file for the navserver isn't deleted and remains activated. The next iconbox launched sees that a navserver port is taken and it takes the next port even if the first port is free. The result is that it will not be possible to start iconbox after 500 crashes or "kill -9" commands. The result is that the navserver is not working and the navigation between USMs is not possible, but the iconbox is working.

3.25.3 Corrective action

Remove the oldest lock files from the directory /alcatel/omc3/iconbox/data/lock. ls -l /alcatel/omc3/iconbox/data/lock/* | wc -l then delete the oldest of them (you can see the date of the files with ls -l) open the iconbox again

3.25.4 Problem solved

Not applicable

3.26 PROBLEMS WITH CDROM MANAGEMENT ON V880 AND 280R

3.26.1 Reference FR
None

3.26.2 Problem description

The CDROM could not be opened. After having managed to open the CDROM, the cdrom partition was not unmounted and continued to refer to the content of the previously present disk.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 47/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

If one cd needs to be ejected, but the cdrom is still used by a certain process, the eject will fail (with the cdrom busy message), any stop of vold ( from rc3.d )will cause the remaining of links to the cd in cause in /cdrom, so another cd would not be mounted.

3.26.3 Corrective action

Check which process is using the cdrom with fuser command ( eventually try to kill the process in the same time with fuser -k /cdrom/cdrom0) then try to eject the cd. If the case of the unused links from /cdrom, those can be removed when vold is stopped and a restart of vold will mount correctly the needed cd.

3.26.4 Problem solved

Not applicable

3.27 V490 PANIC CPU FOLLOWING FIND COMMAND

3.27.1 Reference FR
None

3.27.2 Problem description

Panic CPU on a V490 machine after launching the following command: find / -name flexlm

3.27.3 Corrective action

When the localhost is already mounted on /net (from a mistake), to avoid the loop is to use find command with prune option as bellow: find / -name <localhost> -prune -o -name <filename>

3.27.4 Problem solved

Not applicable

3.28 OMC NOT USABLE AFTER DELETE PRC

3.28.1 Reference FR
None

3.28.2 Problem description

Delete PRC action was hanging during 1h and was causing rnimsc hanging. Ed.03Released CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 48/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.28.3 Corrective action

Login with another users Search the process list; for the rnimprc process ps ef | grep rnimprc<id>

kill this process and login as axadmin user

3.28.4 Problem solved

Not applicable

3.29 SECURITY ERROR WHEN DECLARING A BSS/MFS FROM OAMUSM

3.29.1 Reference FR
None

3.29.2 Problem description

If a user (axadmin) tries to declare a BSS/MFS, a window pops up saying that "Create refused because security is not available".

Checking the traces, there is an error when OAM tries to get the list of existing OADs. The error seems to stem from CORBA and can be seen in sec server traces.

3.29.3 Corrective action

Stop/Start lss process from DSM and close/open OAM window.

3.29.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 49/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

3.30 DURING OSSERVER CRASH, ONE BSSIM DATABASE HAS BEEN CORRUPTED.

3.30.1 Reference FR
3BKA32FBR203031

3.30.2 Problem description

Durring osserver crash, the DB of an MX BSC has been corrupted. The osserver was killed, with kill -9 signal, during a PRC activation of a freq Plan change, when the bssim was working to send datas to BSS.

3.30.3 Corrective action

ForceRemove the BSS from OAM window and redeclare again.

3.30.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 50/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

4. OMC/BSC COMMUNICATION

4.1 COMMUNICATION TO ALL BSS LOST-NOTHING HAPPENS ON THE FIELD WHEREAS AN ACTION WAS TRIGGERED
4.1.1 Reference FR

None

4.1.2 Problem description

An action (e.g. an abort) has been triggered on the OMC but nothing happens on the field. This can be due to BSSIM that stores in the queue the demand because it is still handling always the last download action; it waits the answer of this action.

4.1.3 Corrective action

Waits for the corresponding time-out of the last action triggered (defined in /alcatel/omc3/bss/im/9/conf/ actions.cfg or /alcatel/omc3/bss/im/8conf/actions.cfg, APPENDIX 13) to let previous actions to be sent to the BSC. If the situation is still blocked after such a time, stop and restart BSSCOMM in DSM

This action has an impact for all BSS, because when bsscomm is started, all bssim ask a resynchronization with the BSS (software audit, PM audit, peer entity audit)

4.1.4 Problem solved

Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 51/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

4.2 OMC3 X25 CONFIGURATION PROBLEM

4.2.1 Reference FR

None

4.2.2 Problem description

Symptom: an OMC link is completely out of order: in DCN view (DCN) a primary link or secondary link of the BSC is out of order (small panel in front of the NSAP). For all the bsc, it is always the same link which is concerned (i.e. the primary one or (exclusive) the secondary one). NOTE: to determine the bad OMC link number: if XX is a good BSC address (one without panel):

axadmin@host$ pad 0.XX > status OK/KO (?) axadmin@host$ pad 1.XX > status OK/KO. One of this two pad doesnt' work (i.e. we cannot get the "connected" from bsc). If for this link, the problem is the same on every bsc then the OMC link (1 or 0) can be considered as badly configured.

4.2.3 Corrective action

1) Open the file : /etc/opt/SUNWconn/x25/config/link_config_000x.cfg axadmin@host$ vi /etc/opt/SUNWconn/x25/config/link_config_000x.cfg 2) look at attribute values which are different from link x=0 or link x=1 and correct bad file (depending of which link is ok) 3) stop omc3: by osm/administration : shutdown OMC or Launch under Unix as root # /alcatel/omc3/osm/script/MF-shutdown 4) restart X25 as root under Unix: # /etc/init.d/X25.control stop # /etc/init.d/X25.control start 5) relaunch omc3 (as axadmin under Unix) : axadmin@host$ /alcatel/omc3/osm/script/MF-start

4.2.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 52/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

4.3 HOW TO RECOVER AN X25 LINK WHEN INVALID LICENCE NUMBER WAS APPLIED

4.3.1 Reference FR
None

4.3.2 Problem description

Symptom: an OMC link is completely out of order: in DCN view (DCN) a primary link or secondary link of the BSC is out of order (small panel in front of the NSAP).

4.3.3 Corrective action

Log in as root user and perform the following actions:

Open the file: /install/data/<hostname>_X25_9.2.lic root@host$ vi /install/data/<hostname>_X25_9.2.lic Correct the number of the X25 licence

Open the file: /etc/opt/licenses/licenses_combined root@host$ vi /etc/opt/licenses/licenses_combined Correct the number of the X25 licence

Open the file: /etc/opt/SUNWconn/x25/X25_9.1.1.lic,1 root@host$ vi /etc/opt/SUNWconn/x25/X25_9.1.1.lic,1 Correct the number of the X25 licence

From /etc/opt/SUNWconn/x25/config directory Delete link_config_0000.cfg and link_config_0001.cfg files root@host$ rm link_config_0000.cfg root@host$ rm link_config_0001.cfg

Verify in /install/data/ directory that the .cfg and .tpl. file are correcty set <hostname>_X25_9.2_00.cfg , <hostname>_X25_9.2_01.cfg <hostname>_X25_9.2_00.tpl , <hostname>_X25_9.2_01.tpl

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 53/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

From /install/script launch the axconfigX25 script in the following mode;

root@host$ ./axconfigX25 a x25 /install/data/<hostname>_X25_9.2_00.tpl `cat /install/data/<hostname>_X25_9.2_00.cfg`

root@host$ ./axconfigX25 a x25 /install/data/<hostname>_X25_9.2_01.tpl `cat /install/data/<hostname>_X25_9.2_01.cfg`

Verify in /etc/opt/SUNWconn/x25/config directory that the link_config_0000.cfg and link_config_0001.cfg file are successfully created

Verify that the X25 links are running

Note: <hostname> represent the hostname of the master or agent machine

4.3.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 54/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5. RADIO 5.1 THE "WRONG BCCH/BSIC" ERROR APPEARS DURING A CHECK PRC
5.1.1 Reference FR

None.

5.1.2 Problem description

Symptom: before activation, or before applying a MLU session, the operator does a Check PRC. But the result of this Check gives several errors:"Adjacency: adjacency reveals wrong BCCH/BSIC on cell ... ". These errors prevent the operator from doing the activation. Analysis: The OMC3 contains some further test (in comparison with the OMC2) including the test that all the adjacent cells of a given cell have a different BCCH/BSIC. This problem can happens then after a Frequency Plan if the customer use a tool which does not do such a control (RNO do the control).

5.1.3 Corrective action

The only workaround for such a problem is to edit a PRC, which contain all the cells impacted by these errors and to remove one by one the corresponding adjacencies.

5.1.4 Problem solved

Not applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 55/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.2 A CELL GOES MISALIGNED AFTER AN AUDIT

5.2.1 Reference FR

None.

5.2.2 Problem description

Symptom: after an audit (UploadRadioParameter), a cell becomes misaligned. Analysis: the cause is described in the alignment status cause. To see it, display the "Show Alignment Status Cause" in the popup menu of the cell.

5.2.3 Corrective action Investigation method in order to be know the last actions performed First of all check the Show Alignment Status Cause or the Show MFS Alignment Status Cause of the misaligned cell. Try to find out which action has lead to this misaligned status: a) Use the message retrieval tool from the OSM/User Facilities /Logs management view b) Perform a "Logs Merge on demand" c) Perform a "Logs Viewer" and fill in the Text Search with the key word: "MFSName" or "BSSName"|"CellName" and then "Logical Audit|Force config"
If the GSM alignment cause is " lost of communication with BSC", if the BSC-OMC communication remains lost (see [1]) else as soon as this communication problem is solved, trigger again the same audit If the GSM alignment status cause is job unsuccessful received from the BSC, trigger again the same audit, if the alignment status remains with the same reason contact BSC L3 support.

---> Check the actions after which cells are became misaligned.

If you have many misaligned cells on a same BSC after an upload /discover, you must first do a Logical audit BSS.

The GPRS alignment status becomes always misaligned after audit, you have to perform a resynchronization between the OMC and the MFS

5.2.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 56/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.3 THE CELL/BSC GOES MISALIGNED AFTER AN ACTIVATION OR MODIFICATION OF THE SC

5.3.1 Reference FR
None

5.3.2 Problem description

Symptom: after the activation of a PRC or after an on-line modification (directly on the SC), a cell becomes misaligned. Analysis: the cause is described in the alignment status cause. To see it, display the "Show Alignment Status Cause" in the popup menu of the cell.

5.3.3 Corrective action

If the GSM alignment cause is " lost of communication with BSC", if the BSC-OMC communication remains lost see [1] else as soon as this communication problem is solved, if you have many misaligned cells on a same BSC, trigger the Force Config BSS else trigger Cell force config. If the GSM alignment status cause is job unsuccessful received from the BSC, trigger the BSS Force Config (if you have many misaligned cells on a same BSC, else Cell force config), if the alignment status remains with the same reason contact L3 support.

If the GPRS alignment status becomes misaligned, you have to perform a resynchronization between the OMC and the MFS

Important Information:
If you have a misaligned cell, and if a radio upload or a force config doesnt change nothing, its always possible to delete this cell in a PRC (use the Delete option when you click on the cell and see the deletion in the Provision To Be Deleted Cells) and activate this PRC. You can then recreate it if needed.

We had the fact to forbid the upload action on certain cases of misaligned cells. This function can be activated/deactivated in changing the value of the parameter ENABLE_FORBID_UPLOAD defined the file /alcatel/omc3/bss/im/9/conf/param.cfg or /alcatel/omc3/bss/im/8/conf/param.cfg If the function is activated: ENABLE_FORBID-UPLOAD TRUE

If the function is not activated: ENABLE_FORBID-UPLOAD FALSE.

Warning: to take into account this modification, its necessary to stop/start BSSIM. The list of all error messages sent from the BSC, is the configuration file config.in under the directory /alcatel/omc3/bss/im/9/conf/ or /alcatel/omc3/bss/im/8/conf/
Ed.03Released CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 57/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

The BSC may take long time to respond unusually to the logical parameter modify message sent by the OMC, in this case you have to update the Timer value of the action name logicalparm_modify: from short to medium (if the BSC respond is more than 3 minutes) from short to long (if the BSC respond is more than 5 minutes) from short to veryLong (if the BSC respond is more than 15 minutes) BSSIM needs to stopped and started in order to take into account this modification SEE APPENDIX 1

5.3.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 58/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.4 THE ALIGNMENT STATUS OF A CELL STAYS INCONFIG

5.4.1 Reference FR

None.

5.4.2 Problem description

Symptom: the Alignment Status of a cell stays inConfig. An hourglass stays on the RNUSM SC window at the cell. Analysis: RNIM crashed or was killed during a modification of this cell. This end of the RNIM process caused that RNIMSC didn't receive the final notifications, which allow the cell to become aligned.

5.4.3 Corrective action

1) Edit a cell in the SC window. 2) Change any available parameter (except GPRS parameter) then APPLY (the APPLY & CHECK button isn't available in the SC). The State should become aligned. If Not, Stop/Start the rnimsc process under DSMUSM. 3) Re-Change the changed parameter to the previous value, if necessary. OR only delete one outgoing handover relationship and the alignment status changes first to aligning down and then to aligned. After that you simply re-create the handover relation and the problem is solved.

5.4.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 59/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.5 PROCESSING ERROR - ERROR - BSSIM-5513 : FOR THE CELL (LAC XXX, CI YYY),THERE IS A RESELECTION FOUND ON BSC BUT NOT CREATED ON OMC, WITH BCCH FREQUENCY BBB
5.5.1 Reference FR

None

5.5.2 Problem description

Symptom: this error message is displayed by the BSSUSM at the end of BSS discovery or by the RNUSM at the BSS/cell uploads.

5.5.3 Corrective action

If the External cell with the frequency bbb is not present in RNUSM, create it from the RNUSM view. Create adjacency reselection from the cell (lac xxx, ci yyy) to the external cell (with frequency bbb). Delete this adjacency. If this external cell is not useful, delete it. It is recommended to launch the RNUSM/general/query adjacencies in order to check consistency between source and target cells (ones having the new value when others keep the old one).

For all the displayed cells with BCCH problem detected: You modify the BCCH frequency or BSIC or both of the cell from a PRC in setting a dummy value and activate it, and to re-do the same action in setting the definitive value for your BCCH/BSCIC frequency.

5.5.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 60/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.6 WARNING BSSIM-5529 "FOR THE CELL (LAC XXX,CI YYY), THERE IS A RESELECTION
5.6.1 Reference FR

None

5.6.2 Problem description

Symptom: this warning message is displayed by the BSSUSM at the end of BSS discovery or by the RNUSM at the BSS/cell upload. BSSIM can generates this warning in the 2 following case: 1) The reselection will first -if possible- be attached to a handover: BSSIM has found a cell corresponding with the bcch reselection cell in the Handover's list : a reselection can choose among a set of handovers. Loosing telecom feature means loosing a reselection frequency. In the B8 context it is accepted that when there is two reselections with the same bcch frequency keeping only one does not means a telecom feature has been lost: the MS is still able to listen to this frequency. Moreover a warning is raised in this case in order to prevent the operation that a modelled reselection has been lost: 2) Management of lonely pure reselection: pure reselection with unknown target is able to be attached to a target cell thanks to the bcch criteria (case discover and Change Lac, ci)

5.6.3 Corrective action

It is recommended to launch the RNUSM/general/query adjacencies inconsistencies in order to check consistency between source and target cells (ones having the new value when others keep the old one).

Source Cell Name

Source Cell LAC

Source Cell CI 7601 7602

Target Cell Name remy remy

Target Cell LAC 1234 1234

Target Cell CI 1234 1234

Attribute Name

EML RNL Value value 13 13

cell00760_0 760 7601 cell00760_0 760 7602

BCCH_FR 18 EQ BCCH_FR 18 EQ

Then recommended method is to delete/create in the SC the concerned adjacency or reselection where the inconsistency different from mcc/mnc is detected
REMARK: Be careful the case 2 of pure reselection without corresponding handover is rarely requested by the operators.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 61/101

All rights reserved. Passing on and copying of this document, use and communication of its contents not permitted without written authorization from Evolium.

Not applicable

5.6.4 Problem solved

Ed.03Released CONFIDENTIAL OMC3 B10 Troubleshooting Guide

tsg_b10ed03Released

3BK 29007 LAAA PWZZA 62/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.7 ERROR - RNIM-1005 : STUBBED ACTIVATION MODE: PRC ACTIVATION WILL NOT BE DOWNLOADED TO ANY NE.

5.7.1 Reference FR
Not applicable

5.7.2 Problem description

Symptom: PRC has been activated, the radio parameter modification have been updated in the OMC but not sent to the BSC Analysis: This problem occurs after OMC migration, the clean up has not been launched

5.7.3 Corrective action

Be careful, the error message rnim-1005 means, there is probably discrepancy between the OMC and BSC, it is recommended to resynchronize the BSC(s) and the OMC: If the operator has not kept a backup PRC Launch the BSC audit/ BSS logical audit corresponding to the last PRC modification Else (the operator has kept a backup PRC) This backup PRC has to be activated Endif Launch the 2 following script on the master host: axadmin@host$/alcatel/omc3/migration/script/cleanUp.pl axadmin@host$/alcatel/omc3/migration/script/cleanUpPRC.pl

5.7.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 63/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.8 WRONG LICENSE NUMBER: CRASH RNIMSC

5.8.1 Reference FR

None

5.8.2 Problem description

Rnimsc crashes

5.8.3 Corrective action

Edit the /alcatel/var/maintenance/trace/rnimsc.err If it contains the line wrong license Refers to the document [5] to generate the Key

5.8.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 64/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.9 DUPLICATION OF HANDOVERS AFTER RADIO UPLOAD (RNIM CASE)

5.9.1 Reference FR

None

5.9.2 Problem description

This problem could occurs after patching the DLS or after a DLS backed up and BCCH modification and.lac/ci modification and DLS restore:

Cell1( lacci1 ) Cell2( lacci2 )

If you upload the following BSC configuration

Cell1( lacci1 ) Cell2( lacciOther ) \ \ Cell3( lacci2 ) The OMC will display the following configuration :

Cell1( lacci1 ) --- Cell2( lacciOther )

Cell3( lacci2 )

Note that the Cell1 upload must be performed before the Cell2 and Cell3 upload.

As we can see, the Cell2 changes its lac/ci to the lac/ci lacciOther. But the old lac/ci lacci2 is reused as the target of a new adjacency. At Cell1 upload, the OMC discovers an adjacency toward lacci2, that exists in its configuration with Cell2, so it creates the adjacency toward cell2. Later the lac/ci change of cell2 is discovered but the incoming adjacency is not removed and created again toward the Cell having the right lac/ci. As a consequence the OMC display a wrong adjacency. If this adjacency already exists in the configuration, you obtain a duplicate adjacency. 5.9.3 Corrective action
5.9.3.1 Detection: This case is easily detected using the check adjacency inconsistencies tool that is available from the RNUSM interface.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 65/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this document, use and communication of its contents

5.9.3.2 Workaround:

You just have to perform again the upload of Cell1 and all is correct.

5.9.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 66/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.10 ERROR RNIM 238 "WARNING: THE UPLOADED CELL (LAC/CI) CANNOT BE CREATED DUE TO LAC+CI CONFLICT. (TO UPLOAD SUCCESSFULLY THE NEW CELL, THE EXISTING CELL MUST BEMODIFIED USING CHANGE LAC-CI)."
5.10.1 Reference FR

None

5.10.2 Problem description

There is a wrong rnim behavior in case of swap of two LAC CI during upload:

CellA( lacci1 ) CellB( lacci2 )

If you upload the following configuration

CellA( lacci2 ) CellB( lacci1 )

The OMC will not change its configuration raising the following warning:

loaded cell (LAC = lac , CI = ci ) cannot be created due to LAC+CI conflict. (To upload successfully the new cell, the existing cell must be modified using change LAC-CI)

The warning appears for CellA and CellB.

When loading sector A a lac/ci change is detected and the creation of a cell with lacci2 is triggered. This operation is not allowed raising to the warning because lacci2 is already in use.

5.10.3 Corrective action

One of the two cells must be removed from the OMC configuration. To avoid telecom outage, the following procedure may be used:

Create a PRC and delete one of the cell Apply MLU session Abort MLU session Perform again the upload of the two impacted cells.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 67/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

Here MLU mode is used only to remove the cell from the OMC configuration without sending anything to the BSC. This workaround do not propose to perform a complete MLU session. Only the first step is triggered. The used PRC will contain one cell in the tobeDeleted list and thats all. So apart the word MLU, the proposed workaround is easy and can be rapidly perform.

If MLU is not available by the customer, the activation of the PRC can be performed.

5.10.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 68/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

5.11 HOW TO CREATE A BACKUP PRC IF THE BSC TO BE COPIED ARE RESERVED

5.11.1 Reference FR

None

5.11.2 Problem description

In RNUSM the circuit Alignment Status for the BSCs is AligningUp / aligningDown, the BSC remains in busy state forever with the hourglass on it, is reserved. This problem may occur due to different actions on the concerned BSC. A future copy of the BSS in a backup PRC will not be possible because of the alignment status, with the RNUSM message: The source BSC or BTS <BSS name> is not copied: the BSC is reserved. rnim 122

5.11.3 Corrective action

1) Change the current directory to /alcatel/omc3/rn/im/conf. Edit the file param.cfg modifying the value of the parameter named allowCopyOfReservedBscInPRC from FALSE to TRUE. 2) Stop / start rnim from dsmusm. 3) Open again the SC and the backup PRC, the copy of the reserved BSC must work now.

5.11.4 Problem solved

Not Applicable.

5.12 HOW TO CORRECT INCONSISTENCIES FOR MORE THAN 5000 ADJACENCIES

5.12.1 Reference FR
None

5.12.2 Problem description

The problem could be appears during Check Inconsistencies using RNUSM- > General -> Query Adjacencies Inconsistencies menu

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 69/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

In case that the limit of adjacencies inconsistencies is more than 5000 the query adjacencies inconsistencies action never ends and SC stays blocked

5.12.3 Corrective action

axadmin@host$ ps ef | grep rnusm axadmin@host$ kill 9 <rnusm_pid> Open RNUSM

Perform successively for each BSC Query Adjacencies Inconsistencies (by selecting the BSC, right click and choose the Query Adjacencies Inconsistencies)

5.12.4 Problem solved

Not Applicable.

5.13 MISALIGNED SECTOR AFTER X.25 INTERRUPTION DURING PRC-ACTIVATION

5.13.1 Reference FR
None

5.13.2 Problem description

During the activation of the time triggered PRC the X.25 links were interrupted. The PRC includes a deletion of one cell After the interruption at OMC / RNUSM the cell was deleted, but the sector was displayed with "!" means misaligned. Misaligned cause: "Communication problem with BSC" (synonymously). At the BSSUSM the Cell was not deleted. Cell name still present at the "GSM Functional View". Logical Audit Cells at RNUSM / BTS was not successful. Logical Audit at BSS Level are not possible.Logical audit forbidden for the cell because of Alignment Status

5.13.3 Corrective action

If the GSM alignment cause is " lost of communication with BSC", if the BSC-OMC communication remains lost see [1] else as soon as this communication problem is solved, if you have many misaligned cells on a same BSC, trigger the Force Config BSS else trigger Cell force config.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 70/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

If the GSM alignment status cause is job unsuccessful received from the BSC, trigger the BSS Force Config (if you have many misaligned cells on a same BSC, else Cell force config), if the alignment status remains with the same reason contact L3 support.

If the GPRS alignment status becomes misaligned, you have to perform a resynchronization between the OMC and the MFS

Important Information:
If you have a misaligned cell, and if a radio upload or a force config doesnt change nothing, its always possible to delete this cell in a PRC (use the Delete option when you click on the cell and see the deletion in the Provision To Be Deleted Cells) and activate this PRC. You can then recreate it if needed.

We had the fact to forbid the upload action on certain cases of misaligned cells. This function can be activated/deactivated in changing the value of the parameter ENABLE_FORBID_UPLOAD defined the file /alcatel/omc3/bss/im/9/conf/param.cfg or /alcatel/omc3/bss/im/8/conf/param.cfg If the function is activated: ENABLE_FORBID-UPLOAD TRUE

If the function is not activated: ENABLE_FORBID-UPLOAD FALSE.

Warning: to take into account this modification, its necessary to stop/start BSSIM. The list of all error messages sent from the BSC, is the configuration file config.in under the directory /alcatel/omc3/bss/im/9/conf/ or /alcatel/omc3/bss/im/8/conf/ The BSC may take long time to respond unusually to the logical parameter modify message sent by the OMC, in this case you have to update the Timer value of the action name logicalparm_modify: from short to medium (if the BSC respond is more than 3 minutes) from short to long (if the BSC respond is more than 5 minutes) from short to veryLong (if the BSC respond is more than 15 minutes) BSSIM needs to stopped and started in order to take into account this modification SEE APPENDIX 1

5.13.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 71/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

6. MLU 6.1 SEVERAL BSS IMPACTED BY A MLU SESSION SHOULD NOT BE IMPACTED

6.1.1 Reference FR
None

6.1.2 Problem description

A MLU session is applied on a BSS. Through the adjacencies between cells, some others BSS of the same OMC3 are impacted by the MLU session: in the RNUSM window, the "Current session status of all these BSS becomes "MLUsession". NOTE: this case is applicable for "partial" network modification scenarios. 6.1.3 Workaround

The workaround is to apply the MLU session only on the targeted BSS and to close the session on the others. 1) On the RNUSM SC view, choose the BSCs, which are really impacted by the MLU session and apply this session (by clicking on the different steps of the session). 2) For the other BSSs, close the MLU session by choosing them in the MLU->Close MLU session item. 3) The cells impacted (through their adjacencies) by the MLU session become misaligned. Then do a ForceConfig on these. 4) When the MLU session is finished for targeted BSSs, close the MLU session for these BSS.

6.1.4 Problem solved

Not Applicable.

6.2 ERROR-MESSAGES AFTER ACTIVATION OF MLU

6.2.1 Reference FR
None

6.2.2 Problem description

After activation of MLU the following two error-messages have appeared in the follow-up-window: "Current action is cancelled due to a software activate or reject."

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 72/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

During the activation phase the following alarm (47 = global_bsc, 6 = atglobalbsc audit_completed ) was received by BSSM. Following this alarm BSSIM triggered an alarm audit.

Short time after this audit started the activation phase had finished with success and new audits were to be triggered. But before these audits to be started all the pending audits are stopped/cancelled. The reason for stoping the pending audits is "stopOnActivateReject" and hence the message displayed in the follow-up window.

6.2.3 Workaround

None. Continue the MLU

6.2.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 73/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

7. OMC3/OMC3 MIGRATION 7.1 HOW TO MANUALLY ACTIVATE A SC DURING A MIGRATION

7.1.1 Reference FR
None

7.1.2 Problem description

Not applicable.

7.1.3 Corrective action

Check the PRC SC: a- Click on <RNUSM> and open the PRC SC: Configuration Open PRC

Select SC and click on <OK>. b- Check the SC: Activation Check all

With forcing cell administrative state If errors appear, correct them before continue. In case of BSS link disconnection, the BSC hardware may be not uploaded and then the PRC SC contains unmapped cells . Remove the cells of the unreachable BSC from the PRC SC. When the link is available again, perform a BSS discover. If the BSC was associated to a MFS, this association has to be re-performed. c- Activate the PRC SC: Activation Activate PRC With forcing cell administrative state

Launch the RNUSM window and check the logical configuration: On the right part of the window all cells must appear. Note: we have to activate with forcing cell administrative state, because an unknown state is created locked, which can be useless (for GPRS for example). d- Go back to the "SW Upgrade with database format change" method in the document [6], (Recreate ODMC then cleanUp!!!!).

7.1.4 Problem solved

Not applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 74/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

7.2 HOW TO RUN INSTALLMIG.PL AGAIN IF SOMETHING GOES WRONG DURING THE FIRST INSTALLMIG.PL OR MF-START
7.2.1 Reference FR/CR
None

7.2.2 Problem description

During the B8->B9 and B9->B9 migration the script installMig.pl must be run in order to distribute the data saved from the old OMC version. At the end, the script deletes files from /alcatel/var/migration directory which are useless in the future. The problem appears if something goes wrong during this script because of the wrong OMC internal configurations (e.g the BSS_LOWER_ID is too high, the BSSs may be lost) or other possible reasons. The script .installMig.pl can not be run again after the problem is solved, because important backup datas from /alcatel/var/migration are deleted.

7.2.3 Corrective action

1) First of all correct the internal OMC problem which causes the bad situation after the run of the installMig.pl.(e.g. modify the BSS_LOWER_ID from rnim param.cfg to a new value if the problem was the value of BSS_LOWER_ID pameter), 2) If the MF-start action has been already performed after the installMig.pl, a reboot of the station (init 6) is expected. The reason of this is to have the OMC in the same status as before the first installMig.pl run (osi, secim, started but rnim, bssim and the other application processes stopped) 3) Take the backup of /alcatel/var/migration done by the script installMig.pl, located in /var/tmp/migBackup_< yymmdd>_<hhmm>.tar.gz and unzip it. axadmin@host cd /var/tmp axadmin@host gzip d migBackup_< yymmdd>_<hhmm>.tar.gz 4) As root remove the /alcatel/var/migration directory root@host rm R /alcatel/var/migration 5) Untar the file obtained at point 3 (this will copy in the same time the backuped migration directory from /var/tmp in /alcatel/var/migration) axadmin@host tar xvf migBackup_< yymmdd>_<hhmm>.tar 6) Return to your normal migration procedure at the point with the installMig.pl run and continue from there
Migration Data Distribution

axadmin@host$ cd /alcatel/omc3/migration/script axadmin@host$ ./installMig.pl

7.2.4 Problem solved

Not applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 75/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

7.3 B10/B10 OMC-R SOFTWARE UPGRADE: AFTER CD3 INSTALL, MASTER REBOOT NO POSSIBLE BY "INIT 6" COMMAND

7.3.1 Reference FR
None

7.3.2 Problem description

Problem can appear in some conditions when HMI are not reachable, After install CD3 the reboot of master action is required using "init 6" command The machine is blocked during the reboot with a white screen. The problem is that when init 6 is launched it will first launch MF-shutdown.It will try to stop the processes on all stations (using dsm). If one of them is not reachable by DSM (appears blank in DSM) it will be blocked.

7.3.3 Corrective action

That's why, before starting a SWR it will be better to check that all the stations are reachable by DSM (not blank in DSM)). This check can also be made after the SWR and before "init 6". ps ef |grep i MF-shutdown note the PID of the process which is shown in the previous command ptree p <PID> find the latest children process in the tree note the <PID> of the children process kill 9 <PID> of the children process

check that the return number is decreased ps ef | wc l

7.3.4 Problem solved

Not applicable

7.4 AFTER A BSS B9->B10 MIGRATION, SOME ACTIONS REMAINED PENDING IN THE BSSUSM
7.4.1 Reference FR
None

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 76/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

7.4.2 Problem description

After BSS B9->B10 Migration, the BSS Discover is frozen in the BSSUSM follow-up window. Several actions remain at <100%. In the BSSUSM views, everything seems normal, meaning that all objects can be seen and opened.

7.4.3 Corrective action

Perform a new BSS HW Audit.

7.4.4 Problem solved

Not applicable

7.5 MFS CANNOT BE DECLARED IN DCN

7.5.1 Reference FR
None

7.5.2 Problem description

During OMC migration HW_declare_BSC_MFS.pl from B9 to B10, the redeclaration of MFS failed while launching:

After force remove MFS, the MFS cannot be declared, the MFS supervision cannot be started: the MFSIM process crashed (segmentation violation). The problem was linked to use of DNS combined with LDAP. Hostname of STATION_A (MFS) was resolved different on MFS side and DNS (different IP addresses)

7.5.3 Corrective action

Stop using DNS clients on OMC.during MFS declaration Declare MFS again.from DCN

7.5.4 Problem solved

Not applicable

7.6 RNUSM TREE BROWSER DISPLAYS WRONG NUMBER OF CELLS IN CONFIGURATION VIEW DURING COLD START RNIM

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 77/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

7.6.1 Reference FR

3BKA32FBR159927

7.6.2 Problem description

In the Tree Bowser / Configuration view the number of cells displayed in the bottom bar was different from the number displayed in the main view.

7.6.3 Corrective action

To close and re-open the RNUSM view but ONLY after the Radio upload has finished.

7.6.4 Problem solved

Not applicable

7.7 PROBLEM OBSAL - AFTER CD3 INSTALLATION

7.7.1 Reference FR
None

7.7.2 Problem description

During the B10/B10 OMC-R SWR without back-office machine, an error appears at adding the new version of the OMCGobsal package, after the removel of old version: pkgadd : ERROR : no packages were found in /var/tmp/OMCGobsal

7.7.3 Corrective action

Login as user root. Copy the new package OMCGobsal_xyz.gz on /var/tmp directory.

root@host$cd /var/tmp root@host$ gzip cd OMCGobsal_xyz.gz | cpio idu root@host$ pkgadd d .

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 78/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this document, use and communication of its contents

Answer y to all questions.

Type the number corresponding to OMCGobsal

7.7.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 79/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

8. PERFORMANCE MANAGEMENT

8.1 NOT ENOUGH DISK SPACE ON /DB/BACKUP TO PERFORM FULL BACKUP.

8.1.1 Reference FR

None

8.1.2 Problem description

The partition /db/backup reaches 100%. While trying to perform the full backup on B9 database using the 10 CD3 script the partition of /db/backup reaches 100% and the backup fail. This problem occurred due to the previous backup failures. The backup script does not clean the previous snapshots created during a failed backup due to tape writing problems.

8.1.3 Corrective action

After a backup failure it is necessary to delete the snapshots: 1. fssnap -i 2. fssnap -d <name> 3. rm /spool/bss/snapshot* 4. Launch again the full backup, following the method.

8.1.4 Problem solved

Not applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 80/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

9. SUPERVISION 9.1 ALL THE ALARMS ARE AUTOMATICALLY PURGED

9.1.1 Reference FR

None

9.1.2 Problem description

Symptom: all the alarms are automatically purged, i.e. no alarm is visible in the current alarm list irrespective of OAD. They are all moved to historical alarm list (and can be seen through historical USM) Analysis: the auto purge is badly configured.

9.1.3 Corrective action

Check in AS Current USM Counter summary menu option administration which filter is defined for alarm purge. Note: when something like radio button "severity" or "alarm class" is selected but without any severity or alarm class, all the alarms are purged.

9.1.4 Problem solved

Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 81/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

9.2 EXPORT BSS FROM BSSUSM DOESN'T WORK

9.2.1 Reference FR

None

9.2.2 Problem description

Export BSS from BSSusm doesn't work Bssim database corruption is supposed.

9.2.3 Corrective action

This problem has not been reproducible, please do the following actions in order to investigate: Set BSSIM trace to 1+ (See Appendix VI). Check in the /alcatel/var/maintenance/trace/bssim_XX.err files if there is objecstore errors (xx is the bssim number).

If this is the case:

Edit /alcatel/omc3/bss/im/x/conf/param.cfg (where x is 8 for B9 BSC or 9 for B10 BSC) Set the DebugExport parameter to True instead of False Stop and start bssim_xx from DSMusm Launch again the Export BSS from BSSusm Sends all the /alcatel/var/maintenance/trace/bssim_XX Evolium/TD/O&M/OMC_GSM Put back the trace to level 1 2 for bssim (See Appendix VI). traces and dataflow and err files to

9.2.4 Problem solved

Not Applicable.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 82/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

9.3 ALARM FILTER NOT LOADABLE

9.3.1 Reference FR

None

9.3.2 Problem description

Symptom: In the ASUSM administration, after opening a predefined filter, the following error message is displayed "ERROR! Loading failed"

9.3.3 Corrective action

Exit from ASUSM (if it is not already done).

axadmin@host /alcatel/omc3/as/tools/script/convert51To52 t filter i /alcatel/var/share/as/ascurusm/repository/alarmFilters r o /tmp/filters > /dev/null 2>&1 axadmin@host cd /tmp/filters axadmin@host cp IndetExceptQOS acknowledged cleared errors matchAll matchNone notAcknowledged notCleared notReserved reserved /alcatel/var/share/as/ascurusm/repository/alarmFilters

For Each <user>, run the following commands as the user: axadmin@host cd /tmp/filters axadmin@host cp IndetExceptQOS acknowledged cleared errors matchAll matchNone notAcknowledged notCleared notReserved reserved /alcatel/var/home/<user>/asusmconf/repository/alarmFilters Example for the axadmin user : axadmin@host cd /tmp/filters axadmin@host cp IndetExceptQOS acknowledged cleared errors matchAll matchNone notAcknowledged notCleared notReserved reserved /alcatel/var/home/<user>/asusmconf/repository/alarmFilters BE CAREFUL, if the user filters have been already defined with the same name, the previous one will be lost ; see the APPENDIX IV in this case. Launch again ASUSM.

9.3.4 Problem solved

Not Applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 83/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

10. X-LARGE SPECIFIC 10.1 THE PROCEDURE OF X-LARGE OMC-R BACKUP DOESN`T WORK
10.1.1 Reference FR

None

10.1.2 Problem description

Symptom: The procedure of X-Large OMC-R backup doesnt work because on one agent Machine there is no BSC declared. Analysis: The backup procedure tries unsuccessfully to store the Objecstore database.

10.1.3 Corrective action

Declare a dummy BSC on this agent machine by using OAMUSM.

10.1.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 84/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

10.2 WITH A XLARGE, IMPOSSIBLE TO PERFORM A BACKUP WITH EXTERNAL AND INTERNAL TAPE DRIVER
10.2.1 Reference FR

None

10.2.2 Problem description

With a Xlarge, impossible to perform a backup of the OMC if we have an external driver on the master and internal driver on the agents for example: the mix configuration is not possible.

10.2.3 Corrective action

In order to perform a backup/restore of Xlarge, the device tape on the agent and on the master should have the same name. The backup/restore cannot be used if the device tapes on the agent and the master are not the same ( /dev/rmt/0n or /dev/rmt/1n ). The reason is that the same script is run on each machine( by rsh for the client ) The work-around is the change the name of the active tape device on the machine if necessary. Example : to move /dev/rmt/1n to /dev/rmt/0n, log with the root user: root@host cd /dev/rmt root@host cp 0n 0n.old root@host cp 0 0.old root@host cp 1 0 root@host cp 1n 0n

10.2.4 Problem solved

Not applicable

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 85/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this document, use and communication of its contents

11. MPM/NPOE

For further issues regarding NPO standalone please refer to the following link:

ftp://139.54.68.7/projects_ftp/PORTAL/VAL/M2/Trouble%20Shooting%20Guide%20NPO

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 86/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

12. APPENDIX I: SEEK PROBLEMS OF " SLOWNESS OMC "

12.1 ANALYSE OF THE PROBLEM:

- description of the symptoms met (slowness of cooling of the windows, opening, closing....)

12.2 ANALYSE OF THE ACTIONS IN PROGRESS:

check if there are audits in progress check if there are lots of PRCs opened?

12.3 CHECK IF RNIM IS NOT IN AUTOMATIC SITUATION OF RESTARTING:

In DSMUSM, clicks on RNIMSC: check on the right part the number of lives on going (with a max of 10) is indicated.

12.4 SEEK OF THE ' ORPHANS PROCESSES'

On the hosts and the HMI: to launch the following command: axadmin@host$ps edf | grep usm The following lines are displayed (for example): axadmin 29246 29258 0 Oct 09 pts/20 axadmin 29246 1 0 Oct 09 pts/20 2 :53 /alcatel/omc3/rn/usm/bin/rnusm 2 :54 /alcatel/omc3/rn/usm/bin/rnusm 2 :57 /alcatel/omc3/rn/usm/bin/rnusm

axadmin 29246 29579 0 Oct 09 pts/20

If the third field = '1 ' then it is necessary to carry out a kill ' pid' (here 29246). Analysis of the processes: In the directory: /usr/bin/truss launch the command: truss - p ' pid' (to stop, carry out ' ctrl C) In the directory: /usr/proc/bin To launch the command: pstack ' pid' (that provides a state of the stack)

It is significant to note the number of revolving processes (bssusm, DCN, have and rnusm) on the master and the 2 HMIs.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 87/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

12.5 CONTROL OF THE NUMBER OF SESSIONS / USERS ON THE HMI / HOST:

Check the number of session is consistent with the number of allowed session is defined in /alcatel/var/share/netconf/oflcf.in Observation of the ' perfmeter' and the revolving processes at the time or problem is detected (command ps)

12.6 DETECTION OF THE INPUT OF THE PROCESSES BSSIM / RNIM IN FLOW CONTROL:
In the directory /alcatel/var/maintenance/trace open the file rnimsc.trace. (vi rnimsc.trace) Set at the end of the file While going up since the end to seek ' Flow Control' Locate the blocks of traces and ' Full' Buffer (make the same one for bssim.trace and bssim.dataflow)

The process enters in flow control in order to adapt with the response time of the other processes.

12.7 CHECK IF THE LEVEL OF THE TRACES ' + ' IS ACTIVATED:

See Appendix 18 Example for bssim In the directory /alcatel/omc3/bss/im/9/conf open the file trace.cfg: vi trace.cfg check that beside ALL a sign ' + ' is not placed. Carry out in the same way for the other processes: bssusm, rnusm, rnim...

12.8 CHECK IF OBJECSTORE IS FREEZED

See Chapter 3.6 Objecstore is freezed

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 88/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

13. APPENDIX II: OMC3 TOOLS 13.1 HOW TO SAVE CELL AND BTS NAMES/GPRS CONFIGURATION

Solutions to keep the BTS and Cell names/GPRS configuration in case of BSSIM database corruption where Bssim is crashing all the time (so processingExport/bssusmcmd are not working =>not possible to use procedures described in the TroubleShooting guide to remove the BSC):

1. Try to force the restart of Bssim by modifying in the Bssim param.cfg file: Restart_Action 0 (instead of 1)

restart Bssim, then if it is stable, redo the on-going procedure from the beginning (processingExport). Don't forget to reinitialize the BSSIM parameter to 1 before the "BSS declare".

2. If Bssim is still crashing: (CELLNAME.01 tool) Warning: with this method, the ATER GPRS configuration is lost.

Pre-condition for running the tool :

Copy BSC (all cells) in a PRC Export PRC in a "exportrepository<x>" > cd /alcatel/var/share/AFTR/ACIE/"exportrepository<x>"

Launching the tool:

If you want to recover the ATER GPRS In order to recover the configuration from the last automatic EML export (if there is a latest one in the directory y, replace the 1 by y)

1) Copy the file exportMfsTtpInApd.pl in the /tmp/tools directory with the command "save as" available from the following URL: http://172.25.128.124/Ocg/Ist/checkbcchtool/

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 89/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

dos2unix /tmp/tools/exportMfsTtpInApd.pl /tmp/tools/exportMfsTtpInApd.pl chmod +x /tmp/tools/exportMfsTtpInApd.pl 3) cd /alcatel/var/share/AFTR/ACIE/ACIE_NLexport_Dir<y> , where <y> is the number of the latest export rnimsc 4) /alcatel/omc3/migration/script/generateBtsNameFromRnlExport.sh This will generate apd files for all BSC(s). 5) cd /alcatel/var/share/AFTR/ACIE/ACIE_BLexport/ACIE_BSS1export_Dir<z> where the <z> is the number of the latest export EML 6) launch /tmp/tools/exportMfsTtpInApd.pl Alcatel2MbTTP.csv /tmp/<x>.apd.omc3 where <x> is the BSC id Cabling information are added to file /tmp/<x>.apd.omc3.new mv /tmp/<x>.apd.omc3.new /alcatel/var/share/AFTR/MISC/<id>.apd.bsc

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 90/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

14. APPENDIX III : EXPLANATION ABOUT NEW SUBLISTS/FILTERS INSTALLATION (RESTRICTED TO ADMINISTRATOR USE OR ALCATEL SUPPORT)
Reference FR: None

After migration to the OMC-R release supporting the new default AS sublist and filters definition, the previously defined AS sublists and filters for each user are not impacted at all. When a new user is created (with OSM), the new default sublists are automatically affected to him. If an OMC-R operator wants to replace his existing sublists by the new default delivered one, the OMC-R administrator will use the installDefault script, located under /alcatel/omc3/as/tools/script directory. The syntax is : installDefault -user <userName> NB: script to be run as root on the master server This action will replace the sublists of username by the new default sublists.

Using installDefault with "-recover" option allows recovering the previous configuration. The syntax is : installDefault -user <userName> -recover

To replace the configuration for all users in one command, the "-all" option can be used. The syntax is : installDefault -all

Be careful: the installDefault script replaces entirely the ASUSM configuration for the users it is applied on.

If an operator wants to keep his own configuration, and wants also to take benefit of the new default filters, an intermediate solution consists in adding the filters to the AS configuration. That means that the new filters will be made available to build new sublists.

To perform that, the addDefaultFilter script can be used (located in /alcatel/omc3/as/tools/script directory). The syntax is: addDefaultFilter -user <userName> NB: script to be run as root on the master server. The result is the following: when creating a sublist, there are more defined filters available. The "-all" option extends the operation to all users at the same time.

Additionally, a specific script named changeDefault has been designed to be able to switch from the new default sublists/filters definition to the previous one (the one delivered with previous OMC-R releases) and reciprocally. This script is reserved for Alcatel use only.

To retrieve the default SFR ASUSM customization, use: changeDefault SFR

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 91/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

To summarize: when this version will be installed by Cegetel, use the script installDefault( but be careful, it replaces) or addDefaultFilter (do not change the existing conf but less easy to make it operational). If another customer delivers it, please do not forget to run:

changeDefault DTM. Remark:

If the file is in dos file (asusmconf/repository/alarmSortCriteria/TOC contains ^M characters) use the following unix command:

"dos2unix -ascii <dosfile> <unixfile>"

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 92/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

15. APPENDIX IV : HOW TO HAVE A COMPLETE INFORMATION ABOUT AN ALARM

There are two possibilities to have information about an alarm by using UNIX.

15.1 HISTORICAL ALARMS INFORMATION

Go under the subdirectory /alcatel/omc3/ei/script and launch the export hal debug script :
axadmin@host$ cd /alcatel/omc3/ei/script axadmin@host$ ./export_hal_debug <yyyy1-mm1-dd1> <yyyy2-mm2-dd2>

(where <yyyy1-mm1-dd1> is the date of the end, for instance 2001-05-17 and <yyyy2-mm2-dd2> is the date of the beginning, for instance 2001-05-16) A file named HAL_debug.tgz is produced under the subdirectory /tmp/HAL_debug.tgz.

Copy this file under another directory by using the command cp :

axadmin@host$ cd /tmp axadmin@host$ cp HAL_debug.tgz/otherdirectory

Go under otherdirectory, then decompress the HAL_debug.tgz file by using the commands gzip d HAL_debug.tgz then tar xvf HAL_debug.tar :
axadmin@host$ cd otherdirectory axadmin@host$ gzip d HAL_debug.tgz axadmin@host$ tar xvf HAL_debug.tar

The decompressed file is composed of each files (.csv) of the days you wanted the information : for instance hal_20010516.csv and hal_20010517.csv files.

Then rename all new created files like this :

axadmin@host$ mv hal_yyyymmdd.csv hal_yyyymmdd Ed.03Released CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 93/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

(for instance : mv hal_20010516.csv hal_20010516)

Open Excel program. The files created before must be transferred first on the PC. Then open (one by one) the new created files named hal_yyyymmdd (choose All Files option in Files of type). Select Delimited option then click on Next button and in Other option, write; character. End by clicking on Next and Finish button. You have now the information about all selected alarms.

15.2

CURRENT ALARMS INFORMATION

Select all alarms in the view current alarm list and do the action purge. All alarms will be put in the historical alarms feature, then, follow the method described in the paragraph 15.1

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 94/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

16. APPENDIX V : OMC TRACES

The OMC traces are present in /alcatel/var/maintenance/trace directory with the following name: <executable>_<Instance>.<trace | dataflow>, with: Instance = instance number of the containment tree higher object for BSSIM or BSSCOM, username in case of RNUSM or BSSUSM.

per instance. The trace file names are given in the param.cfg file When the size of traces files is too large, these file are saved in the /alcatel/var/maintenance/hist directory For level 1, only the error Statuses are written in the trace For level 2, only the process Events are written in the trace, and for bssim all the messages exchange with the BSC For level 3, all the Data Flows are detailed For level 4 (Method Call and Result): never trace result. When tracing a call (used in the called method entry), the input parameters are traced. Level 5 (platformCalls) can be used, as an example, to trace significant Database calls. By default the trace level is 1 2, in order to have the full trace, the level 1 2 must be modified by level 1+, it is not necessary to stop and start the involved process to take into account the level modification. Example 1 for /alcatel/omc3/bss/im/9/conf/trace.cfg with the default level: 1, 2
TRACE_OUTPUT { corona } /alcatel/var/maintenance/trace/bssim.trace

DATAFLOW_OUTPUT { corona } /alcatel/var/maintenance/trace/bssim.dataflow

TRACE_FILTERS { ALL { ALL 1 2 } }

Example 2 for /alcatel/omc3/bss/im/9/conf/trace.cfg with the maximum level:trace

TRACE_OUTPUT { corona } /alcatel/var/maintenance/trace/bssim.trace

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 95/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

DATAFLOW_OUTPUT { corona /alcatel/var/maintenance/trace/bssim.dataflow

TRACE_FILTERS { ALL { ALL 1+ }

16.1 BSSCOMM TRACES

16.1.1 BSSCOMM traces levels In the /alcatel/omc3/bsscomm/conf/param.cfg file of bsscomm, it is possible to configure the Retix traces and Ftam traces. For Retix traces set at TRUE : RetixTrace CmiseTrace RoseTrace PresentationTrace SessionTrace For Ftam traces set at TRUE : FtamTrace FtamFullTrace

When the param.cfg file is modified, stop and restart Bsscomm.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 96/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

16.1.2

BSSCOMM traces information

16.1.2.1 Cmise Actions

In bsscomm.trace search the word Abort or Timeout .

If you find Received Abort Indication , there are two possibilities:

Abort generated by BSC because the applications are not ok or the BSC doesnt accept the connection. In this case you can read in bsscomm trace : Reason of the abort : 6 - Abort because the network failed or it is a OSI problem. In this case you can read in bsscomm trace : Reason of the abort : 0 .

If you find Timeout : Bsscomm sent an action but the Bsc didnt sent an event report.

16.1.2.2 Ftam transfer

Search in bsscomm.trace : result = 255 few lines after you can read the error FTxxxx indicate the type of the error. For example : Local FTAM error FT1011 Lower layer failure in this case the x25 level has a problem, perform a x25trace. See the document [1] for more details

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 97/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

16.2 HOW TO LOOK AFTER A FLOW OF IMSI TRACES?

16.2.1 Find in the directory /alcatel/var/share/traces/ some files (realIMSI traces) with the name :
- MIB.<i>.<Bsc name>.* - MIR.<i>.<Bsc name>.* where <i> is the number of the BSC and Bsc name is the userLabel

16.2.2 In BSSIM Level 1+, find traces like :

- TRACE file : MIB.<i>.<Bsc name> - TRACE file : MIB.<i>.<Bsc name> - E R R O R : Unknown IMSI trace file type. This last trace is also present with normal level trace and corresponds to the case already seen on the field where we receive CALL traces and ,not IMSI traces but the symptoms (for BSSCOMM) are the same.

16.3 INFORMATIONS NEEDED TO ANALYSE A PROBLEM OF A CELL IN GSM MISALIGNED STATE :

1- What actions lead to the misaligned state? Activation of a PRC ? Modification of parameters in the SC ? logical audit ? ... 2- Which actions have been triggered to try to recover an aligned state? Use of TSG recommendations? Did it work? 3- Day/time of the problem as precise as possible. 4- The value of alignmentStatusCause, the number and name of the impacted BSCs, the LAC/CI, the name and the number per BSC of the misaligned cells. 5- Traces necessary for the analysis: - At least, the traces of the bssim corresponding the BSC (in /alcatel/var/maintenance/trace or /alcatel/var/lmaintenance/hist if the problem occurs in a date no more present in the file under the /alcatel/var/maintenance/trace). To study such kind of problems, it is no more necessary to be in 1+ level, but if you have the 1+ traces it is obviously interesting. - Logs of rnusm of the user which did the action (files /alcatel/var/maintenance/log/rnusm.log.<user> and the historical one in /alcatel/var/lmaintenance/hist if necessary). Be careful, they are located on the HMI where the action has been triggered (not necessary the master) - The files present under the directory /alcatel/var/maintenance/log/SAVEDLOG of the master. - If possible and if the misaligned state came when activating a PRC, traces of rnimsc and of the activated prc.

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 98/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

16.4 IMPORT/EXPORT TRACES

The import/export directory file is saved in tar file in /alcatel/var/mantenance/trace and after moved in gz file in /alcatel/var/mantenance/hist/ With the name like: import_<i>_<prc_name>_<j>.tar.old.<YY.MM.DD>.<hh.mm.ss>.gz Or export_<i>_<prc_name>_<j>.tar.old.<YY.MM.DD>.<hh.mm.ss>.gz Where <i> = directory number where the import/export is performed <prc_name>.= PRC userlabel

16.5 <J> = PRC NUMBER TRACEOBJECTORE TRACES

They are present by default in /tmp/ostore/oss_out

SAME

AS

USED

FOR

RNIMPR

16.6 HOW TO GET ASCII FILE INFORMATION ABOUT THE EML BSSIM HW AUDIT
Update the GenerateDecodedHwFiles in the /alcatel/omc3/bss/im/9/conf/param.cfg stop and start the corresponding bssim and launch again the HW audit, the ASCII file be present in /tmp/*_dec

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 99/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

INDEX 3BKA20FBR076796,21 3BKA20FBR079298,8 3BKA20FBR079787,38 3BKA20FBR080657,59 3BKA20FBR084310,29 3BKA20FBR084852,40 3BKA20FBR101545,63 3BKA20FBR102026,64 3BKA20FBR102455,51 3BKA20FBR102456,46, 47, 56 3BKA20FBR102457,53 3BKA20FBR108030,23 3BKA20FBR108149,56 3BKA20FBR109097,31 3BKA20FBR109284,62 3BKA20FBR115486,63 3BKA20FBR115503,65 3BKA20FBR116885,35, 36 3BKA20FBR118323,13 3BKA20FBR120127,57 3BKA20FBR121702,8 3BKA20FBR122635,8 3BKA32CBR102358,8 3BKA32FBR088945,22 3BKA32FBR106330,13 3BKA32FBR113851,37 3BKA32FBR114088,32 3BKA32FBR118108,60 Adjacency adjacency reveals wrong BCCH/BSIC on cell ...,41 auto purge alarm,61 BSSCOMM,38 BSSIM database,8 busy,17

busy (BSC),18

cleanUp,56 cpio: Cannot create temporary file, errno 18, Cross-device link,31 Date/Time Synchro,40 discover is refused,17 dsmim,30 DTM,72 ENABLE_FORBID-UPLOAD,43 flow control,68 IconBox,23 inConfig,45 inittab,30 logicalparm_modify,44 m64config,37 metrica,59 migration,56 misaligned,42 MLUsession,55 moveBscInterOmc.pl,10 nameserver,13 NML not available,8 nprdb,59 Objecstore,68 Objectstore,20, 25 ObjectStore,26 ODI,20 oflcf.in,68 oscmrf,26 osmd,23 osserver,25 ossvrchkpt,25 pad,39 perfmeter,68

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 100/101

not permitted without written authorization from Evolium.

All rights reserved. Passing on and copying of this

document, use and communication of its contents

pstack,67

timeout,66 trace.cfg,68 TRACES,75 truss,67 txn.log,25 unlockaccount,21 UploadRadioParameter,42 X25 configuration problem,39

qtip,59 RNIM database,13 RNIM not responding,8 SEC,29 SFR,71

sublis,71

tar -cv,13

the user is unknown to the security manager,19

Ed.03Released

CONFIDENTIAL
tsg_b10ed03Released

OMC3 B10 Troubleshooting Guide 3BK 29007 LAAA PWZZA 2/101