+ Reply to Thread
Results 1 to 12 of 12
  1. Senior Member
    Join Date
    Dec 2011
    Posts
    330

    Certifications
    CCNP Collab, CCDP, CCNP R&S, CCNP V, CCP-N NetScaler, Net+, A+
    #1

    Default APB for Voice people here. Jump Upgrade to 9.1.2

    Doing a jump upgrade from 6.1.4+ or 7.1.5 to 9.1(2). P2V as well (MCS to UCS). Pretty easy procedure.

    1. DRS Production Cluster
    2. New nodes matching production cluster in VM environment.
    - Create VMs with OVA templaces (Change OS to RHEL v3 32-bit in VM settings)
    - Install same version as production cluster using bootable ISOs from TAC
    Note: NTP and DNS must be working and reachable in lab environment for clean installation of fresh cluster.
    3. Restore DRS from step 1 to new VM cluster environment.
    4. Reboot cluster: Should have clone of production environment in new VM lab minus licenses at this point.
    4. Install Refresh COP file to Pub and Sub nodes.
    6. Start Upgrade to 9.1(2) using downloaded ISO from Cisco CCO.
    7. Shutdown VM after upgrade to 9 complete via CLI on VM console. Change OS to RHEL v5 32-bit in VM Settings.
    8. Boot back up, Apply new v9 licenses
    9. DRS v9 Lab cluster
    10. Install fresh new 9.1(2) cluster with bootable media. Required step to normalize unaligned partitions from jump upgrade process.
    11. DRS from step 9 to new nodes from step 10.

    Not bad right? Well I was running into error in database component when upgrading 6.1.5 to 9.1(2) from step 6. Not having a reachable NTP server can cause this but wasn't the issue in my case. Tons of searching on support forums and nothing... well after going though database logs on inactive partition something interesting came up.

    file view inactivelog cm/trace/dbl/sdi/installdb_ru.log.err

    12:23:09.085 |*ERROR* Error executing "insert into EndUser (allowcticontrolflag,assocpc,building,deletedtimes tamp,department,enablemobilevoice,enablemobility,f acsimiletelephonenumber,firstname,fkdirectoryplugi nconfig,fkmatrix_presence,
    homephone,mailid,manager,maxdeskpickupwaittime,mid dlename,mobile,nickname,ocsprimaryuseraddress,page r,passwordreverse,pkid,remotedestinationlimit,site ,status,telephonenumber,title,tkuserprofile,unique identifier,userid) values
    ('T','','',123456,'','F','F','','******','32b66ffb-48d6-c3d5-33c1-1653b28396fb','ad243d17-98b4-4118-8feb-5ff2e1b781ac','XXX XXX-XXXX','*****.Null@*******.com','G***A',10000,'','X XX XXX-XXXX','',
    'sip:*****.Null@************.com','','69c4f936f9cd f45f6bbca2570c31215629bb5d6fb97493478b8ff3db6fffbc 55','a677593d-d61e-4327-91e1-031bf4bc90ad',4,'',1,'2848','District',1,'f195e958 a1652943a1a2c7ab03885743',
    'null'): [Informix][Informix ODBC Driver][Informix]Cannot insert a null into column (enduser.lastname).
    12:23:16.504 |*ERROR* Error executing "insert into credential"




    This particular end user has a last name of NULL, which is a valid SQL command in the Informix database. So when it tried to insert his last name in that field it ran the null command instead and killed the whole database upgrade. Changed his last name in AD to aNULL, synced with CUCM, new DRS and started the process above again and it worked.

    If you ever run into problems upgrading CUCM or Unity...make sure you don't have an end user with the name NULL anywhere!!!!!!!!!
    Last edited by aaron0011; 01-23-2014 at 01:43 AM.
    Reply With Quote Quote  

  2. SS -->
  3. Senior Member
    Join Date
    Mar 2013
    Posts
    103
    #2
    Wow nuts.. who would've thought. Did you inform Cisco?
    Reply With Quote Quote  

  4. Senior Member
    Join Date
    Dec 2011
    Posts
    330

    Certifications
    CCNP Collab, CCDP, CCNP R&S, CCNP V, CCP-N NetScaler, Net+, A+
    #3
    Yes, nothing they can do I was told.
    Reply With Quote Quote  

  5. Senior Member
    Join Date
    May 2009
    Location
    DMV
    Posts
    2,205

    Certifications
    CCNP, CCNP(V), S+ CCIE V(written)
    #4
    WoW.


    You need to get this on a BUG tracker. Did you open a TAC case?
    Currently Reading

    CUCM SRND 9x/10, UCCX SRND 10x, QOS SRND, SIP Trunking Guide, anything contact center related
    Reply With Quote Quote  

  6. Senior Member
    Join Date
    Dec 2011
    Posts
    330

    Certifications
    CCNP Collab, CCDP, CCNP R&S, CCNP V, CCP-N NetScaler, Net+, A+
    #5
    Quote Originally Posted by shodown View Post
    WoW.


    You need to get this on a BUG tracker. Did you open a TAC case?
    Yes, I did and it was logged. This is the first instance of this happening according to the escalation engineer I spoke with who finally understood what was happening. The first few guys to take the case were a bit clueless. Who knows if it will find it's way into a bug id or not. It's a rare occurrence for sure.
    Reply With Quote Quote  

  7. Senior Member
    Join Date
    May 2009
    Location
    DMV
    Posts
    2,205

    Certifications
    CCNP, CCNP(V), S+ CCIE V(written)
    #6
    You can't expect first line support to catch those problems. You prob has to speak to a customer engineer IV to get this solved. Also the higher you go in TAC the less they know about call manager. The first few guys know call manager, then after that you may get a LINUX guy, SQL Guy, and so on.
    Currently Reading

    CUCM SRND 9x/10, UCCX SRND 10x, QOS SRND, SIP Trunking Guide, anything contact center related
    Reply With Quote Quote  

  8. Senior Member
    Join Date
    Dec 2011
    Posts
    330

    Certifications
    CCNP Collab, CCDP, CCNP R&S, CCNP V, CCP-N NetScaler, Net+, A+
    #7
    Quote Originally Posted by shodown View Post
    You can't expect first line support to catch those problems. You prob has to speak to a customer engineer IV to get this solved. Also the higher you go in TAC the less they know about call manager. The first few guys know call manager, then after that you may get a LINUX guy, SQL Guy, and so on.
    Agreed but there is a special Drive to 9 TAC designation going on right now. Any case in that queue should have engineers intimate with the upgrade process. I found the errors indicating exact time and where in the process it failed using RTMT instal logs. It took the escalation engineer to use that data to dig up the above from that one specific db log on the inactive partition.

    Also an FYI. If a refresh fails starting from 6.1.4+ it will not recover to previous partition. You can't even run a recovery to boot back to previous version. I had to upgrade from 6.1.5 to 7.1.5, then refresh jump upgrade just to have it a fail again. Going from 7.1.5 it actually recovers back to previous version where the logs can actually be accessed. Without an end user with the name NULL going from v6 directly to 9 works fine.
    Last edited by aaron0011; 01-23-2014 at 04:27 AM.
    Reply With Quote Quote  

  9. Senior Member
    Join Date
    May 2009
    Location
    DMV
    Posts
    2,205

    Certifications
    CCNP, CCNP(V), S+ CCIE V(written)
    #8
    Yeah these upgrades are a pain in the ASS, but way better that 8.6 upgrade process where we had to have jump servers. I have around 10 of these scheduled until june, no Greenfield's.
    Currently Reading

    CUCM SRND 9x/10, UCCX SRND 10x, QOS SRND, SIP Trunking Guide, anything contact center related
    Reply With Quote Quote  

  10. Senior Member
    Join Date
    Dec 2011
    Posts
    330

    Certifications
    CCNP Collab, CCDP, CCNP R&S, CCNP V, CCP-N NetScaler, Net+, A+
    #9
    Quote Originally Posted by shodown View Post
    Yeah these upgrades are a pain in the ASS, but way better that 8.6 upgrade process where we had to have jump servers. I have around 10 of these scheduled until june, no Greenfield's.
    Agree, the jump upgrade is huge progress in going from 6, 7, 8.0 straight to the latest 9.1 build. The fact that you can do it all virtual now is plus. Kudos to Cisco for addressing this.
    Reply With Quote Quote  

  11. Senior Member wintermute000's Avatar
    Join Date
    Jan 2013
    Location
    Melbourne, Australia
    Posts
    170

    Certifications
    CCNP R&S, CCNP Voice, CCNP Security, CCDP
    #10
    Without virtual I wouldn't even want to know. Our vendor did 4--> 6 and IIRC they actually sucked all the info out of 4 using a custom tool then repopulated the config into v6 (good thing back then there were no local RGs, E164 routing, SAF, SNR, sip dial plans yada yada).

    At least nowadays whatever happens you can always fire up the old VMs again for foolproof rollback.... a lot safer than relying on that backup RAID-1 drive you pulled from the MCS pub chassis.

    Thanks for documenting above, good knowledge.
    Reply With Quote Quote  

  12. Senior Member JeanM's Avatar
    Join Date
    Mar 2012
    Location
    California
    Posts
    1,105

    Certifications
    CCNA, MCP, S+, N+, A+
    #11
    Nice catch!
    Reply With Quote Quote  

  13. /threadkiller ande0255's Avatar
    Join Date
    Sep 2013
    Location
    Around
    Posts
    1,160

    Certifications
    CCNA R&S, Voice, Security
    #12
    Thanks for the post, really good knowledge!
    Reply With Quote Quote  

+ Reply to Thread

Social Networking & Bookmarks