For CentOS servers an update for the nss-softokn package was release today – nss-softokn-3.14.3-19
However, nss-softokn-3.14.3-19 needs nss-softokn-freebl-3.14.3-19 to operate properly, and vice versa, but those packages do not have checks in place to make sure that a matching version of the other package are also installed.
Thus if you yum update only installed one of the packages you will end up with a broken YUM and RPM.
You might see error messages like these when trying to run YUM and RPM commands:
error: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID xxx BAD
error: rpmdbNextIterator: skipping h# 1784 Header V3 RSA/SHA1 Signature, key ID xxx BAD
Most of the time you will have had nss-softokn-3.14.3-19 installed but not nss-softokn-freebl-3.14.3-19
To fix this you have to:
1. Manually download nss-softokn-freebl-3.14.3-19
yumdownloader nss-softokn-freebl
or wget the RPMs
64-Bit servers / x86_64 run
wget ftp://195.220.108.108/linux/centos/6.6/updates/x86_64/Packages/nss-softokn-freebl-3.14.3-19.el6_6.x86_64.rpm
32-Bit Servers / i686 run
wget ftp://195.220.108.108/linux/centos/6.6/updates/i386/Packages/nss-softokn-freebl-3.14.3-19.el6_6.i686.rpm
2. Extract the RPM
64-Bit servers / x86_64 run
rpm2cpio nss-softokn-freebl-3.14.3-19.el6_6.x86_64.rpm | cpio -idmv
32-Bit Servers / i686 run
rpm2cpio nss-softokn-freebl-3.14.3-19.el6_6.i686.rpm | cpio -idmv
3. Copy .libfreeblpriv3.* to correct location
64-Bit servers / x86_64 run
cp ./lib64/libfreeblpriv3.* /lib64
32-Bit Servers / i686 run
cp ./lib/libfreeblpriv3.* /lib
4. Rerun Yum Update to update nss-softokn-freebl and FIX YUM and RPM
yum update
If you still have problems you might have to reinstall yum as below for a 64bit server, 32bit servers will just need to locate the 32bit versions of the rpms! But most servers won’t need this:
wget http://mirror.centos.org/centos/6/os/x86_64/Packages/yum-3.2.29-60.el6.centos.noarch.rpm wget http://mirror.centos.org/centos/6/os/x86_64/Packages/yum-plugin-fastestmirror-1.1.30-30.el6.noarch.rpm wget http://mirror.centos.org/centos/6/os/x86_64/Packages/centos-release-6-6.el6.centos.12.2.x86_64.rpm
rpm -ivh --nodeps yum-3.2.29-60.el6.centos.noarch.rpm rpm -ivh yum-plugin-fastestmirror-1.1.30-30.el6.noarch.rpm rpm -ivh centos-release-6-6.el6.centos.12.2.x86_64.rpm
I have also had some questions about how to restore the rpmdb that got corrupted while trying to fix the problem, there are two ways:
1. Copy a good copy of the /var/lib/rpm folder over from an identical good server of from a backups – you do have backups right?
On good server or from backup:
tar -cf rpmbackup.tar /var/lib/rpm
Then send the tar over to the bad server
On broken server:
cd /var/lib
mv /var/lib/rpm /var/lib/rpm.bak
tar -xf /location/to/rpmbackup.tar
Then you should be good to go!
The second way is to reinstall every rpm ever installed via checking the rpm install log. See the comment below
Bug Report: https://bugzilla.redhat.com/show_bug.cgi?id=1182337
You are a life saver! Donation on the way 😉 <3
Great! Thanks for the donation!
Donation sent.
Thanks!!!
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug! Thank you so much for the Donation!
Repeating what everyone below is saying. You just saved my weekend. I was about to wipe the machine and start over. Now if google would bump this to the top result…
Thanks!
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug! What keywords where you using when searching on Google?
Thanks a lot! Your post saved me at least 3-4 hours to get it fixed!
Hello, it’s just a right on time fix for me.
I was just not sure why webmin failed to upgrade.
It’s all fixed now. thanks!
many many thanks. it finally solved the problem. my production server was broken for about a week and i was frustrated because i can’t find the solution. god bless you – kindly man.
This helped me to save a machine left in a weird state after being “fenced” during an update (multilib issues afterwards). Restoring the rpm-db and /etc/pki didn’t help at all (although it fixed the multilib issues, rpm and yum refused to install new packages or remove old ones).
Although I also needed to copy the lib-files from the package nss-softokn too, but hey … it worked perfectly after that 🙂
Thanks a million for this post. Been scratching my head for days.
A real life saver – Initially reported the Bug to bugs.Centos.org but now also see your reference at bugzilla.redhat.com
Kudos all round.
I to greatly appreciate the info, I however get the following error.. Any thought?
[root@mail ~]# rpm -ivh –nodeps yum-3.2.29-60.el6.centos.noarch.rpm
Preparing… Segmentation fault (core dumped)
Thank you, it resolved my problem and also a problem with epel repo that was giving an error “Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again”
Very good, thank you for posting this fix
Thnaks
Thanks for that, saved me definitely a lot of time!!!
Thanks in advance ! Work’s perfect !!!!
Thank you!
muchas gracias amigo, thanks a lot
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Well, all I can do here is join the chorus in thanking you for the info. I was encountering the same problem and your solution fixed it.
Thank you for the donation! I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Thank you very much.
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
O rezolvare excelenta de catre un pasionat care intelege sistemul de operare centos, multumesc!
I love you, Thank you very much. Muchas gracias AMIGO!!
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
I attempted your fix, but rpm2cpio just gives this output:
When I do the command (I renamed the softokn archive from the original download filename to nss-softokn.rpm. It’s the same file that I downloaded via these instructions).
rpm2cpio nss-softokn.rpm | cpio -idmv
OUTPUT:
Error: header not recognized
cpio: premature end of archive
Is there any way to fix this? I can’t fix anything on my server because rpm2cpio won’t work in the least bit.
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug! That error seems to indicate that the rpm you downloaded is corrupted. Try to redownload or try downloading from another mirror.
You Sir are a genius. Your solution worked perfectly. Thank you, thank you, thank you.
🙂
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Hello – Just wanted to say thank you for posting this. I had an idea of the issue from various bug reports, but couldn’t work out how to fix. cheers Pete
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Thanks a lot !!! . This solved my issue with yum.
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Risolto il problema su 108 server! / Fixed issue on 108 servers!
Grazie! / thank you!
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Thanks a million, I was scratching my head on this one.. But you helped me get it resolved.
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Sheesh thank you so much this was a freaking nightmare.
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
After applying your fix yum now seems to work but the rpm database only has a few entries instead of all packages installed on the system. I see someone had the same experience but is there an easier way to restore the database without downloading all rpm packages?
Another way to fix the rpmdb is to copy over the /var/lib/rpm folder from another similar good working server or from a recent backup! Remember to back up your original /var/lib/rpm folder just in case!
Thank You!!! was driving me crazy, our secure cc charge forms stopped working on our sites and this solved it, thanks for taking the time to post it
No problem! I am happy yo hear that this fix helped you with your secure credit card form authentication problems caused by the broken nss-softokn update!
Thank you very much. After much searching and hair pulling, this corrected the issues.
THANKS
Glad I could help with your nss-softokn update problems!
Thanks, this has solved our sudden PayPal connection issue, which turns out was caused by this nss-softokn package update.
Now we are back to working order.
Thanks a million!!!!
Glad I could help you with your PayPal connection issues caused by the nss-softokn update that breaks yum update!
thank you very much for this.
i have spent hours looking for a solution for this
it is always so good to find someone who actually knows how things work!
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
You are a life SAVER!!!! Did not have to reinstall yum in my case. Great instructions
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug! Most of the time you WONT need to reinstall yum!
You’re a GENIUS man.
They should hire you, no information whatsoever on Centos site and my server was out, no yum or rpm, all broken.
I really can’t understand how could they release a package with such a bug that could absolutely break yum and rpm, this is serious.
Thanks so much, again, your tutorial should be on main site of CentOS they should be ashame of doing this and not telling anything about it.
Glad you found my guide! I totally agree with what you mean! It took me several hours to find a bug report and finally a fix. Glad it worked for you!
This worked perfectly to fix the softokn issue. But since I did some damages before, I had to reinstall yum:
wget http://mirror.centos.org/centos/6/os/x86_64/Packages/yum-3.2.29-60.el6.centos.noarch.rpm
wget http://mirror.centos.org/centos/6/os/x86_64/Packages/yum-plugin-fastestmirror-1.1.30-30.el6.noarch.rpm
wget http://mirror.centos.org/centos/6/os/x86_64/Packages/centos-release-6-6.el6.centos.12.2.x86_64.rpm
rpm -ivh –nodeps yum-3.2.29-60.el6.centos.noarch.rpm
rpm -ivh yum-plugin-fastestmirror-1.1.30-30.el6.noarch.rpm
rpm -ivh centos-release-6-6.el6.centos.12.2.x86_64.rpm
Then yum update worked ok.
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug! I will add the how to on how to reinstall yum to my guide.
I forgot to say, thank you. 🙂
More notes… I messed up really bad before I found and applied your solution. So much that my rpmdb was broken, meaning that when doing a *yum list installed* there was about 10 lines instead of 454, that meant *bye bye yum updates*. So I had to rebuild it by downloading all 454 rpms, then do an *rmp -i –justdb* on each one. I did have an rpmpkgs log, but even with that it took me some 6 hours to complete.
So, my lesson learned, make a regular backup of the rpmdb. 😉
thank you again.
c.
No problem! Glad I could help! Another way to fix the rpmdb is to copy over the /var/lib/rpm folder from another similar good working server. Remember to back up your original /var/lib/rpm folder just in case!
Great advice, saved a lot of time fault finding.
Many thanks
… oh merci ! MERCI !!! bug corrigé !
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
You saved my day, Thanks a million
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Hi,
Thanks so much….
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Excellent advice, worked a treat. Many thanks
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Thank you for the explanation and precise instructions, saved me a lot of grief. After being unable to use anything related to nss and unable to repair the database with the mismatched packages this saved me.
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Thanks ! That was causing me headache directly on my WordPress, CURL & SSL on some plugins were faulty !
Perfect
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
Thanks for this! This issue was causing some real problems on 2 servers of mine…
I am glad you found my guide on how to fix yum and rpm after the softokn update bug!
Pure Genius! Had patches installed on around 30 servers. Only noticed the issue when trying to install new packages on one of the servers. Worked beautifully.
I am glad you found my guide on how to fix yum and rpm after the softokn update bug!
Thanks for the easy instructions, I had the following errors:
Header V3 RSA/SHA1 Signature, key ID c105b9de: BAD
error: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: BAD
Your instructions on resolving them worked perfectly, and now the yum update is Solved.
I am glad you found my guide on how to fix yum and rpm after the softokn update bug!
Thanks a million, you save my day ( and night ) !!!
I am glad you found my guide on how to fix yum and rpm after the softokn update bug!
I’m almost certain this is what broke yum for me, but following the steps did not fix it. Is there anything else I can try?
I am glad you found my guide on how to fix yum and rpm after the softokn update bug – What problems are you still having? Can you post some error messages? Are the permissions on those files you copied the same as the rest in the folder?
Many thanks DieSkim.
Your solution solved CA cert issues with my VPS.
I am glad you found my guide on how to fix yum and rpm after the nss softokn update bug!
It looks like I mangled my rpm db before applying your fix. I’m going to try the suggestions from Claude Nadon above and I’ll be sure to post back here if that works. If I don’t have any luck I’ll come back with the error messages I’m getting.
Glad you found the fix for the broken yum update caused by the nss-softokn update. I also added some options to fix the rpm database