All new accounts created on Gitlab now require administrator approval. If you invite any collaborators, please let Flux staff know so they can approve the accounts.

Commit 6c6b2ee2 authored by Mike Hibler's avatar Mike Hibler

Started getting a strange RPC timeout from ops when trying to do the

"umount -A -t nfs".  As far as I can tell, the umount really works even with
the error.  I also cannot reproduce this manually.  Since we always intended
for a failure of this step to be non-fatal, I actually made it that way.

Why go after the root cause when you can treat the symptom!
parent 31c75950
......@@ -509,8 +509,18 @@ sub SetupFsNode()
# Clean up a few things on the image and create symlinks into ${TBDIR} for
# /proj, /users, /groups and /scratch. Also allows /share to be created/
#
mysystem("umount -A -t nfs");
# In case umount fails!
print "Command: 'umount -A -t nfs'\n";
print "Started at: " . libsetup::TBTimeStamp() . "\n";
system("umount -A -t nfs");
if ($?) {
print STDERR "*** umount of NFS filesystems failed\n";
}
#
# Uber-paranoid: even if the umount says it works, don't trust it.
# Move the old mount points out of the way no matter what.
# It is not super critical that these be unmounted at this time as
# we will be rebooting shortly anyway; we just need them out of the way.
#
mysystem("cd /; mv -f users users.old");
mysystem("cd /; mv -f proj proj.old");
mysystem("cd /; mv -f share share.old");
......@@ -870,8 +880,18 @@ sub SetupOpsNode($)
# Clean up a few things on the image and create symlinks into ${TBDIR} for
# /proj, /users, /groups and /scratch. Also allows /share to be created/
#
mysystem("umount -A -t nfs");
# In case umount fails!
print "Command: 'umount -A -t nfs'\n";
print "Started at: " . libsetup::TBTimeStamp() . "\n";
system("umount -A -t nfs");
if ($?) {
print STDERR "*** umount of NFS filesystems failed\n";
}
#
# Uber-paranoid: even if the umount says it works, don't trust it.
# Move the old mount points out of the way no matter what.
# It is not super critical that these be unmounted at this time as
# we will be rebooting shortly anyway; we just need them out of the way.
#
mysystem("cd /; mv -f users users.old");
mysystem("cd /; mv -f proj proj.old");
mysystem("cd /; mv -f share share.old");
......@@ -1245,7 +1265,28 @@ sub SetupBossNode()
# We no longer need anything from NFS, and we need to unmount everything
# so we can mount new NFS filesystems in their proper places.
#
mysystem("umount -A -t nfs");
print "Command: 'umount -A -t nfs'\n";
print "Started at: " . libsetup::TBTimeStamp() . "\n";
system("umount -A -t nfs");
if ($?) {
print STDERR "*** umount of NFS filesystems failed\n";
}
#
# Uber-paranoid: even if the umount says it works, don't trust it.
# Move the old mount points out of the way no matter what.
# It is not super critical that these be unmounted at this time as
# we will be rebooting shortly anyway; we just need them out of the way.
#
mysystem("cd /; mv -f users users.old; mkdir users");
mysystem("cd /; mv -f proj proj.old; mkdir proj");
mysystem("cd /; mv -f share share.old; mkdir share");
# This might not exist
mysystem("cd /; mv -f groups groups.old; mkdir groups")
if (-d "/groups");
if ($SCRATCHFS) {
mysystem("cd /; mv -f scratch scratch.old; mkdir scratch")
if (-d "/scratch");
}
#
# Lets mount the package dir so that we can pass off some stuff to
......@@ -1872,7 +1913,7 @@ sub mysystem($;$)
if (!defined($retrycount));
while ($retrycount--) {
print "Command: '$command\'\n";
print "Command: '$command'\n";
print "Started at: " . libsetup::TBTimeStamp() . "\n";
system($command);
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment