A rescue near The Hidden Castle

The Hidden Castle is a castle-themed resort in Medak district near Hyderabad, India.  To reach there from the city is approx. 2 hours drive.  We were still astonished by the grand theme of this place when we checked in to the attic room that we had booked during our family vacation.

While gazing out of the window of our room, we heard cries of a dog coming from some distance.  From the farms that were beyond the resort.  The dog was nowhere to be seen. But the cry for help was repeating.  During the afternoon and the evening I made attempts to cross the boundary of the resort, but the fence was an obstacle I could not overcome.  All night the dog was crying for help.

The next morning, a short hike was arranged for all guests by the resort staff.  While going for the hike, I found the way to walk out of the resort towards the area where the dog was crying.

After the trek, I and my daughter stepped out to find where the dog was and what was the matter with it.  A search through some thorny bushes towards the source of the sound lead me to the dog who was tied to the fence with a metal chain.  I untied the chain from the fence.  But the dog was too tired to even stand up.  The two biscuits I gave were not sufficient to restore the energy.

 

Khushi jumped over the fence and untied the chain from the dog's neck.  There was nobody to be seen in the rice farms.  Why to tie the dog in the farm and leave him starving for days.

The dog was completely exhausted and could not stand up.  I ran to the kitchen of the resort and got some slices of bread.

After eating the bread pieces the doggy made attempts to stand up.

After some time we had to leave the doggy and come back in the resort.  Our room checkout time was already over.  When we started walking, the doggy got up and we saw he had thee legs.  One leg was missing.  He knew the path where he wanted to go.  Khushi and I jumped over the fence.  Observing that we left the farm and went on the other side, the doggy kept howling and asking us to join him instead.

Our time with the doggy was limited and over.  And the same is for all of us.

यथा काष्ठं च काष्ठं च समेयातां महोदधौ ।
समेत्य च व्यपेयातां तद्वद् भूतसमागम: ॥

If this world is considered to be a big ocean, then we all are like logs of wood floating around.  We come together and then are separated.  None of us are permanent companions.  All we have is to spend our time together while we are together.  The dog is also a living being.  Why tie him to starve?  Wouldn't we want to help our fellow beings to be free?  Isn't that the purpose we are given.  In medieval times folks used to go to Kashi to attain freedom from this mortal world.  In this modern age, someday I want to visit Muktinath, known to be a Mukti Kshetra, which literally means the 'liberation arena'.  What worked in the Golden Age (Satya Yug) would not work in the modern times (Kali Yug).  Rather than the rituals and the mantras, Karma is the formula for Kali Yug, the modern times.

How to uninstall IBM Spectrum Scale

There are situations when IBM Spectrum Scale needs to be uninstalled.  Here a step by step guide for doing this.

What is IBM Spectrum Scale
IBM's clustered file system that provides parallel data access from multiple nodes was branded as Spectrum Scale.  It is now re-branded as IBM Storage Scale.

To know more about IBM Spectrum Scale, please visit https://www.ibm.com/docs/en/storage-scale/5.1.8?topic=overview-storage-scale

IBM Spectrum Scale is a file system. But not an ordinary file system running locally on a single computer. Spectrum Scale runs on multiple computers. These computers together make a cluster. They are known as the nodes of the cluster.  Some of the nodes are arranged access to a storage that is present in the network. The storage is available to the nodes in the form of Network Shared Disks (NSDs). The available NSDs are used to create file system.  Customers use the file system to store and access data, via NFS, CIFS, or object protocols.

IBM Spectrum Scale provides concurrent high-speed file access. Applications that access the data may be running on multiple systems and accessing the data in parallel.

A Spectrum Scale cluster may consist of 1 to 56 nodes.  The nodes could be assigned the roles of quorum nodes and data nodes.

Before we begin
A spectrum scale cluster consists of one or more nodes.  At every node of the cluster, the spectrum scale packages are installed.  Apart from the packages, some files and directories are present which are required for the functioning of the node.  This procedure is of uninstalling Spectrum Scale from a single node.

Pre-checks
Before uninstalling spectrum scale from a node, please check :
(a) Is the node part of a spectrum scale cluster?  Run command mmlscluster to check this.
(b) Does the node have NSDs?  Are the NSDs local to the node or are they shared NSDs?  Are any of the NSDs being used?  Run command mmlsnsd to check these.
(c) Are any spectrum scale file systems mounted at this node?  Run command mmlsmount to check this.

Uninstallation Procedure

1. If the node has any file systems mounted, then all file systems must be unmounted from this node.  Unmount all file systems from the node by running command mmumount all
Verify that the node does not have any spectrum scale file system mounted by running command mmlsmount all

2. If the node has shared NSDs, then all shared NSDs should be removed.  In this case, please think for a while "why are we really uninstalling spectrum scale from this node?  Is this really necessary?"  If answer is affirmative, then please proceed to remove all the shared NSDs.  For doing so, the shared NSDs must be in unused state.  Please check if any of the shared NSDs are being used in any spectrum scale file system(s).  If yes, then remove the NSDs from the file system(s) using command mmdeldisk
Verify that the NSDs in question are in unused state by running command mmlsnsd
Delete the NSDs by running command mmdelnsd
Verify that the NSDs in question are deleted by running command mmlsnsd

3. If the node has local NSDs, then all local NSDs should be removed.  Please check if any of the local NSDs are being used in any spectrum scale file system(s).  If yes, then remove the NSDs from the file system(s) using command mmdeldisk
Verify that the NSDs in question are in unused state by running command mmlsnsd
Delete the NSDs by running command mmdelnsd
Verify that the NSDs in question are deleted by running command mmlsnsd

4. If the node is part of a spectrum scale cluster, then it should be removed from the cluster.  To do so, login to any other node in the cluster, and run command mmdelnode
Verify that the node is not part of the cluster by running command mmlscluster

5. If the spectrum scale software is running at the node, stop it by running command mmshutdown

6. Remove all spectrum scale packages that are currently installed.
Some of the spectrum scale packages have dependencies, and can not be removed individually.  So all spectrum scale packages should be removed together using a single command.

On a node running Red Hat Enterprise Linux, you could run `rpm -qa | grep gpfs` to get a list of spectrum scale packages that are currently installed.  For example :

# rpm -qa | grep gpfs
gpfs.base-5.1.8-2.230821.114705.ppc64le
gpfs.adv-5.1.8-2.230821.114705.ppc64le
gpfs.license.dev-5.1.8-2.230821.114705.ppc64le
gpfs.gpl-5.1.8-2.230821.114705.noarch
gpfs.crypto-5.1.8-2.230821.114705.ppc64le
gpfs.compression-5.1.8-2.230821.114705.ppc64le
gpfs.msg.en_US-5.1.8-2.230821.114705.noarch
gpfs.gskit-8.0.55-19.1.ppc64le
gpfs.base-debuginfo-5.1.8-2.230821.114705.ppc64le
gpfs.docs-5.1.8-2.230821.114705.noarch
#

Remove the packages by running command `rpm -e package_name`.  For example :

# rpm -e gpfs.base-5.1.8-2.230821.114705.ppc64le gpfs.adv-5.1.8-2.230821.114705.ppc64le gpfs.license.dev-5.1.8-2.230821.114705.ppc64le gpf
s.gpl-5.1.8-2.230821.114705.noarch gpfs.crypto-5.1.8-2.230821.114705.ppc64le gpfs.compression-5.1.8-2.230821.114705.ppc64le gpfs.msg.en_US-5.1.8-2.230821.114705.noarch gpfs.gskit-8.0.55-19.1.ppc64le gpfs.base-debuginfo-5.1.8-2.230821.114705.ppc64le gpfs.docs-5.1.8-2.230821.114705.noarch

Verify that the packages are removed by running command `rpm -qa | grep gpfs`


On a node running Ubuntu, you could run `apt list | grep gpfs` to get a list of spectrum scale packages that are currently installed.  For example :

# apt list | grep gpfs
gpfs.adv/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.base-debuginfo/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.base/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.compression/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.crypto/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.docs/now 5.1.8-2.230821.114705 all [installed,local]
gpfs.gpl/now 5.1.8-2.230821.114705 all [installed,local]
gpfs.gskit/now 8.0.55-19.1 ppc64el [installed,local]
gpfs.license.dev/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.msg.en-us/now 5.1.8-2.230821.114705 all [installed,local]
#

Remove the packages by running command `apt remove package_name`
For example :

# apt remove gpfs.adv gpfs.base-debuginfo gpfs.base gpfs.compression gpfs.crypto gpfs.docs gpfs.gpl gpfs.gskit gpfs.license.dev gpfs.msg.en-us

Verify that the packages are removed by running command `apt list | grep gpfs`.  For example :

# apt list | grep gpfs
gpfs.adv/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.base/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.compression/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.crypto/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.docs/now 5.1.8-2.230821.114705 all [residual-config]
gpfs.gpl/now 5.1.8-2.230821.114705 all [residual-config]
gpfs.gskit/now 8.0.55-19.1 ppc64el [residual-config]
gpfs.license.dev/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.msg.en-us/now 5.1.8-2.230821.114705 all [residual-config]
#

Notice that the packages are not installed, but the configuration data still remains.  To completely remove spectrum scale from the node, the configuration data needs to be removed as well.  If this is not done, and later if this node needs to be added to another spectrum scale cluster, then that would be a problem, because of the existing old configuration.

7. Remove the /var/mmfs directory
# cd /var
# rm -rf mmfs/

8. Remove the /usr/lpp/mmfs directory
# cd /usr/lpp
# rm -rf mmfs/

9. Remove all files in the /var/adm/ras directory
# cd /var/adm/ras
# rm -rf *

10. Remove the /tmp/mmfs directory
# cd /tmp/
# rm -rf mmfs/

11. Reboot the node


This is how spectrum scale could be uninstalled from a node.  If you want to uninstall spectrum scale from a cluster, then uninstall spectrum scale from all nodes of the cluster.


Summary
Uninstalling spectrum scale from a node is not simply removing the packages that are installed.  There are some steps involved before removing the packages and after removing the packages.  If the node is part of a cluster, then appropriate care should be taken so that the NSDs in use and the other nodes in the cluster are not affected.