Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Thursday, January 29, 2026

Is a DoS attack similar to Slowloris possible on the layer 3 network protocols? Or a Network interface controller (NIC)?

I have written this article in collaboration with Krishna Bhandarge  

Background

Slowloris is a type of denial of service (DoS) attack which exploits the application‑layer (HTTP) connection handling weaknesses.  The Slowloris is a technique of capitalizing on how web servers keep open partial HTTP requests.

A web server allocates a thread for each incoming request.  Slowloris opens a connection to the target web server and sends partial HTTP headers.  Slowloris holds the connection open as long as possible by sending subsequent HTTP headers, adding to, but never completing the request.  By holding many such connections, Slowloris attempts to exhaust the server's thread / connection pool.  If the affected server keeps the connections open, the maximum concurrent connection pool would get filled completely, eventually denying additional connection attempts from the web clients.



Can the Slowloris attack concept be used to exploit the layer 3 network protocols?  Can the Slowloris attack concept be used to exploit a Network interface controller (NIC)?

A Network interface controller (NIC) operates at Layer 1 (physical layer) and Layer 2 (data link layer).  Slowloris relies entirely on connection handling weaknesses.  Layer 1, layer 2 and layer 3 devices, including NICs, do not maintain a per‑connection state.  Hence, this attack pattern is impossible at Layer 2 and layer 3.


Layer 3 DoS attacks are possible, but they are different from the Slowloris concept.  They are :

- ICMP floods

- Fragmentation floods

- IP packet storms

- Malformed IP packet attacks


These attacks attempt to saturate:

- NIC Rx queues (ring buffers)

- Host CPU network stack processing

- PCIe bandwidth

- Device driver's interrupt-handling capacity


None of these attacks resemble the Slowloris pattern because:

- They do not use partial requests

- They do not require keeping connections open

- They do not exploit HTTP behavior


If a Slowloris attack is happening on a web server, the web server's NIC has no way to detect the attack.  The NIC can not provide any protection.  Here is why.

A Network interface controller (NIC) does not do layer 7 filtering.  A NIC typically provides :

- hardware offloads for TCP

- virtualization support (SR-IOV)

- traffic shaping / QoS

- RDMA (RoCE/iWARP) capabilities

But none of these features prevent Slowloris, because Slowloris is not a bandwidth or packet-rate attack. Slowloris is a protocol resource exhaustion attack.


However, SmartNICs and DPUs are the exception.  Some vendors (NVIDIA BlueField, Intel IPU, AMD Pensando) manufacture SmartNICs or Data Processing Units which can offload Layer 7 logic.


On a side note, here are some ways to prevent a Slowloris attack on a web server.  Layer 7 filtering is typically performed by firewalls, proxies, DPI engines, or service meshes.

Mitigations for Slowloris attack are :

- SYN cookies / TCP stack hardening

- connection rate-limiting via OS firewall

- load balancers or reverse proxies (nginx, Varnish)


Tuesday, February 20, 2024

The fsck of IBM Storage Scale

What is fsck
Wikipedia says : The system utility fsck (file system consistency check) is a tool for checking the consistency of a file system in Unix and Unix-like operating systems, such as Linux, macOS, and FreeBSD.


A file system is a method of storing, organizing, and managing the data in the available storage medium.
File system consistency refers to the correctness and validity of a file system.  Faults in a file system are usually caused by power failures, hardware failures, or improper shutdown of the system.

All file systems have their own ways of storing the data structures stored on the storage medium.  So fsck works with the data structures on disk.  Thus fsck is always available for a specific file system.  In other words, every file system must have its own fsck program.

Before running fsck to check the health of a file system, the file system must be unmounted.


Why is fsck needed

Considering the reliability of the hardware available these days and the robustness of the software, it is rare for a file system to have data corruptions. However, if in case data corruption happends in a file system, it needs to be detected and repaired.

In case of a power failure, the server may not shutdown correctly.  In this case, the data in the memory may not get written to the disk.  This creates inconsistency.  Such inconsistencies, if not corrected, may create further trouble later.

In rare cases, disks have bad sectors. The disks with bad sectors need to be replaced. Then fsck needs to be run to ensure data integrity.

If the applications are reporting input/output errors when accessing or storing data, the file system may have inconsistencies and running fsck is needed in such cases.



What is IBM Storage Scale
IBM's clustered file system that provides parallel data access from multiple nodes is branded as Storage Scale.

To know more about Storage Scale, please visit https://www.ibm.com/docs/en/storage-scale/5.1.8?topic=overview-storage-scale

Here is a brief overview of IBM Storage Scale.
Storage Scale is a file system. But not an ordinary file system running locally on a single computer. Storage Scale runs on multiple computers. These computers together make a cluster. They are known as the nodes of the cluster.  Some of the nodes are arranged access to a storage that is present in the network. The storage is available to the nodes in the form of Network Shared Disks (NSDs). The available NSDs are used to create file system.  Customers use the file system to store and access data, via NFS, CIFS, or object protocols.

Storage Scale provides concurrent high-speed file access. Applications that access the data may be running on multiple systems and accessing the data in parallel.

A storage Scale cluster may consist of 1 to 56 nodes.  The nodes could be assigned the roles of quorum nodes and data nodes.  Rather than entering into the details of Storage Scale, let's focus on our agenda - fsck.



The fsck of IBM Storage Scale
As mentioned earlier, the fsck tool is always specific to a particular file system. For Storage Scale, IBM's engineers have written their own fsck program which is specific to Storage Scale.  It is named mmfsck.  All commands of Storage Scale begin with mm.  If you wonder why, mm stands for multi media.  This file system was developed 25 years ago.  In those days, having a big storage was a super luxury. Multi media was an emerging technology. And the storage required for multi media was supposed to be in huge quantity. As per the trend, all commands of the new file system were named with mm.

When IBM engineers developed this new file system, they developed the mmfsck tool as well. The file system has to be unmouted before running mmfsck on it. Are you thinking "this is a limitation"?  Well, although unmouting the file system for running fsck is a necessity, but some of us do think differently.  Why do I must have a downtime for running mmfsck? Why can't it be done while the file system is online and in use?  We took this thought forward and developed another variant of fsck which does not require the file system to be unmounted. It works while the file system is mounted and in use. We named it mmfsckx.  The x stands for eXtended.

So for Storage Scale now we have two variants of fsck.  mmfsck which requires the file system to be unmounted and mmfsckx which works while the file system is mounted and in use.  Typically we refer mmfsck as offline fsck and mmfsckx as online fsck.

Although the user may not be aware of this, the fsck is a separate program from the file system kernel code. Their intentions are also different. The fsck analyzes the file system’s metadata for the purpose of performing repairs. The kernel code manages the file system’s operations during normal usage.


Is Storage Scale the only file system that has online fsck?  Obviously not.  For example, see XFS Online Fsck Design and BeeGFS File System Check


What next?
Okay, you have a niche file system that allows to run fsck without unmounting, that means without a downtime involved. So can you improve what you already have?  Can you go beyond?  When this thought came to my mind, here are the subsequent thoughts that followed.

1. fsck for filesets
Say we have a huge file system that is being used in a multi-tenant cloud environment.  In some cases, we have a separate fileset for every customer, or "fileset-based multi-tenancy".  Filesets is a method of dividing the file system to have separate access and administrative operations.

Running online fsck on a huge file system would take a long time. It could take be a few days, depending on the amount of data present in the file system. And if a problem is reported by a particular customer, then we would know the fileset in question.  A fileset is a part of the file system that could be used separately.  So why not run online fsck on a single fileset or multiple filesets, rather than on the entire file system?  Of course, running online fsck on a fileset may not detect all issues. But it is a good start. If we could detect and repair all issues in the fileset, then we would avoid to run online fsck on the entire file system.  That would be a big bonus. If we could defect some of the issues in a fileset, I'd say that is still a battle half won. So running online fsck on filesets is a useful functionality we'd want to have.
To know more about filesets, please visit https://www.ibm.com/docs/en/storage-scale/5.1.8?topic=scale-filesets

2. Self healing filesystems
If some part of a disk goes bad, the data corruption may not be noticed immediately.  Detecting such data corruption much later may lead to unwanted consequences. So what if these situations could be avoided proactively?  Is there a way?  What if we have a program that identifies that the system is now idle and runs online fsck during the idle interval?  So the data corruptions are detected and repaired before they are noticed by the users of the data.  A self healing file system is the true masterpiece we'd want to have.

3. Self healing filesets
As mentioned earlier, running online fsck on a huge file system would take a long time, depending how much is the data stored in the file system.  So if we could have the online fsck to run on a single fileset or multiple filesets, then we could use that feature during the idle intervals of the Storage Scale cluster.  So by combining the two features together, we would have self healing filesets.  Is there a cloud that has already implemented this?  If you know, you tell me  :-)

4. Performance
When we think about performance, there are two aspects to consider :
(a) For a given file system, how much time is taken for running offline fsck vs online fsck
(b) What can be done to improve the performance of running online fsck
Let's consider these one at a time.

(a) For a given file system, how much time is taken for running offline fsck vs online fsck
When we run offline fsck, the file system is unmounted and so there is no IO workload.  So load on the worker nodes is low.  While running online fsck, the file system is mounted and IO workload is in progress.  So load on the worker nodes may be high, depending on the IO workload.  Considering these situations, time required for running offline fsck is usually much less than the time required running online fsck.  In theory, we all would agree to this.  The results obtained during functional testing indicate that the time difference is not much when the amount of data in the file system is of small quantity.  The more the amount of data in the file system, the more is the time difference.  When the file system contained enormous amount of data, say in quantities of petabytes, then the time difference is huge.  In one particular case the online fsck took 10 times more time than offline fsck.

(b) What could be done to improve the performance of running online fsck
By default, online fsck uses all available nodes of the cluster to do the work.  The total work is divided in portions. Each node does some part of the work.  The file system manager node manages the distribution of work.
Not all nodes of the cluster would have the same amount of memory and processing power.  Also IO workload may not be the same on all nodes of the cluster.  So if there are say 15 worker nodes, making 15 portions of the entire work and allocating one portion to each node may not be the efficient strategy.

Depending on the available memory, available processing power, and load on each node, certain amount of work could be allocated for each node which would be appropriate to distribute the work evenly amongst all available nodes.  Moreover, if the execution is taking huge amount of time, then the amount IO workload on the nodes may vary at later times.  So the original calculation of evenly distributing the work may not remain to be the most efficient at a later time.  In such cases, a recalculation to evenly distribute the remaining work between the nodes would be beneficial.

Another simpler strategy is also possible. Not all nodes would finish their portion of the work at the same time.  Some may finish earlier than others, depending on the IO workload. The nodes which complete their portion of the work may be allocated a smaller portion of the remaining work. This redistribution of work would help to complete the entire work in lesser time.

There could be more ways to improve performance of online fsck that I could not list here.  Some food for thought for the reader.



References
1. https://en.wikipedia.org/wiki/Fsck
2. https://lwn.net/Articles/248180/
3. https://www.adminschoice.com/repairing-unix-file-system-fsck
4. https://linux.die.net/man/8/fsck
5. https://www.ibm.com/docs/en/aix/7.3?topic=f-fsck-command
6. https://www.ibm.com/docs/en/storage-scale/5.1.9?topic=reference-mmfsckx-command

Wednesday, August 30, 2023

How to uninstall IBM Spectrum Scale

There are situations when IBM Spectrum Scale needs to be uninstalled.  Here a step by step guide for doing this.

What is IBM Spectrum Scale
IBM's clustered file system that provides parallel data access from multiple nodes was branded as Spectrum Scale.  It is now re-branded as IBM Storage Scale.

To know more about IBM Spectrum Scale, please visit https://www.ibm.com/docs/en/storage-scale/5.1.8?topic=overview-storage-scale

IBM Spectrum Scale is a file system. But not an ordinary file system running locally on a single computer. Spectrum Scale runs on multiple computers. These computers together make a cluster. They are known as the nodes of the cluster.  Some of the nodes are arranged access to a storage that is present in the network. The storage is available to the nodes in the form of Network Shared Disks (NSDs). The available NSDs are used to create file system.  Customers use the file system to store and access data, via NFS, CIFS, or object protocols.

IBM Spectrum Scale provides concurrent high-speed file access. Applications that access the data may be running on multiple systems and accessing the data in parallel.

A Spectrum Scale cluster may consist of 1 to 56 nodes.  The nodes could be assigned the roles of quorum nodes and data nodes.

Before we begin
A spectrum scale cluster consists of one or more nodes.  At every node of the cluster, the spectrum scale packages are installed.  Apart from the packages, some files and directories are present which are required for the functioning of the node.  This procedure is of uninstalling Spectrum Scale from a single node.

Pre-checks
Before uninstalling spectrum scale from a node, please check :
(a) Is the node part of a spectrum scale cluster?  Run command mmlscluster to check this.
(b) Does the node have NSDs?  Are the NSDs local to the node or are they shared NSDs?  Are any of the NSDs being used?  Run command mmlsnsd to check these.
(c) Are any spectrum scale file systems mounted at this node?  Run command mmlsmount to check this.

Uninstallation Procedure

1. If the node has any file systems mounted, then all file systems must be unmounted from this node.  Unmount all file systems from the node by running command mmumount all
Verify that the node does not have any spectrum scale file system mounted by running command mmlsmount all

2. If the node has shared NSDs, then all shared NSDs should be removed.  In this case, please think for a while "why are we really uninstalling spectrum scale from this node?  Is this really necessary?"  If answer is affirmative, then please proceed to remove all the shared NSDs.  For doing so, the shared NSDs must be in unused state.  Please check if any of the shared NSDs are being used in any spectrum scale file system(s).  If yes, then remove the NSDs from the file system(s) using command mmdeldisk
Verify that the NSDs in question are in unused state by running command mmlsnsd
Delete the NSDs by running command mmdelnsd
Verify that the NSDs in question are deleted by running command mmlsnsd

3. If the node has local NSDs, then all local NSDs should be removed.  Please check if any of the local NSDs are being used in any spectrum scale file system(s).  If yes, then remove the NSDs from the file system(s) using command mmdeldisk
Verify that the NSDs in question are in unused state by running command mmlsnsd
Delete the NSDs by running command mmdelnsd
Verify that the NSDs in question are deleted by running command mmlsnsd

4. If the node is part of a spectrum scale cluster, then it should be removed from the cluster.  To do so, login to any other node in the cluster, and run command mmdelnode
Verify that the node is not part of the cluster by running command mmlscluster

5. If the spectrum scale software is running at the node, stop it by running command mmshutdown

6. Remove all spectrum scale packages that are currently installed.
Some of the spectrum scale packages have dependencies, and can not be removed individually.  So all spectrum scale packages should be removed together using a single command.

On a node running Red Hat Enterprise Linux, you could run `rpm -qa | grep gpfs` to get a list of spectrum scale packages that are currently installed.  For example :

# rpm -qa | grep gpfs
gpfs.base-5.1.8-2.230821.114705.ppc64le
gpfs.adv-5.1.8-2.230821.114705.ppc64le
gpfs.license.dev-5.1.8-2.230821.114705.ppc64le
gpfs.gpl-5.1.8-2.230821.114705.noarch
gpfs.crypto-5.1.8-2.230821.114705.ppc64le
gpfs.compression-5.1.8-2.230821.114705.ppc64le
gpfs.msg.en_US-5.1.8-2.230821.114705.noarch
gpfs.gskit-8.0.55-19.1.ppc64le
gpfs.base-debuginfo-5.1.8-2.230821.114705.ppc64le
gpfs.docs-5.1.8-2.230821.114705.noarch
#


Remove the packages by running command `rpm -e package_name`.  For example :

# rpm -e gpfs.base-5.1.8-2.230821.114705.ppc64le gpfs.adv-5.1.8-2.230821.114705.ppc64le gpfs.license.dev-5.1.8-2.230821.114705.ppc64le gpf
s.gpl-5.1.8-2.230821.114705.noarch gpfs.crypto-5.1.8-2.230821.114705.ppc64le gpfs.compression-5.1.8-2.230821.114705.ppc64le gpfs.msg.en_US-5.1.8-2.230821.114705.noarch gpfs.gskit-8.0.55-19.1.ppc64le gpfs.base-debuginfo-5.1.8-2.230821.114705.ppc64le gpfs.docs-5.1.8-2.230821.114705.noarch


Verify that the packages are removed by running command `rpm -qa | grep gpfs`


On a node running Ubuntu, you could run `apt list | grep gpfs` to get a list of spectrum scale packages that are currently installed.  For example :

# apt list | grep gpfs
gpfs.adv/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.base-debuginfo/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.base/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.compression/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.crypto/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.docs/now 5.1.8-2.230821.114705 all [installed,local]
gpfs.gpl/now 5.1.8-2.230821.114705 all [installed,local]
gpfs.gskit/now 8.0.55-19.1 ppc64el [installed,local]
gpfs.license.dev/now 5.1.8-2.230821.114705 ppc64el [installed,local]
gpfs.msg.en-us/now 5.1.8-2.230821.114705 all [installed,local]
#


Remove the packages by running command `apt remove package_name`
For example :

# apt remove gpfs.adv gpfs.base-debuginfo gpfs.base gpfs.compression gpfs.crypto gpfs.docs gpfs.gpl gpfs.gskit gpfs.license.dev gpfs.msg.en-us


Verify that the packages are removed by running command `apt list | grep gpfs`.  For example :

# apt list | grep gpfs
gpfs.adv/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.base/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.compression/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.crypto/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.docs/now 5.1.8-2.230821.114705 all [residual-config]
gpfs.gpl/now 5.1.8-2.230821.114705 all [residual-config]
gpfs.gskit/now 8.0.55-19.1 ppc64el [residual-config]
gpfs.license.dev/now 5.1.8-2.230821.114705 ppc64el [residual-config]
gpfs.msg.en-us/now 5.1.8-2.230821.114705 all [residual-config]
#


Notice that the packages are not installed, but the configuration data still remains.  To completely remove spectrum scale from the node, the configuration data needs to be removed as well.  If this is not done, and later if this node needs to be added to another spectrum scale cluster, then that would be a problem, because of the existing old configuration.

7. Remove the /var/mmfs directory
# cd /var
# rm -rf mmfs/


8. Remove the /usr/lpp/mmfs directory
# cd /usr/lpp
# rm -rf mmfs/


9. Remove all files in the /var/adm/ras directory
# cd /var/adm/ras
# rm -rf *

10. Remove the /tmp/mmfs directory
# cd /tmp/
# rm -rf mmfs/


11. Reboot the node


This is how spectrum scale could be uninstalled from a node.  If you want to uninstall spectrum scale from a cluster, then uninstall spectrum scale from all nodes of the cluster.


Summary
Uninstalling spectrum scale from a node is not simply removing the packages that are installed.  There are some steps involved before removing the packages and after removing the packages.  If the node is part of a cluster, then appropriate care should be taken so that the NSDs in use and the other nodes in the cluster are not affected.

Thursday, December 17, 2020

UTF-8 : Where Ινδία is bigger than India

A peculiar problem was once reported by a customer of our data storage product.
They were copying files from a Netapp solution to their newly setup IBM Storwize V7000 Unified system.
Many files were having names with Greek characters in them.  Some of those had really long names.
Files whose names were Greek and longer than 125 characters failed to copy.
Being the Linux flag bearer, the issue came my way.

My investigation involved checking all the possibilities of where the issue could be.  I wanted to check all the possibilities.  Robocopy, the tool used for copying the files.  Samba, the protocol used for copying the files.  The locale being used at the Unified system.  Since we use RHEL in IBM Storwize V7000 Unified system, I had opened a dialogue with Red Hat as well.

Investigation converged to the limit of file names as defined by the operating system in use, Linux, and file system in use, GPFS.  255 bytes was the limit in case of both.  In case of Linux, this came from NAME_MAX macro in limits.h header.

     #define NAME_MAX         255    /* # chars in a file name */

RHEL and GPFS both were using UTF-8 encoding for file names.  UTF-8 requires 2 bytes for storing one Greek character.  Hence files whose names were longer than 125 Greek characters were failing to copy.

 

Someday I want to do this experiment.  Update the NAME_MAX macro in limits.h and recompile, so that files having longer names could see light of the day.

What is noquery in NTP configuration file

Sometime last year, a security vulnerability was reported at a system that acts as NTP server.  After investigation, I found that the system in question was not having noquery in its NTP configuration file /etc/ntp.conf and this resulted in the security vulnerability.  This was probably found by doing a vulnerability scan of the system using Nessus or OpenVAS or something similar.

As a mitigation, I suggested to add noquery in the NTP configuration file and then restart NTP service, so that update to the NTP configuration file comes into effect.


What is this noquery and what is the necessity to have it?

In the NTP configuration file, noquery is a flag that could be used along with restrict command.  When this flag is added to the restrict command, all NTP mode 6 and 7 packets are ignored from the specified source.  In other words, ntpq and ntpdc queries are denied from the specified source.
In absence of noquery flag, that is, if ntpq and ntpdc queries are allowed, then NTP server status information (such as OS and ntpd version) could be identified by doing a scan of the system using Nessus.  This open channel that could be used for revealing system information is reported as a security vulnerability.


So, whether to have noquery flag or not depends on the decision - do we want to allow others to see our server status information?
If answer is yes, then do not have noquery flag along with the restrict command.
If answer is no, then do have the noquery flag.

noquery flag does not affect time service of the NTP server.


Please be aware, the UDP nature of NTP makes NTP servers prone to be abused in case of reflection and amplification attacks.

Friday, July 17, 2020

Python code to generate CAPTCHA

Today I wrote python code to generate CAPTCHA images.  Before we jump into the code, let's recap what a CAPTCHA is.  CAPTCHA is an acronym for Completely Automated Public Turing test to tell Computers and Humans Apart.  This is one type of a challenge–response test that is used in computing to determine whether the user is a human or not.  This is implemented at places where we want only humans to proceed ahead, and stop others (bots et al.)  How effective is the method of using CAPTCHA, and can it be circumvented is not a topic I take here.  Right now, let us get to writing python code for generating CAPTCHA images.

Well, a python library named captcha is available that you could use to generate audio and image CAPTCHAs.  You could install this library and use it to write your python code for generating audio and image CAPTCHAs.  If you want to generate image CAPTCHAs from python code yourself, without using the python's captcha library, do read on.  Wait.  Why would someone write their own code, while a standard library is available?  This is an obvious question.  Well, in most cases we should use the standard library.  Only in some exceptional cases we should write our own code.  In cases when standard library is not available, or is not allowed to be installed.  And in cases when we want some customization in the implementation.  What we have here could be one such case, where we want to customize how the image CAPTCHAs are being generated.

Here is the python code I wrote to generate image CAPTCHAs.

from PIL import Image
from PIL import ImageDraw
from PIL import ImageFont
import string
import random


def draw_text(img, x_given, y_given, text_given, font_given, fill_given):
    d = ImageDraw.Draw(img)
    d.text((x_given, y_given), text_given, font=font_given, fill=fill_given)


def char_selecting(size=6):
    characters = "abdefghijklmnqrtuy123456789ABDEFGHIJKLMNQRTUY"
    # Exclude letters that look similar in capital and small cases
    # Why?
    # We are randomly choosing a font size.  Bigger to smaller.
    # So can't tell whether you are looking at C or c in that situation
    # Exclude zero because we want to avoid confusion between 0 and O and o
    selection = ""
    for x in range(0, size):
        char = characters[random.randint(0, len(characters) - 1)]
        selection = selection + char
    return selection


image_x_size = 200
image_y_size = 100
img = Image.new('RGB', (image_x_size, image_y_size), color = (250, 250, 250))

word = char_selecting(6)    # We want 6 characters in our CAPTCHA
print word

x_pos = 10    # Position of the first character in the image

for char in (list(word)):
    print char

    font_size = random.randint(16, 50)
    # Font size smaller than 16 is too small
    # Font size bigger than 50 is too big
    font_selected = ImageFont.truetype('/usr/share/fonts/gnu-free/FreeMono.ttf', font_size)

    fill_selected = (random.randint(0, 200), random.randint(0, 200), random.randint(0, 200), random.randint(0, 255))
    # Chose a random colour

    y_random = random.randint(0, 30)
    # All characters should not be in a horizontal line.
    # So shift position of each character randomly

    draw_text(img, x_pos, y_random, char, font_selected, fill_selected)

    x_pos = x_pos + 30
    # Position of the next character
    # We are randomly choosing a font size for each character
    # So position of the next character should not be chosen randomly

mid_x = image_x_size / 2
mid_y = image_y_size / 2

first_half_x = random.randint(1, mid_x - 1)
first_half_y = random.randint(1, mid_y - 1)

second_half_x = mid_x + random.randint(1, mid_x - 1)
second_half_y = mid_y + random.randint(1, mid_y - 1)

print "Going to draw a line from", first_half_x, ",", first_half_y, "to", second_half_x, ",", second_half_y
d = ImageDraw.Draw(img)
d.line((first_half_x, first_half_y, second_half_x, second_half_y), 10)

img.save('captcha.png')   # Save the image as a file on the disk


Here are some of the CAPTCHA images that I generated from this code.








Now let us see this code in detail.

from PIL import Image
We are using the Image module from the PIL library.

from PIL import ImageDraw
We are using the ImageDraw module from the PIL library.

from PIL import ImageFont
We are using the ImageFont module from the PIL library.


def draw_text(img, x_given, y_given, text_given, font_given, fill_given):
    d = ImageDraw.Draw(img)
    d.text((x_given, y_given), text_given, font=font_given, fill=fill_given)

I have written this function to draw the given text at the specified place in the given image.  The font and color to be used must be specified.  So why did I write a separate function for this?  Why not simply do this in the main code?  Well, you could do that.  Here I wrote a generic function, and used it.  So that tomorrow if I have to extend this code, a generic function would be useful.


def char_selecting(size=6):
    characters = "abdefghijklmnqrtuy123456789ABDEFGHIJKLMNQRTUY"

    # Exclude letters that look similar in capital and small cases
    # Why?
    # We are randomly choosing a font size.  Bigger to smaller.
    # So can't tell whether you are looking at C or c in that situation
    # Exclude zero because we want to avoid confusion between 0 and O and o
    selection = ""
    for x in range(0, size):
        char = characters[random.randint(0, len(characters) - 1)]
        selection = selection + char
    return selection

This is the function where the magic happens.  The magic of choosing a set of characters that we want to show in the CAPTCHA image.  Caller may choose the length of the string, by passing a numeric value.  Else, a default of 6 is taken.  We randomly choose the characters and form a string using them.  While choosing the characters in the string, I have not considered all of the upper case letters, small case letters, and numbers.  Why?  Allow me to explain please.  By looking at the CAPTCHA images that I have generated, you must have noticed that font size of each character varies.  I did that deliberately.  To avoid the text being recognized by an OCR software.  But when we have a mix of small and big font sizes, how could you tell the difference between c and C, or between z and Z?  Same for P, S, V, W, and X.  O is even difficult, because a 0 (zero) also looks similar.  So I chose not to use these characters.  So that we don't create a confusion for humans.

image_x_size = 200
image_y_size = 100
img = Image.new('RGB', (image_x_size, image_y_size), color = (250, 250, 250))
This is the start of the main code.  I have chosen image size of 200 x 100 pixels.  I have chosen background color which is almost white, but not exactly white.

word = char_selecting(6)    # We want 6 characters in our CAPTCHA
print word
A call to function char_selecting gets us a string of length 6, made up of randomly selected characters.

x_pos = 10    # Position of the first character in the image
In the loop that follows, you will observe that I keep on increasing this value, for the subsequent characters in the image.

for char in (list(word)):
    print char
From the string that we have prepared, here we take one character at a time, and work on it.

    font_size = random.randint(16, 50)
    # Font size smaller than 16 is too small
    # Font size bigger than 50 is too big
    font_selected = ImageFont.truetype('/usr/share/fonts/gnu-free/FreeMono.ttf', font_size)
As mentioned earlier, I have deliberately chosen a random font size for each character.  So that the resulting text could not be recognized by an OCR software.  Here I am using the same font for all characters.  If you want, you could obtain a list of fonts available in your system, and randomly choose a different font for each character.  That implementation I leave up to you.  Some weird fonts may be present in the system, which would render some of the characters beyond recognition by humans.  Considering this possibility, I did not do that.  I found a font in which the characters are rendered in a way that is easy for recognition by humans.  And I am sticking to that font.

    fill_selected = (random.randint(0, 200), random.randint(0, 200), random.randint(0, 200), random.randint(0, 255))
    # Chose a random colour
We are deliberately choosing a different color for each character.  So that the resulting text could not be recognized by an OCR software.

    y_random = random.randint(0, 30)
    # All characters should not be in a horizontal line.
    # So shift position of each character randomly
In my opinion, this is an important trick.  If all characters appear in an horizontal line, there is a possibility of them being recognized by an OCR software.  To minimize this possibility, we shift position of each character.  In the range of 30 pixels.

    draw_text(img, x_pos, y_random, char, font_selected, fill_selected)

    x_pos = x_pos + 30
    # Position of the next character
    # We are randomly choosing a font size for each character
    # So position of the next character should not be chosen randomly
Function draw_text is where we draw the character in the image.  Now let's see what precautions we have taken so that the text could not be recognized by an OCR software.
1. Color of each character is different.
2. Font size of each character is different.
3. Characters are not in a horizontal line.  They are slightly off-positioned.  Well enough to fool the OCR software.  But not too out-of-position.  So that humans don't have a difficulty in recognizing the sequence.

Now let's add something more in the image, which will prevent the OCR software from recognizing the text.  Let's draw a line.

mid_x = image_x_size / 2
mid_y = image_y_size / 2

first_half_x = random.randint(1, mid_x - 1)
first_half_y = random.randint(1, mid_y - 1)

second_half_x = mid_x + random.randint(1, mid_x - 1)
second_half_y = mid_y + random.randint(1, mid_y - 1)

print "Going to draw a line from", first_half_x, ",", first_half_y, "to", second_half_x, ",", second_half_y
d = ImageDraw.Draw(img)
d.line((first_half_x, first_half_y, second_half_x, second_half_y), 10)
First we decide the position of the line in the image.  We consider the image in two parts, say left part and right part.  In the left part, we chose a random position, one end point of the line.  In the right part, we chose a random position, the other end point of the line.  And we draw the line, using the end points that we have chosen.  Well, you could argue that a better method could have been used to draw the line.  Yes, I agree.  I leave this part up to you.  To implement a better method to draw a line.  Or may be some other geometrical object.  Or may be two geometrical objects.  Basically we want to have something that would prevent the OCR software from recognizing the text.

img.save('captcha.png')   # Save the image as a file on the disk
Finally we save the image as a file named captcha.png
Is this code perfect?  I don't claim it to be.  Does this code generate CAPCHAs that are easily recognizable by humans, but could never be recognized by OCR software?  I don't claim that either.  If you try hard enough, you might find an OCR software that recognizes the text.  And then what?  Well, if I'd get to know that, may be I'd change my code so that the flaw is removed.  After all, this is a game of cat and mouse that we all are playing, right?

And we did not generate audio CAPCHA yet.  So let's do that.

from gtts import gTTS

word = "TAAaGr"

text = ""
for char in (list(word)):
    text =  text + char + " "

speech = gTTS(text = text, lang = 'en', slow = True)

speech.save("captcha.mp3")

We are using the gtts library for this purpose.  For each character to be clearly audible, we are separating them by spaces.  You'd notice there is no difference in which small case letters and upper case letters are spoken.  So, if we have to implement image and audio CAPTCHA together, then along with numbers we should use only lower case letters or only upper case letters.

What if we could make the challenge–response test a little more difficult to be solved by a non-human, while keeping it easy for humans.  One way to do this is to show two numbers, ask what is their addition, and check the response.  Let's see python code for this.

from PIL import Image
from PIL import ImageDraw
from PIL import ImageFont
import random



def draw_text(img, x_given, y_given, text_given, font_given, fill_given):
    d = ImageDraw.Draw(img)
    d.text((x_given, y_given), text_given, font=font_given, fill=fill_given)


image_x_size = 150
image_y_size = 70
img = Image.new('RGB', (image_x_size, image_y_size), color = (250, 250, 250))

num1 = random.randint(1,9)
num2 = random.randint(1,9)
total = num1 + num2
print num1, "+", num2, "=", total

font_size = random.randint(16, 50)
# Font size smaller than 16 is too small
# Font size bigger than 50 is too big

font_selected = ImageFont.truetype('/usr/share/fonts/gnu-free/FreeMono.ttf', font_size)

fill_selected = (random.randint(0, 200), random.randint(0, 200), random.randint(0, 200), random.randint(0, 255))
# Chose a random colour

x_pos = 10    # Position of the first character in the image

y_random = random.randint(0, 20)
# All characters should not be in a horizontal line.
# So shift position of each character randomly

draw_text(img, x_pos, y_random, str(num1), font_selected, fill_selected)

fill_selected = (random.randint(0, 200), random.randint(0, 200), random.randint(0, 200), random.randint(0, 255))
x_pos = x_pos + 30
y_random = random.randint(0, 20)
draw_text(img, x_pos, y_random, "+", font_selected, fill_selected)

fill_selected = (random.randint(0, 200), random.randint(0, 200), random.randint(0, 200), random.randint(0, 255))
x_pos = x_pos + 30
y_random = random.randint(0, 20)
draw_text(img, x_pos, y_random, str(num2), font_selected, fill_selected)

fill_selected = (random.randint(0, 200), random.randint(0, 200), random.randint(0, 200), random.randint(0, 255))
x_pos = x_pos + 30
y_random = random.randint(0, 20)
draw_text(img, x_pos, y_random, "=", font_selected, fill_selected)

img.save('captcha_num.png')   # Save the image as a file on the disk

Here are some of the CAPTCHA images that I generated from this.




For simplicity, I chose single-digit integers.  You could choose bigger numbers, if that suits you.

Tuesday, May 12, 2020

Python script to find broken links in a web page

Today I was told to find broken links in a web page.  Someone could have done this manually, by opening the web page in a web browser, and checking each link.  That is not the right way, though.  I wrote a python script for this job.  Here it is.

import sys
import requests
from bs4 import BeautifulSoup
from urlparse import urlparse
from urlparse import urljoin


link_count = 0
searched_links = []
broken_links = []

def getLinksFromHTML(html):
    def getLink(el):
        return el["href"]
    return list(map(getLink, BeautifulSoup(html, features="html.parser").select("a[href]")))



def check_links(domainToSearch, URL, parentURL, depth):
    if (depth == 2):
    # We do not want to search all links recursively.
        return
    if (not (URL in searched_links)) and (not URL.startswith("mailto:")) and (not ("javascript:" in URL)) and (not URL.endswith(".png")) and (not URL.endswith(".jpg")) and (not URL.endswith(".jpeg")):
        try:
            requestObj = requests.get(URL);
            searched_links.append(URL)
            global link_count
            link_count = link_count + 1
            if(requestObj.status_code == 404):
                broken_links.append("Broken: link " + URL + " from " + parentURL)
                print(broken_links[-1])
            else:
                print("Not broken: link " + URL + " from " + parentURL)
                if urlparse(URL).netloc == domainToSearch:
                    for link in getLinksFromHTML(requestObj.text):
                        check_links(domainToSearch, urljoin(URL, link), URL, (int(depth)+1))
                        pass
        except Exception as e:
            print("ERROR: " + str(e));
            searched_links.append(domainToSearch)



# Written by y.sawant @ gmail.com
if (len(sys.argv) != 2):
    print "Please provide a URL.\n"
    sys.exit()
depth = 0
check_links(urlparse(sys.argv[1]).netloc, sys.argv[1], "", depth)

print("\n--- Checked " + str(link_count) + " links ---\n")

if not broken_links:
    print("No broken links are found.")
else:
    print("Broken links are listed below:")
    for link in broken_links:
        print ("\t" + link)


I know this script is not perfect.  But it gets the job done.  If you are seeking perfection in this script, I leave that part to you.

The three external python libraries that I used in this script are :
1. requests
2. BeautifulSoup
3. urlparse

If you want to check if these external Python libraries are available or not, here is how to.  At the Linux command prompt, type python and hit enter.  At the Python prompt, type import followed by the name of the library and press enter.  For example, to check whether requests library is available or not, type import requests and press enter.  Here is an example.

    # python
    Python 2.7.5 (default, Jun 11 2019, 14:33:56)
    [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import requests
    >>>

If you do not see any error, you have that Python library available.  If you see an error, you need to install that Python library.  You could use pip for installing the required Python libraries.

    # pip install requests

I used my script to check if https://www.google.co.in contains any broken links.  Here it is.

# python find_broken_links.py "https://www.google.co.in"
Not broken: link https://www.google.co.in from
Not broken: link https://www.google.co.in/imghp?hl=en&tab=wi from https://www.google.co.in
Not broken: link https://maps.google.co.in/maps?hl=en&tab=wl from https://www.google.co.in
Not broken: link https://play.google.com/?hl=en&tab=w8 from https://www.google.co.in
Not broken: link https://www.youtube.com/?gl=IN&tab=w1 from https://www.google.co.in
Not broken: link https://news.google.co.in/nwshp?hl=en&tab=wn from https://www.google.co.in
Not broken: link https://mail.google.com/mail/?tab=wm from https://www.google.co.in
Not broken: link https://drive.google.com/?tab=wo from https://www.google.co.in
Not broken: link https://www.google.co.in/intl/en/about/products?tab=wh from https://www.google.co.in
Not broken: link http://www.google.co.in/history/optout?hl=en from https://www.google.co.in
Not broken: link https://www.google.co.in/preferences?hl=en from https://www.google.co.in
Not broken: link https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=https://www.google.co.in/ from https://www.google.co.in
Not broken: link https://www.google.co.in/advanced_search?hl=en-IN&authuser=0 from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=hi&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCAU from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=bn&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCAY from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=te&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCAc from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=mr&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCAg from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=ta&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCAk from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=gu&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCAo from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=kn&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCAs from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=ml&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCAw from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefs?sig=0_V3GwPGYEtv57hWTx9gHH5SRjSjo%3D&hl=pa&source=homepage&sa=X&ved=0ahUKEwijhYvltK7pAhWPxTgGHfF2AZ8Q2ZgBCA0 from https://www.google.co.in
Not broken: link https://www.google.co.in/intl/en/ads/ from https://www.google.co.in
Not broken: link http://www.google.co.in/services/ from https://www.google.co.in
Not broken: link https://www.google.co.in/intl/en/about.html from https://www.google.co.in
Not broken: link https://www.google.co.in/setprefdomain?prefdom=US&sig=K_8Rgf8LawsO1reHRLJSf5TzwNn9E%3D from https://www.google.co.in
Not broken: link https://www.google.co.in/intl/en/policies/privacy/ from https://www.google.co.in
Not broken: link https://www.google.co.in/intl/en/policies/terms/ from https://www.google.co.in

--- Checked 28 links ---

No broken links are found.
#

Thursday, November 28, 2019

A Python based web scrapper for blogs written at blogspot.com

If you want to programmatically extract data from a website, what you'd do is known as web scraping.  Some websites such as Facebook provide API for accessing data in their website.  The API provided by Facebook is Graph API.  Only a handful of websites provide such API.  For programmatically extracting data from all other websites, what we'd have to do is web scraping.
Let's see this Python based web scrapper I have prepared for extracting data from blogs written at blogspot.com

Web scraping that we are going to do is :
1. Send HTTP (or HTTPS) request to the web server.  The web server would respond by returning HTML content of the URL.
2. From the HTML content that is received, parse the data so that we obtain what we were looking for.

Let us write a Python program for this purpose.  We will need a Linux system where we will write and execute our Python program.  Also we will need three external Python libraries, listed below.  Before we start writing our python program, let us check if we have those external Python libraries available in the system.  And if not, let us install those.
The three external python libraries that we are going to use are :
1. requests
2. BeautifulSoup
3. html5lib

Let us check if these external Python libraries are available or not.  At the Linux command prompt, type python and hit enter.  At the Python prompt, type import requests and press enter.  Here is an example.
# python
Python 2.7.5 (default, Jun 11 2019, 14:33:56)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>
If you do not see any error, you have that Python library available.  If you see an error, you need to install that Python library.  You could use pip for installing the required Python libraries.

# pip install requests
When we have the required external Python libraries available, it is time to write our Python program.

import requests
from bs4 import BeautifulSoup
import re

blog_url = "https://ysawant.blogspot.com/"
r_blog = requests.get(blog_url)

blog_soup = BeautifulSoup(r_blog.content, 'html5lib')
Now let us obtain a list of URLs mentioned in this webpage.  And also a count of how many URLs are mentioned in this webpage.
The links are mentioned using the anchor tag of HTML.  Here is an example from my blog.
< a class='timestamp-link' href='https://ysawant.blogspot.com/2019/10/how-to-disable-weak-arcfour-cipher-in.html' rel='bookmark' title='permanent link'>< abbr class='published' title='2019-10-30T20:14:00+05:30'>8:14 PM< /abbr>< /a>
print 'A list of _all_ links on this webpage :'
link_count = 0
for link in blog_soup.find_all('a'):
    href = link.get('href')
    if href == None:
        # Empty.  So skipping.
        continue
    found = re.search("^http", href)
    if found:
        print found.string
        link_count = link_count + 1
print '\nTotal', link_count, 'links found.\n'

You'd notice that some URLs are listed more than once.  So a possible improvement in our code is to remove duplicate URLs.  For this, we'll have to store all URLs in an array.  Then remove the duplicate entries in that array.  Then print the elements of the array.  I leave this as an exercise to be done by the readers.

Next, let us obtain the dates on which articles were published in this blog.  The blogs written at blogspot.com have this detail in the < h2 class='date-header'> HTML tag.  Here is an example from my blog.
< h2 class='date-header'>< span>Wednesday, October 30, 2019< /span>< /h2>

How did I get to know this?  By looking at the page source.  We, the programmers, have to decide exactly what data to grab from the whole lot of HTML content that is available.  For this, we have to closely look at the HTML content of the webpage.
print '\nArticles in this webpage were written on these dates :'
all_dates = blog_soup.find_all('h2', attrs = {'class':'date-header'})
for a_date in all_dates:
    print a_date.text

Next, let us obtain the titles of the articles that are published in this blog.  In blogs written at blogspot.com website, the titles of the articles are present in the the < h3 class='post-title entry-title'> HTML tag.  Here is an example from my blog.
< h3 class='post-title entry-title'>
< a href='https://ysawant.blogspot.com/2019/10/how-to-disable-weak-arcfour-cipher-in.html'>How to disable the weak arcfour cipher in Linux< /a>
< /h3>

And how did I get to know this?  By looking at the page source.
print '\nTitles of the Articles in this webpage :'
all_titles = blog_soup.find_all('h3', attrs = {'class':'post-title entry-title'})
for a_title in all_titles:
    print a_title.text
I checked this Python program with few of the blogs I know at blogspot.com
blog_url = "http://sudhirdeore29.blogspot.com/"
blog_url = "http://bhadkamkar.blogspot.com/"
blog_url = "https://navinraomhatre.blogspot.com/"
blog_url = "https://pakhandkhandinee.blogspot.com/"

We have our own Python based web scrapper, albeit a simple one.  Please note, this web scrapper would work with blogs written at blogspot.com only.  For other websites, we'll have to write web scrappers according to the HTML content of each website.


Let us see another web scrapper.  The wikinews.org website is full of news from around the world. The Main page of this website lists latest news in short one-liners.  Here is a web scrapper for obtaining the short one-liner latest news.
import requests
from bs4 import BeautifulSoup
from datetime import date

wikinews_url = "https://en.wikinews.org/wiki/Main_Page"

r_wikinews = requests.get(wikinews_url)

wikinews_soup = BeautifulSoup(r_wikinews.content, 'html5lib')
# print(wikinews_soup.prettify())

today = date.today()
print "Latest news on", today
latest_news = wikinews_soup.find_all('div', attrs = {'class':'latest_news_text'})
for news in latest_news:
    if news.text:
        print(news.text)
For writing this web scrapper, I looked at the page source of the Main page at wikiews.org and identified the HTML content that needs to be fetched.
Our desired content is included in < div class="latest_news_text" id="MainPage_latest_news_text">

You'd notice that the news at wikinews.org are not updated daily.  And in my opinion, they are not much useful as well.  I find that the Main page at wikipedia.org has some brief news that are updated regularly.  So here is another Python program to grab the news from the Main page at wikipedia.org
Here it is.
import requests
from bs4 import BeautifulSoup
from datetime import date
import re

wikipedia_url = "https://en.wikipedia.org/wiki/Main_Page"

r_wikipedia = requests.get(wikipedia_url)

wikipedia_soup = BeautifulSoup(r_wikipedia.content, 'html5lib')
# print(wikipedia_soup.prettify())

today = date.today()
print "In the news, ", today
in_the_news = wikipedia_soup.find_all('div', attrs = {'id':'mp-itn'})
count = 0
for news in in_the_news:
    for line in news.find_all('ul'):
        if count == 0:
            print(line.text)
            count = count + 1

I looked at page source of the Main page at wikipedia.org and found the HTML content that is useful in this case.

< div id="mp-itn" style="padding:0.1em 0.6em;">< div role="figure" class="itn-img" style="float: right; margin-left: 0.5em;">

Wednesday, October 30, 2019

How to disable the weak arcfour cipher in Linux

A cipher is an algorithm for performing encryption or decryption.
 
Routine nessus scan of one of my lab machines revealed that the weak RC4 (arcfour) cipher was available there.  Multiple vulnerabilities have been discovered in this cipher.  This insecure cipher should not be used.  So I disabled it.  Here's how.

The ciphers that are available are mentioned in the SSH configuration file /etc/ssh/sshd_config
Open the /etc/ssh/sshd_config file using an editor such as vi, and check for a line that begins with Ciphers.  A comma separated list of ciphers should be present after the Ciphers keyword.  In this list, check for arcfour,arcfour128,arcfour256
If any of these are found, remove them and save the file.
In order for the change to come in effect, ssh daemon needs to be restarted.  `service ssh restart` restarts the ssh daemon.

And here's how to verify that the change you made has actually removed arcfour cipher from being used.  From another machine in your network, you could use the ssh command along with the -c option.  The -c option allows us to specify which cipher to use.

# ssh -p 22  user@your.ip.address.here  -c arcfour

If arcfour cipher is not available in your machine, you should see message : no matching cipher found.
If prompt appears, asking for username and password, it indicates that arcfour cipher is available in your machine, and is being used.

Another way to check is using the nmap utility.  What is nmap and what all magical things it can do is not what we want to see here.  Right now, let's use nmap to check which ciphers are available in your machine.

# nmap -Pn -sV --script ssh2-enum-algos your.ip.address.here

If arcfour is not listed anywhere in the output, it is not available in your machine.
If you see arcfour in the output, it is available in your machine.

If you don't have arcfour mentioned in your ssh configuration file /etc/ssh/sshd_config and it is still available in your machine, then check if the Ciphers keyword is altogether absent in the SSH configuration file.  If you don't mention which ciphers to use, then the default list of ciphers is offerred.  Likelihood is, the default list contains arcfour.  In this case, add the line listed below in your SSH configuration file.

Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

In order for the change to come in effect, ssh daemon needs to be restarted.  `service ssh restart` restarts the ssh daemon.

Please note, the /etc/ssh/ directory contains sshd_config and also ssh_config file.  sshd_config is the one that is for SSH daemon.  ssh_config is used by the SSH client.

If you have correctly removed arcfour from /etc/ssh/sshd_config file, and arcfour is still available in your machine, then it is time to check further.  Check for the possibility that more than one ssh daemons are running.  The default port used by sshd is 22.  And sshd could use some other port as well.  More than one sshd could be running, one of them listening on port 22 and the other listening on some other port.  In this case, arcfour cipher needs to be disabled for all ssh daemons.

Here's how to check which all ports are open.  Use the netstat utility along with -tlnp options.
# netstat -tlnp

The options that are useful to us in this case are :
        -t, --tcp                  tcp only
        -n, --numeric              don't resolve names
        -l, --listening            display listening server sockets
        -p, --programs             display PID/Program name for sockets