Bash scripts to diagnose network issues
These are 2 simple scripts I wrote that run various types of tests to help diagnose network issues between a source and target. They can be safely run as a non-root user as long as the commands exist.
Script 1 - Various network tests
This script runs 3 separate tests and generates 3 logs.
Logs Generated
1. Check port availability (test_network_checkport.log)
This simply uses the nc command to see if the port is available or not. No packets are sent or received, but if you notice high response times occasionally, then something is fishy.
# TIMESTAMP | SOURCE HOSTNAME | TARGET IP | TARGET PORT | RESPONSE TIME (SECS)
2021-12-17 21:14:14.731991069|soadev|192.168.1.31|5901|0.01
2021-12-17 21:14:15.284783737|soadev|192.168.1.31|5901|0.01
2021-12-17 21:14:15.837768193|soadev|192.168.1.31|5901|0.012. Check error and dropped packets (test_network_packets.log)
This uses the ip command to count the number of dropped and error packets during transmission and receive.
# TIMESTAMP | SOURCE HOSTNAME | TRANSMISSION PACKETS ERRORS | TRANSMISSION PACKETS DROPS | RECEIVE PACKET ERRORS | RECEIVE ERROR DROPS
2021-12-17 21:14:13.626336229|soadev|0|0|0|0
2021-12-17 21:14:14.179361714|soadev|0|0|0|0
2021-12-17 21:14:14.731991069|soadev|0|0|0|03. Check bad and retransmitted packets (test_network_segments.log)
The netstat command is used here to observe the bad and retransmitted segments. Don't look at the actually values returned, but rather if they are growing instead.
# TIMESTAMP | SOURCE HOSTNAME | BAD SEGMENTS | RETRANSMITTED SEGMENTS | RETRANSMITTED SEGMENTS %
2021-12-17 21:14:13.626336229|soadev|297|30846213|6.25439
2021-12-17 21:14:14.179361714|soadev|297|30846213|6.25439
2021-12-17 21:14:14.731991069|soadev|297|30846213|6.25439Running the Script
ifconfig -a
./test_network.sh 192.168.1.31 443 8 1 eth0The ifconfig command is used to get the network interface name to be used in the script (as the last parameter).
In this script, the target IP and target port (443) are passed, and the number of loops (8) with a delay (1 second) on an interface (ens3).
Source Code
Source code for the script:
#!/bin/bash
#--------------------------------------------------------------#
# FILENAME: test_network.sh #
# CREATION DATE: 2021-12-06 #
# DESCRIPTION: Various port, packet, segment checks #
# AUTHOR: Ahmed Aboulnaga #
# LOG: test_network_checkport.log #
# test_network_packets.log #
# test_network_segments.log #
#--------------------------------------------------------------#
#--------------------------------------------------------------#
# Help |
#--------------------------------------------------------------#
if [ $# -ne 5 ]; then
echo ""
echo "Usage:"
echo " ./test_network.sh [target_ip] [target_port] [number_of_loops] [delay_in_secs] [interface]"
echo ""
echo "Logs:"
echo ""
echo " test_network_checkport.log"
echo " # Port availability, no packets transmitted"
echo " # TIMESTAMP | SOURCE HOSTNAME | TARGET IP | TARGET PORT | RESPONSE TIME (SECS)"
echo " 2021-12-17 21:14:14.731991069|soadev|192.168.1.31|5901|0.01"
echo " 2021-12-17 21:14:15.284783737|soadev|192.168.1.31|5901|0.01"
echo " 2021-12-17 21:14:15.837768193|soadev|192.168.1.31|5901|0.01"
echo ""
echo " test_network_packets.log"
echo " # Packet transmission and receive packet errors and drops"
echo " # TIMESTAMP | SOURCE HOSTNAME | TARGET IP | TRANSMISSION PACKETS ERRORS | TRANSMISSION PACKETS DROPS | RECEIVE PACKET ERRORS | RECEIVE ERROR DROPS"
echo " 2021-12-17 21:14:13.626336229|soadev|0|0|0|0"
echo " 2021-12-17 21:14:14.179361714|soadev|0|0|0|0"
echo " 2021-12-17 21:14:14.731991069|soadev|0|0|0|0"
echo ""
echo " test_network_segments.log"
echo " # Bad and retransmitted segments"
echo " # TIMESTAMP | SOURCE HOSTNAME | TARGET IP | BAD SEGMENTS | RETRANSMITTED SEGMENTS | RETRANSMITTED SEGMENTS %"
echo " 2021-12-17 21:14:13.626336229|soadev|297|30846213|6.25439"
echo " 2021-12-17 21:14:14.179361714|soadev|297|30846213|6.25439"
echo " 2021-12-17 21:14:14.731991069|soadev|297|30846213|6.25439"
echo ""
echo "Example:"
echo " ./test_network.sh"
echo " ./test_network.sh 192.168.1.31 443 8 1 eth0"
echo " nohup ./test_network.sh 192.168.1.31 443 345600 0.5 eth0 &"
echo ""
exit 0
fi
#--------------------------------------------------------------#
# Parameters |
#--------------------------------------------------------------#
V_TARGETIP=${1}
V_TARGETPORT=${2}
V_LOOP=${3}
V_DELAY=${4}
V_INTERFACE=${5}
#--------------------------------------------------------------#
# Loop #
#--------------------------------------------------------------#
i=1
while [ ${i} -le ${V_LOOP} ]; do
V_TIMESTAMP=`date +'%Y-%m-%d %H:%M:%S.%N'`
i=`expr ${i} + 1`
sleep ${V_DELAY}
#--------------------------------------------------------------#
# Check port |
#--------------------------------------------------------------#
# Log: number of seconds for positive response
if command -v nc &> /dev/null; then
nc -vz ${V_TARGETIP} ${V_TARGETPORT} > test_network.tmp 2>&1
echo "${V_TIMESTAMP}|`hostname`|${V_TARGETIP}|${V_TARGETPORT}|`cat test_network.tmp | tail -1 | awk '{print $9}'`" >> test_network_checkport.log
rm -f test_network.tmp
fi
#--------------------------------------------------------------#
# Packet errors #
#--------------------------------------------------------------#
# Log: Transmitted Packet Errors | Transmitted Packet Drops | Received Packet Errors | Received Packet Drops
if command -v ip &> /dev/null; then
V_TX_PACKETERRORS=`cat /proc/net/dev | grep ${V_INTERFACE} | awk {'print $12'}`
V_TX_PACKETDROPS=`cat /proc/net/dev | grep ${V_INTERFACE} | awk {'print $13'}`
V_RX_PACKETERRORS=`cat /proc/net/dev | grep ${V_INTERFACE} | awk {'print $4'}`
V_RX_PACKETDROPS=`cat /proc/net/dev | grep ${V_INTERFACE} | awk {'print $5'}`
echo "${V_TIMESTAMP}|`hostname`|${V_TX_PACKETERRORS}|${V_TX_PACKETDROPS}|${V_RX_PACKETERRORS}|${V_RX_PACKETDROPS}" >> test_network_packets.log
fi
#--------------------------------------------------------------#
# Segment retransmissions & bad #
#--------------------------------------------------------------#
# Log: Segments Bad Count | Segments Retransmitted Count | Segments Retransmitted %
if command -v netstat &> /dev/null; then
V_SEGS_RETRANSMIT=`netstat -s | grep retransmited | awk {'print $1'}`
V_SEGS_BAD=`netstat -s | grep bad | grep segments | awk {'print $1'}`
V_SEGS_RETRANSMITPER=`gawk 'BEGIN {OFS=" "} $1 ~ /Tcp:/ && $2 !~ /RtoAlgorithm/ {print ($13/$12*100)}' /proc/net/snmp`
echo "${V_TIMESTAMP}|`hostname`|${V_SEGS_BAD}|${V_SEGS_RETRANSMIT}|${V_SEGS_RETRANSMITPER}" >> test_network_segments.log
fi
done
exit 1
Script 2 - Traffic loss
This script generates a single log file.
Log Generated
1. Check traffic loss (test_trafficloss.log)
This script uses the mtr command, which combines the functionality of traceroute and ping into a single network diagnostic tool.
Start: Fri Dec 17 21:10:33 2021
HOST: soadev Loss% Snt Last Avg Best Wrst StDev
1.|-- 140.91.196.17 0.0% 10 0.1 0.1 0.1 0.1 0.0
2.|-- 4.16.73.206 0.0% 10 0.5 6.7 0.5 57.1 17.7
3.|-- ae60.edge5.Washington12.L 0.0% 10 20.5 6.6 0.5 27.6 9.6
4.|-- 142.250.166.242 0.0% 10 2.9 2.9 2.7 3.2 0.0
5.|-- 108.170.240.97 0.0% 10 1.9 1.7 1.5 2.0 0.0
6.|-- 108.170.240.112 0.0% 10 0.7 2.5 0.6 15.4 4.5
7.|-- 142.251.49.17 70.0% 10 1.1 1.3 1.1 1.5 0.0
8.|-- 142.251.49.29 50.0% 10 1.5 1.7 1.5 1.9 0.0
9.|-- 216.239.56.73 0.0% 10 8.2 14.5 8.2 66.7 18.3
10.|-- 216.239.40.130 20.0% 10 15.0 14.9 14.8 15.2 0.0
11.|-- 209.85.240.16 60.0% 10 32.1 32.5 31.8 33.6 0.0
12.|-- 216.239.62.212 0.0% 10 32.2 33.2 32.2 41.3 2.8
13.|-- 108.170.252.129 0.0% 10 31.5 31.5 31.5 31.5 0.0
14.|-- 209.85.244.59 0.0% 10 32.6 33.3 32.2 41.5 2.8
15.|-- atl14s07-in-f142.1e100.ne 0.0% 10 31.7 31.7 31.7 31.7 0.0
- If traffic loss is observed in the last hop, this is generally not an issue with the connection to the target.
- If traffic loss is observed in the middle, this is possibly due to ICMP rate limiting which is not an issue.
- If traffic loss is observed from the middle to the end, there is likely loss in traffic.
Running the Script
./test_trafficloss.sh 192.168.1.31 10The target IP is passed with a count of number of times you want to loop.
Source Code
Source code for the script:
#!/bin/bash
#--------------------------------------------------------------#
# FILENAME: test_trafficloss.sh #
# CREATION DATE: 2021-12-17 #
# DESCRIPTION: Test traffic loss #
# AUTHOR: Ahmed Aboulnaga #
# LOG: test_trafficloss.log #
#--------------------------------------------------------------#
#--------------------------------------------------------------#
# Help |
#--------------------------------------------------------------#
if [ $# -ne 2 ]; then
echo ""
echo "Usage:"
echo " ./test_trafficloss.sh [target_ip] [number_of_loops]"
echo ""
echo "Description:"
echo " If traffic loss is in the last hop, not an issue with the connection to target."
echo " If traffic loss is in the middle, possibly due to ICMP rate limiting; not an issue."
echo " If traffic loss is in the middle to the end, likely losing some traffic."
echo ""
echo "Example:"
echo " ./test_trafficloss.sh"
echo " ./test_trafficloss.sh 192.168.1.31 10"
echo " nohup ./test_trafficloss.sh 192.168.1.31 10 &"
echo ""
exit 0
fi
#--------------------------------------------------------------#
# Parameters |
#--------------------------------------------------------------#
V_TARGETIP=${1}
V_LOOP=${2}
#--------------------------------------------------------------#
# Loop #
#--------------------------------------------------------------#
i=1
while [ ${i} -le ${V_LOOP} ]; do
V_TIMESTAMP=`date +'%Y-%m-%d %H:%M:%S.%N'`
i=`expr ${i} + 1`
#--------------------------------------------------------------#
# Traffic loss |
#--------------------------------------------------------------#
if command -v mtr &> /dev/null; then
mtr --report ${V_TARGETIP} >> test_trafficloss.log
fi
done
exit 1