Recent Posts

61

General Linux / Test Internet Speed from the Terminal on Ubuntu Linux

« Last post by akhilt on August 11, 2018, 04:48:29 pm »

Test Internet Speed from the Terminal on Ubuntu Linux

We generally use flash based websites (like speedtest.net & whatismyip.com) to test our Internet speed but as Linux admin we love to test Internet speed from the command line or terminal.

speedtest.cli is command line tool which is used to test Internet bandwidth from the terminal on Linux like Operating system. In this post we will discuss how to install and use speedtest.cli on Ubuntu Linux.

Download the Script Speedtest.cli & assign Execute Permissions

Code: [Select]

# wget https://github.com/sivel/speedtest-cli/archive/master.zip
--2018-08-11 07:01:01--  https://github.com/sivel/speedtest-cli/archive/master.zip

# ll
total 36
drwxr-xr-x  2 root root  4096 Aug 11 07:01 ./
drwxr-xr-x 24 root root  4096 Aug 11 06:55 ../
-rw-r--r--  1 root root 27770 Aug 11 07:01 master.zip
-rw-r--r--  1 root root     0 Aug 11 06:55 speedtest-cli

# unzip master.zip
Archive:  master.zip
72ed585c6f9d19b39a80e0c9364592095d070f62
   creating: speedtest-cli-master/
  inflating: speedtest-cli-master/.gitignore 
  inflating: speedtest-cli-master/.travis.yml 
  inflating: speedtest-cli-master/CONTRIBUTING.md 
  inflating: speedtest-cli-master/LICENSE 
  inflating: speedtest-cli-master/MANIFEST.in 
  inflating: speedtest-cli-master/README.rst 
 extracting: speedtest-cli-master/setup.cfg 
  inflating: speedtest-cli-master/setup.py 
  inflating: speedtest-cli-master/speedtest-cli.1 
  inflating: speedtest-cli-master/speedtest.py 
   creating: speedtest-cli-master/tests/
   creating: speedtest-cli-master/tests/scripts/
  inflating: speedtest-cli-master/tests/scripts/source.py 
  inflating: speedtest-cli-master/tox.ini

# ll
total 40
drwxr-xr-x  3 root root  4096 Aug 11 07:01 ./
drwxr-xr-x 24 root root  4096 Aug 11 06:55 ../
-rw-r--r--  1 root root 27770 Aug 11 07:01 master.zip
-rw-r--r--  1 root root     0 Aug 11 06:55 speedtest-cli
drwxr-xr-x  3 root root  4096 May 24 12:06 speedtest-cli-master/

# ll
total 124
drwxr-xr-x 3 root root  4096 May 24 12:06 ./
drwxr-xr-x 3 root root  4096 Aug 11 07:01 ../
-rw-r--r-- 1 root root   232 May 24 12:06 .gitignore
-rw-r--r-- 1 root root  1247 May 24 12:06 .travis.yml
-rw-r--r-- 1 root root  1671 May 24 12:06 CONTRIBUTING.md
-rw-r--r-- 1 root root 11358 May 24 12:06 LICENSE
-rw-r--r-- 1 root root    35 May 24 12:06 MANIFEST.in
-rw-r--r-- 1 root root  5450 May 24 12:06 README.rst
-rw-r--r-- 1 root root    20 May 24 12:06 setup.cfg
-rw-r--r-- 1 root root  3277 May 24 12:06 setup.py
-rw-r--r-- 1 root root  2923 May 24 12:06 speedtest-cli.1
-rwxr-xr-x 1 root root 61423 May 24 12:06 speedtest.py*
drwxr-xr-x 3 root root  4096 May 24 12:06 tests/
-rw-r--r-- 1 root root   497 May 24 12:06 tox.ini



Now test Internet Speed by executing the below command:

Code: [Select]

# ./speedtest.py
Retrieving speedtest.net configuration...
Testing from Softcom Technology Consulting (xxx.xx.xx.xx)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by VMedia Inc (Toronto, ON) [2.92 km]: 30.739 ms
Testing download speed................................................................................
Download: 715.14 Mbit/s
Testing upload speed................................................................................................
Upload: 98.87 Mbit/s
Generate the result of Internet Speed in an Image hosted on speedtest.net

Code: [Select]

# ./speedtest.py --share
Retrieving speedtest.net configuration...
Testing from Softcom Technology Consulting (xxx.xx.xx.xx)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by VMedia Inc (Toronto, ON) [2.92 km]: 3.91 ms
Testing download speed................................................................................
Download: 904.91 Mbit/s
Testing upload speed................................................................................................
Upload: 10.86 Mbit/s
Share results: http://www.speedtest.net/result/7543683921.png

Go to Link  http://www.speedtest.net/result/7543683921.png, there you will see the result of your internet speed as an image as shown below.




Show the list of ‘speedtest.net’ Servers against which we can test Internet Speed.

Code: [Select]

# ./speedtest.py --list

Check Internet Speed against the Specific Server.

Code: [Select]

# ./speedtest.py --server 1777
Retrieving speedtest.net configuration...
Testing from Softcom Technology Consulting (xxx.xx.xx.xx)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Comcast (Winchester, VA) [504.68 km]: 75.547 ms
Testing download speed................................................................................
Download: 118.31 Mbit/s
Testing upload speed................................................................................................
Upload: 0.00 Mbit/s

Getting help and Options used in Speedtest.cli

Code: [Select]

# ./speedtest.py --help
usage: speedtest.py [-h] [--no-download] [--no-upload] [--bytes] [--share]
                    [--simple] [--csv] [--csv-delimiter CSV_DELIMITER]
                    [--csv-header] [--json] [--list] [--server SERVER]
                    [--exclude EXCLUDE] [--mini MINI] [--source SOURCE]
                    [--timeout TIMEOUT] [--secure] [--no-pre-allocate]
                    [--version]

Command line interface for testing internet bandwidth using speedtest.net.
--------------------------------------------------------------------------
https://github.com/sivel/speedtest-cli

optional arguments:
  -h, --help            show this help message and exit
  --no-download         Do not perform download test
  --no-upload           Do not perform upload test
  --bytes               Display values in bytes instead of bits. Does not
                        affect the image generated by --share, nor output from
                        --json or --csv
  --share               Generate and provide a URL to the speedtest.net share
                        results image, not displayed with --csv
  --simple              Suppress verbose output, only show basic information
  --csv                 Suppress verbose output, only show basic information
                        in CSV format. Speeds listed in bit/s and not affected
                        by --bytes
  --csv-delimiter CSV_DELIMITER
                        Single character delimiter to use in CSV output.
                        Default ","
  --csv-header          Print CSV headers
  --json                Suppress verbose output, only show basic information
                        in JSON format. Speeds listed in bit/s and not
                        affected by --bytes
  --list                Display a list of speedtest.net servers sorted by
                        distance
  --server SERVER       Specify a server ID to test against. Can be supplied
                        multiple times
  --exclude EXCLUDE     Exclude a server from selection. Can be supplied
                        multiple times
  --mini MINI           URL of the Speedtest Mini server
  --source SOURCE       Source IP address to bind to
  --timeout TIMEOUT     HTTP timeout in seconds. Default 10
  --secure              Use HTTPS instead of HTTP when communicating with
                        speedtest.net operated servers
  --no-pre-allocate     Do not pre allocate upload data. Pre allocation is
                        enabled by default to improve upload performance. To
                        support systems with insufficient memory, use this
                        option to avoid a MemoryError
  --version             Show the version number and exit
Hoping the information was useful, Thank you!

62

General Linux / Check Hard drive for bad sectors or bad blocks in linux

« Last post by akhilt on August 11, 2018, 04:18:33 pm »

Check Hard drive for bad sectors or bad blocks in Linux

badblocks is the command or utility in linux like operating system which can scan or test our hard disk and external drive for bad sectors. Bad sectors or bad blocks is the space of the disk which can’t be used due to the permanent damage or OS is unable to access it.

Badblocks  command will detect all bad blocks(bad sectors)  on our hard disk and save them in a text file so that we can use it with e2fsck to configure  Operating System(OS) to not store our data on these damaged sectors.

Step:1 Use fdisk command to identify your hard drive info

Code: [Select]

# sudo fdisk -l
Step:2  Scan your hard drive for Bad Sectors or Bad Blocks

Code: [Select]

# sudo badblocks -v /dev/sdb > /tmp/bad-blocks.txt
Just replace “/dev/sdb” with your own hard disk / partition. When we execute above command  a text file “bad-blocks” will be created under /tmp , which will contains all bad blocks.

Step:3 Inform OS not to use bad blocks  for storing data

Once the scanning is completed , if the bad sectors are reported , then use file “bad-blocks.txt” with e2fsck command  and force OS not to use these bad blocks for storing data.

Code: [Select]

# sudo e2fsck -l /tmp/bad-blocks.txt  /dev/sdb
Note : Before running e2fsck command , you just make sure the drive is not mounted.

For any futher help on badblocks & e2fsck command , read their man pages

Code: [Select]

# man badblocks
# man e2fsck
Hope this was a useful piece of information, Thank you!

63

General Linux / Install Apache Tomcat 9 on Ubuntu 18.04 / 16.04 LTS Server

« Last post by akhilt on August 11, 2018, 04:07:53 pm »

Install Apache Tomcat 9 on Ubuntu 18.04 / 16.04 LTS Server

Apache Tomcat 9 is a free and open source web server which is used to serve java based web applications. In other words we can say Tomcat server is used to implement Java Servlet and JSP (Java Server Pages) technologies.

In this article we will demonstrate how to install latest version of tomcat i.e Tomcat 9.0.10 on Ubuntu 18.04 / 16.04 LTS Server. One of the important perquisite of Tomcat 9 is Java 8 or higher version, lets jump into the installations steps.

Step:1) Install Java 8 or its higher version

Using the default configured apt repositories on ubuntu 18.04 & 16.04 server, we can install the java using the package name “default-jdk“. Run the beneath apt commands,

Code: [Select]

$ sudo apt update
$ sudo apt install default-jdk -y
Once above packages are installed then verified the Java version using below command,

Code: [Select]

$ java --version
openjdk 10.0.1 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Ubuntu-3ubuntu1)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Ubuntu-3ubuntu1, mixed mode)
$
Step:2 Create tomcat User

Tomcat service will be controlled by tomcat user, so let create a user with name “tomcat“, whose home directory will be “/opt/tomcat” and shell will be “/bin/false”.

If the folder or home directory of tomcat (/opt/tomcat) doesn’t exist then create it using mkdir command,

Code: [Select]

$ sudo mkdir /opt/tomcat
Create a tomcat user using below useradd command,

Code: [Select]

$ sudo useradd -s /bin/false -d /opt/tomcat tomcat
Step:3 Download & Install Tomcat 9 from command line

Download the latest version of tomcat 9 (i.e 9.0.10) from its official Web site https://tomcat.apache.org/download-90.cgi. In my case I am directly downloading it on my server using below wget command,

Code: [Select]

$ wget http://www-us.apache.org/dist/tomcat/tomcat-9/v9.0.10/bin/apache-tomcat-9.0.10.tar.gz
Once the above file is downloaded, extract it into the tomcat user’s home folder (/opt/tomcat) using below tar command

Code: [Select]

$ sudo tar xzvf apache-tomcat-9.0.10.tar.gz -C /opt/tomcat --strip-components=1
Set the required permissions on /opt/tomcat and its sub folder and files.

Code: [Select]

$ sudo chgrp -R tomcat /opt/tomcat
$ cd /opt/tomcat/
/opt/tomcat$ sudo chmod -R g+r conf
/opt/tomcat$ sudo chmod g+x conf
Make the tomcat as file owner of the following directories webapps, work, temp, and logs using the beneath chown command

Code: [Select]

/opt/tomcat$ sudo chown -R tomcat webapps/ work/ temp/ logs/
So now all the required permissions are set, let create tomcat systemd service so that tomcat service can be controlled by systemctl command.

Code: [Select]

Step:4) Create Tomcat Systemd Service
To configure tomcat as a service on ubuntu server we need to create systemd file. In the systemd file we must specify the java path, so first find java path using below command,

Code: [Select]

$ sudo update-java-alternatives -l
java-1.11.0-openjdk-amd64      1101       /usr/lib/jvm/java-1.11.0-openjdk-amd64
$
Now create a systemd file with name “/etc/systemd/system

Code: [Select]

/tomcat.service[/font]” and insert the following contents

$ sudo vi /etc/systemd/system/tomcat.service
[Unit]
Description=Apache Tomcat Web Application Container
After=network.target

[Service]
Type=forking

Environment=JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64
Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
Environment='CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC'
Environment='JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom'
ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/opt/tomcat/bin/shutdown.sh

User=tomcat
Group=tomcat
UMask=0007
RestartSec=10
Restart=always

[Install]
WantedBy=multi-user.target
Reload the systemd daemon and start tomcat service,

Code: [Select]

$ sudo systemctl daemon-reload
$ sudo systemctl start tomcat
$
$ sudo systemctl status tomcat
Output of above command should be something like below,

Step:5) Test Your Tomcat 9 Server

In case OS firewall is enabled and configured on your server then we need to allow 8080 port. Use the below command

Code: [Select]

$ sudo ufw allow 8080
Rule added
Rule added (v6)
$
By default, we can access the tomcat web gui from Server itself either via local ip or localhost and if you want to access it from outside then we must comment out one line in both the files “/opt/tomcat/webapps/host-manager/META-INF/context.xml” and “/opt/tomcat/webapps/manager/META-INF/context.xml”

Code: [Select]

$ sudo vi /opt/tomcat/webapps/host-manager/META-INF/context.xml
----------------------------------------------------------
<Valve className="org.apache.catalina.valves.RemoteAddrValve"
  allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" />
----------------------------------------------------------
Change to or comment out
----------------------------------------------------------
<!--Valve className="org.apache.catalina.valves.RemoteAddrValve"
  allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" /> -->
-----------------------------------------------------------
$ sudo vi /opt/tomcat/webapps/manager/META-INF/context.xml
-----------------------------------------------------------
<Valve className="org.apache.catalina.valves.RemoteAddrValve"
         allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" />
-----------------------------------------------------------
Change to or Comment out
-----------------------------------------------------------
<!--Valve className="org.apache.catalina.valves.RemoteAddrValve"
         allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" />-->
-----------------------------------------------------------
Restart the tomcat service using the below command,

Code: [Select]

$ sudo systemctl restart tomcat
Now Open your web browser and type url :  http://<Tomcat-Server-IP-Addres>:8080

As of now we have successfully installed Tomcat 9, but to Access Application Web manager and Virtual Host Manager we must define some user and its password in “tomcat-users.xml” file. So add the following line before </tomcat-users>

Code: [Select]

$ sudo vi /opt/tomcat/conf/tomcat-users.xml
----------------------------------------------------------
<role rolename="manager-gui"/>
<user username="linuxtechi" password="XXXXXXX" roles="manager-gui,admin-gui"/>
----------------------------------------------------------
Save and exit the file then restart the tomcat service

Code: [Select]

$ sudo systemctl restart tomcat
$
Now Try to access Web Application Manager and Virtual Host Manager, use the same user name and password that you have specified in the tomcat-users.xml file.







This confirms that you have successfully installed Tomcat 9 on your Ubuntu 18.04 / 16.04 LTS Server. That’s all from this tutorial, Thank you!!

64

General Linux / Install and use Open vSwitch 2.9 with KVM on CentOS 7 / RHEL 7 Server

« Last post by akhilt on August 11, 2018, 03:27:12 pm »

Install and use Open vSwitch 2.9 with KVM on CentOS 7 / RHEL 7 Server


Open vSwitch is a free and open source multi-layer software switch, which is used to manage the traffic between virtual machines and physical or logical networks.  It provides the features like traffic isolation in OpenStack tenants using the overlay techniques likes GRE, VXLAN and 802.1Q VLANs.

Using open vSwitch packet forwarding engine in kernel space or user space can be implemented and Link aggregation can also be accomplished through LACP.

Here we will discuss how to install latest version of Open vSwitch 2.9.2 on CentOS 7 and RHEL 7 Server. Apart from this we will see how open vSwitch can be used in KVM virtual machines for their networking.


Step 1) Install the required packages using yum command

Login to your CentOS 7 or RHEL 7 server as root and run the beneath yum command:

Code: [Select]

# yum install wget openssl-devel  python-sphinx gcc make python-devel openssl-devel kernel-devel graphviz kernel-debug-devel autoconf automake rpm-build redhat-rpm-config libtool python-twisted-core python-zope-interface PyQt4 desktop-file-utils libcap-ng-devel groff checkpolicy selinux-policy-devel -y

Step 2) Create ovs user and download Open vSwitch 2.9

Create a user with name ovs using below command

Code: [Select]

# useradd ovs
# su - ovs
Download OVS 2.9 tar file and build the openvswitch rpm using beneath commands,

Code: [Select]

$ mkdir -p ~/rpmbuild/S$ wget http://openvswitch.org/releases/openvswitch-2.9.2.tar.gzOURCES
$ cp openvswitch-2.9.2.tar.gz ~/rpmbuild/SOURCES/
$ tar xfz openvswitch-2.9.2.tar.gz
$ rpmbuild -bb --nocheck openvswitch-2.9.2/rhel/openvswitch-fedora.spec
$ exit
logout
#
Now install Open vSwitch rpm using below yum command:

Code: [Select]

# yum localinstall /home/ovs/rpmbuild/RPMS/x86_64/openvswitch-2.9.2-1.el7.x86_64.rpm -y

Step 3) Start and enable Open vSwitch Service

Use below systemctl commands to start and enable open vSwitch service

Code: [Select]

# systemctl start openvswitch.service
# systemctl enable openvswitch.service
Created symlink from /etc/systemd/system/multi-user.target.wants/openvswitch.service to /usr/lib/systemd/system/openvswitch.service.
# systemctl status openvswitch.service
● openvswitch.service - Open vSwitch
Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled)
Active: active (exited) since Sun 2018-08-05 10:16:12 EDT; 17s ago
Main PID: 73958 (code=exited, status=0/SUCCESS)
Aug 05 10:16:12 compute02 systemd[1]: Starting Open vSwitch...
Aug 05 10:16:12 compute02 systemd[1]: Started Open vSwitch.
#
Use below command to check the OVS version

Code: [Select]

# ovs-vsctl -V
ovs-vsctl (Open vSwitch) 2.9.2
DB Schema 7.15.1
#
Step:4 Create the OVS bridge and add interfaces to it.

Use the below command to create ovs bridge,

Code: [Select]

# ovs-vsctl add-br ovs-br0
Now flush or remove IP from the interface, in my case IP as was assigned to eno16777736

Code: [Select]

# ip addr flush dev eno16777736
Now assign this IP address to ovs bridge(ovs-br0)

Code: [Select]

# ip addr add 192.168.1.4/24 dev ovs-br0
Add interface as port in ovs-br0 using below command,

Code: [Select]

# ovs-vsctl add-port ovs-br0 eno16777736
Now bring up the bridge using below ‘ip link’ command,

Code: [Select]

# ip link set dev ovs-br0 up
Note : Above changes are not persistent, so make these changes persistent across the reboot, we have to create ovs bridge file, steps are shown below

Code: [Select]

# cd /etc/sysconfig/ne
# cd /etc/sysconfig/network-scripts/twork-scripts/
# cp ifcfg-eno16777736 ifcfg-ovs-br0
#
# vi ifcfg-eno16777736
DEVICE=eno16777736
HWADDR="00:0c:29:c1:c3:4e"
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=ovs-br0
ONBOOT=yes
Save and exit the file

Code: [Select]

# vi ifcfg-ovs-br0
DEVICE=ovs-br0
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
IPADDR=192.168.1.4
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
ONBOOT=yes
Save & exit the file

Restart the network service using below command,

Code: [Select]

# systemctl restart network
Now Verify ovs bridge and its ports using “ovs-vsctl” command

Code: [Select]

# ovs-vsctl show
8dc5f8e7-0e54-4d9d-ba7a-cd6b9b94f470
    Bridge "ovs-br0"
        Port "ovs-br0"
            Interface "ovs-br0"
                type: internal
        Port "eno16777736"
            Interface "eno16777736"
    ovs_version: "2.9.2"
#
Step:5 ) Create and define virsh ovs network

Create ovs network file with the following contents ,

Code: [Select]

# vi /tmp/ovs-network.xml
<network>
<name>ovs-network</name>
<forward mode='bridge'/>
<bridge name='ovs-br0'/>
<virtualport type='openvswitch'/>
</network>
Now define the ovs-network using below virsh command,

Code: [Select]

# virsh net-define /tmp/ovs-network.xml
Network ovs-network defined from /tmp/ovs-network.xml
# virsh net-start ovs-network
Network ovs-network started
# virsh net-autostart ovs-network
Network ovs-network marked as autostarted
#
Now verify the virsh network using beneath command,

Code: [Select]

# virsh net-list
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 ovs-network          active     yes           yes
[root@compute02 ~]#
Step:6) Create Virtual machines and attach ovs-network

Let’s create two test VMs using below virt-install command, specify the network for these VMs as ovs-network,

Code: [Select]

# virt-install  -n testvm1  --description "Test VM1 for OVS "  --os-type=Linux  --os-variant=rhel7  --ram=1096  --vcpus=1  --disk path=/var/lib/libvirt/images/testvm1.img,bus=virtio,size=10  --network network:ovs-network --graphics none  --location /root/CentOS-7-x86_64-DVD-1511.iso --extra-args console=ttyS0
Similarly Create a second test vm with name “testvm2”

Code: [Select]

# virt-install  -n testvm2  --description "Test VM2 for OVS "  --os-type=Linux  --os-variant=rhel7  --ram=1096  --vcpus=1  --disk path=/var/lib/libvirt/images/testvm2.img,bus=virtio,size=10  --network network:ovs-network --graphics none  --location /root/CentOS-7-x86_64-DVD-1511.iso --extra-args console=ttyS0
Once the VMs are created then their interfaces should be added in ovs bridge (ov-br0) automatically and we can verify this from ovs-vsctl command,

Code: [Select]

# ovs-vsctl show
8dc5f8e7-0e54-4d9d-ba7a-cd6b9b94f470
    Bridge "ovs-br0"
        Port "ovs-br0"
            Interface "ovs-br0"
                type: internal
        Port "eno16777736"
            Interface "eno16777736"
        Port "vnet0"
            Interface "vnet0"
        Port "vnet1"
            Interface "vnet1"
    ovs_version: "2.9.2"
# ovs-vsctl list-ports ovs-br0
eno16777736
vnet0
vnet1
#
In the above command vnet0 & vnet1 are VMs tap interface. When we shutdown these VMS then these ports will be removed from OVS Bridge (ovs-br0) automatically. Log files for ovs (openvswitch) are kept under the folder “/var/log/openvswitch“.

This conclude this article, I hope you got an idea how to install and use Open vSwitch 2.9 with KVM on CentOS 7 and RHEL 7 Servers.

Thank You!!

65

General Linux / Install Python 3.6 and pip3 in Ubuntu

« Last post by vichithrakumart on August 11, 2018, 03:15:09 pm »

1. Download Python-3.6.1.tar.xz from https://www.python.org/
2. Unzip the file and keep the folder in home directory.
3. Open terminal in that directory and perform the following commands:
./configure
make
make test
sudo make install
4. This will install Python 3.6 but pip3 may not be working.Install necessary modules using:
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
5. Now write the following to re run the installation:
sudo make
sudo make install
6. Now you can install packages with Python 3.6 using pip3 command. For example:
sudo pip3 install numpy

66

General Linux / ngrep – A Network Packet Analyzer for Linux

« Last post by alext on August 11, 2018, 03:03:56 pm »

Ngrep (network grep) is a simple yet powerful network packet analyzer. It is a grep-like tool applied to the network layer – it matches traffic passing over a network interface. It allows you to specify an extended regular or hexadecimal expression to match against data payloads (the actual information or message in transmitted data, but not auto-generated metadata) of packets.

This tool works with various types of protocols, including IPv4/6, TCP, UDP, ICMPv4/6, IGMP as well as Raw on a number of interfaces. It operates in the same fashion as tcpdump packet sniffing tool.

The package ngrep is available to install from the default system repositories in mainstream Linux distributions using package management tool as shown

Code: [Select]

$ sudo apt install ngrep
$ sudo yum install ngrep
$ sudo dnf install ngrep
After installing ngrep, you can start analyzing traffic on your Linux network using following examples.

1. The following command will help you match all ping requests on the default working interface. You need to open another terminal and try to ping another remote machine. The -q flag tell ngrep to work quietly, to not output any information other than packet headers and their payloads.

Code: [Select]

[code]$ sudo ngrep -q '.' 'icmp'interface: enp0s3 (192.168.0.0/255.255.255.0)
filter: ( icmp ) and ((ip || ip6) || (vlan && (ip || ip6)))
match: .
I 192.168.0.104 -> 192.168.0.103 8:0
]...~oG[....j....................... !"#$%&'()*+,-./01234567                                                                                                             
I 192.168.0.103 -> 192.168.0.104 0:0
]...~oG[....j....................... !"#$%&'()*+,-./01234567                                                                                                             
I 192.168.0.104 -> 192.168.0.103 8:0
]....oG[............................ !"#$%&'()*+,-./01234567                                                                                                             
I 192.168.0.103 -> 192.168.0.104 0:0
]....oG[............................ !"#$%&'()*+,-./01234567  [/code]

You can press Ctrl + C to terminate it.

2. To match only traffic going to a particular destination site, for instance ‘google.com’, run the following command, then try to access it from a browser.

Code: [Select]

[code]$ sudo ngrep -q '.' 'host google.com'interface: enp0s3 (192.168.0.0/255.255.255.0)
filter: ( host google.com ) and ((ip || ip6) || (vlan && (ip || ip6)))
match: .
T 172.217.160.174:443 -> 192.168.0.103:54008 [AP]
..................;.(...RZr..$....s=..l.Q+R.U..4..g.j..I,.l..:{y.a,....C{5>......p..@..EV..                                                                       
T 172.217.160.174:443 -> 192.168.0.103:54008 [AP]
.............l.......!,0hJ....0.%F..!...l|.........PL..X...t..T.2DC..... ..y...~Y;.$@Yv.Q6[/code]

3. If you are surfing the web, then run the following command to monitor which files your browser is requesting:.

Code: [Select]

$ sudo ngrep -q '^GET .* HTTP/1.[01]'interface: enp0s3 (192.168.0.0/255.255.255.0)
filter: ((ip || ip6) || (vlan && (ip || ip6)))
match: ^GET .* HTTP/1.[01]
T 192.168.0.104:43040 -> 172.217.160.174:80 [AP]
GET / HTTP/1.1..Host: google.com..User-Agent: Links (2.13; Linux 4.17.6-1.el7.elrepo.x86_64 x86_64;
GNU C 4.8.5; text)..Accept: */*..Accept-Language: en,*;q=0.1..Accept-
Encoding: gzip, deflate, bzip2..Accept-Charset: us-ascii,ISO-8859-1,ISO-8859-2,ISO-8859-3,ISO-8859-4,
ISO-8859-5,ISO-8859-6,ISO-8859-7,ISO-8859-8,ISO-8859-9,ISO-8859-10,I
SO-8859-13,ISO-8859-14,ISO-8859-15,ISO-8859-16,windows-1250,windows-1251,windows-1252,windows-1256,
windows-1257,cp437,cp737,cp850,cp852,cp866,x-cp866-u,x-mac,x-mac-ce,x-
kam-cs,koi8-r,koi8-u,koi8-ru,TCVN-5712,VISCII,utf-8..Connection: keep-alive....

4. To see all activity crossing source or destination port 25 (SMTP), run the following command.

Code: [Select]

$ sudo ngrep port 25
5. To monitor any network-based syslog traffic for the occurrence of the word “error”, use the following command.

 

Code: [Select]

$ sudo ngrep -d any 'error' port 514
Importantly, this tool can convert service port names stored in “/etc/services” (on Unix-like systems such as Linux) to port numbers. This command is equivalent to the above command.

Code: [Select]

$ sudo ngrep -d any 'error' port syslog
6. You can also run ngrep against an HTTP server (port 80), it will match all requests to the destination host as shown.

Code: [Select]

$ sudo ngrep port 80
interface: eth0 (64.90.164.72/255.255.255.252)
filter: ip and ( port 80 )
####
T 67.169.59.38:42167 -> 64.90.164.74:80 [AP]
GET / HTTP/1.1..User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i
686) Opera 7.21  [en]..Host: www.darkridge.com..Accept: text/html, applicat
ion/xml;q=0.9, application/xhtml+xml;q=0.9, image/png, image/jpeg, image/gi
f, image/x-xbitmap, */*;q=0.1..Accept-Charset: iso-8859-1, utf-8, utf-16, *
;q=0.1..Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0..Cookie: SQ
MSESSID=5272f9ae21c07eca4dfd75f9a3cda22e..Cookie2: $Version=1..Connection:
Keep-Alive, TE..TE: deflate, gzip, chunked, identity, trailers....
##
As you can see in the above output all HTTP headers transmission are displayed in their gory detail. It’s hard to parse though, so let’s watch what happens when you apply -W byline mode.

Code: [Select]

$ sudo ngrep -W byline port 80
interface: eth0 (64.90.164.72/255.255.255.252)
filter: ip and ( port 80 )
####
T 67.169.59.38:42177 -> 64.90.164.74:80 [AP]
GET / HTTP/1.1.
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686) Opera ...
Host: www.darkridge.com.
Accept: text/html, application/xml;q=0.9, application/xhtml+xml;q=0.9 ...
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1.
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0.
Cookie: SQMSESSID=5272f9ae21c07eca4dfd75f9a3cda22e.
Cookie2: $Version=1.
Cache-Control: no-cache.
Connection: Keep-Alive, TE.
TE: deflate, gzip, chunked, identity, trailers.

7. To print a timestamp in the form of YYYY/MM/DD HH:MM:SS.UUUUUU every time a packet is matched, use the -t flag.

Code: [Select]

[code]$ sudo ngrep -t -W byline port 80interface: enp0s3 (192.168.0.0/255.255.255.0)
filter: ( port 80 ) and ((ip || ip6) || (vlan && (ip || ip6)))
####
T 2018/07/12 16:33:19.348084 192.168.0.104:43048 -> 172.217.160.174:80 [AP]
GET / HTTP/1.1.
Host: google.com.
User-Agent: Links (2.13; Linux 4.17.6-1.el7.elrepo.x86_64 x86_64; GNU C 4.8.5; text).
Accept: */*.
Accept-Language: en,*;q=0.1.
Accept-Encoding: gzip, deflate, bzip2.
Accept-Charset: us-ascii,ISO-8859-1,ISO-8859-2,ISO-8859-3,ISO-8859-4,ISO-8859-5,utf-8.
Connection: keep-alive.
[/code]
8. To avoid putting the interface being monitored into promiscuous mode (where it intercepts and reads each network packet that arrives in its entirety), add the -p flag.

Code: [Select]

$ sudo ngrep -p -W byline port 80
9. Another important option is -N which is useful in case you are observing raw or unknown protocols. It tells ngrep to display the sub-protocol number along with single-character identifier.

Code: [Select]

$ sudo ngrep -N -W byline
For more information, see the ngrep man page.

Code: [Select]

$ man ngrep

67

Plesk / How to recreate mail handlers in Plesk 10.4 and later

« Last post by vichithrakumart on August 08, 2018, 11:19:27 pm »

1. Check if mail handlers exist on your server:

 

Code: [Select]

# /usr/local/psa/admin/sbin/mail_handlers_control --list

2. Examine the mail log to see if the mail handlers are executed:

Code: [Select]

# tail /usr/local/psa/var/log/maillog | grep qmail-queue-handlers
3. Run mchk to regenerate mail handlers:

Code: [Select]

# /usr/local/psa/admin/sbin/mchk --with-spam

68

General Linux / Can't exec “locale”: No such file or directory

« Last post by vichithrakumart on August 06, 2018, 10:22:59 pm »

This error shows when you repairing the Ubuntu using the repair button

To resolve the issue please follow below steps:

Code: [Select]

# apt-get download libc-bin
# dpkg -x libc-bin*.deb unpackdir/

# sudo cp unpackdir/sbin/ldconfig /sbin/

# sudo apt-get install --reinstall libc-bin
# sudo apt-get install -f

69

General Linux / How to automatically dim your screen on Linux

« Last post by akhilt on August 05, 2018, 04:48:31 pm »

How to automatically dim your screen on Linux

When you start spending the majority of your time in front of a computer, natural questions start arising. Is this healthy? How can I diminish the strain on my eyes? Why is the sunlight burning me? Although active research is still going on to answer these questions, a lot of programmers have already adopted a few applications to make their daily habits a little healthier for their eyes. Among those applications, there are two which I found particularly interesting: Calise and Redshift.

Calise

In and out of development limbo, Calise stands for "Camera Light Sensor." In other terms, it is an open source program that computes the best backlight level for your screen based on the light intensity received by your webcam. And for more precision, Calise is capable of taking in account the weather in your area based on your geographical coordinates. What I like about it is the compatibility with every desktops, even non-X ones.

It comes with a command line interface and a GUI, supports multiple user profiles, and can even export its data to CSV. After installation, you will have to calibrate it quickly before the magic happens.

What is less likeable is unfortunately that if you are as paranoid as I am, you have a little piece of tape in front of your webcam, which greatly affects Calise's precision. But that aside, Calise is a great application, which deserves our attention and support. As I mentioned earlier, it has gone through some rough patches in its development schedule over the last couple of years, so I really hope that this project will continue.

Find the tool from here:
http://calise.sourceforge.net/wordpress/


Redshift

If you already considered decreasing the strain on your eyes caused by your screen, it is possible that you have heard of f.lux, a free proprietary software that modifies the luminosity and color scheme of your display based on the time of the day. However, if you really prefer open source software, there is an alternative: Redshift. Inspired by f.lux, Redshift also alters the color scheme and luminosity to enhance the experience of sitting in front of your screen at night. On startup, you can configure it with you geographic position as longitude and latitude, and then let it run in tray. Redshift will smoothly adjust the color scheme or your screen based on the position of the sun. At night, you will see the screen's color temperature turn towards red, making it a lot less painful for your eyes.

Just like Calise, it proposes a command line interface as well as a GUI client. To start Redshift quickly, just use the command:

Code: [Select]

$ redshift -l [LAT]:[LON]
Replacing [LAT]:[LON] by your latitude and longitude.

However, it is also possible to input your coordinates by GPS via the gpsd module. For Arch Linux users, I recommend this wiki page.

Find the tool from here:
http://jonls.dk/redshift/


Conclusion

To conclude, Linux users have no excuse for not taking care of their eyes. Calise and Redshift are both amazing. I really hope that their development will continue and that they get the support they deserve. Of course, there are more than just two programs out there to fulfill the purpose of protecting your eyes and staying healthy, but I feel that Calise and Redshift are a good start.

70

General Linux / Back up a WordPress website to remote cloud storage

« Last post by akhilt on August 05, 2018, 04:23:39 pm »

Back up a WordPress website to remote cloud storage

There are many different ways to archive the current snapshot of a WordPress site. Some web hosting or VPS companies offer automatic daily backup service for an extra fee. Many web-based hosting control panels (e.g., cPanel, Webmin) come with a full website backup option for you to back things up interactively. There are also WordPress plugins dedicated to firing scheduled WordPress backup cron jobs. Even some third-party online services enable you to back up and version-control a WordPress deployment off-site and restore any previous snapshot at your command.

Yet another WordPress backup option is Linux command line based WordPress backup. As you can imagine, this is the most flexible option, and you retain complete control over the entire backup process. This option is applicable only if your hosting provider or VPS allows SSH remote access. Assuming this applies to you, here is how you can back up your WordPress website and store it to offsite cloud storage, all from the command line.

There are two steps to WordPress backup. One is to back up a WordPress database which stores WordPress content (e.g., postings, comments). The other is to back up PHP files or design files hosted on your WordPress site. After going over these steps one by one, I will present full scripts for complete WordPress backup, so you can easily copy and paste them for your own use.

Backing up a WordPress Database

Backing up a WordPress database can easily be done from the command line with a tool called mysqldump. This command line tool comes with MySQL client package, which you can install easily if you are running a VPS. If your hosting provider provides SSH remote access, most likely MySQL client is installed on the server you are on.

The following command will dump the content of your WordPress database into a MySQL dump file called db.sql.

Code: [Select]

$ mysqldump --add-drop-table -h<db-host> -u<db-user> -p<db-password> <db-name> > db.sql
You can find <db-host>, <db-user>, <db-password> and <db-name> in wp-config.php of your WordPress installation.

The "--add-drop-table" option above tells mysqldump to add "DROP TABLE IF EXISTS" statement before each table creation statement in the MySQL dump file. When you import a MySQL dump into a database, if the database already has a table with the same name as the one to import, the import will fail due to duplicate table name. Thus adding "DROP TABLE IF EXISTS" statement is a precaution to prevent this potential import failure.

Note that if MySQL server is running remotely (i.e., "db-host" is not localhost), you have to make sure that the MySQL server allows remote database access. Otherwise, the mysqldump command will fail.

Backing up WordPress Files

The next step is to back up all PHP and design files hosted in your WordPress site.

First, go to the root directory (e.g., ~/public_html) of your WordPress site. The exact path may vary depending on your setup.

Code: [Select]

$ cd ~/public_html
Then create a compressed archive which contains all the files in the root directory using tar command.

One useful option to use is "--exclude=PATTERN", which allows you to exclude files/directories which you do not want to include in the TAR archive. You can repeat this option as many time as you want. For example, you can exclude the "wp-content/cache" directory which is used by different plugins to hold temporarily cached files for speedup purposes. You can also exclude any plugins/themes (e.g., wp-content/plugins/unused_plugin) you do not and will not plan to use.

The following command creates a bzip-compressed archive of the root directory of your WordPress site, excluding the WordPress cache directory.

Code: [Select]

$ tar -jcvf ../backup.tar.bz2 --exclude='wp-content/cache/*' .
Store a WordPress Backup at Off-site Cloud Storage

Once a backup archive is created, it is best to store the archive at a remote location off of the hosting server to prevent any accidental data loss. An affordable option for offsite backup is cloud storage. For example, Dropbox offers 2GB of free space, while Amazon S3 gives away 5GB via AWS Free Usage Tier.

Here I demonstrate how to upload a WordPress backup archive to both Dropbox and Amazon S3.

Upload to Dropbox Cloud Storage

If you prefer AWS S3 as a WordPress backup storage, here is the shell script to upload to AWS S3. Replace bucket, s3Key and s3Secret with your own information.

file=backup.tar.bz2
bucket="xxxxxxxxxxxxxxxxxxx"
s3Key="XXXXXXXXXXXXXXXXXXX"
s3Secret="YYYYYYYYYYYYYYYYYYYYY"
 
resource="/${bucket}/${file}"
contentType="application/x-compressed-tar"
dateValue=`date -R`
stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
curl -X PUT -T "${file}" \
  -H "Host: ${bucket}.s3.amazonaws.com" \
  -H "Date: ${dateValue}" \
  -H "Content-Type: ${contentType}" \
  -H "Authorization: AWS ${s3Key}:${signature}" \
  https://${bucket}.s3.amazonaws.com/${file}

Note that this script requires curl and openssl installed.

Full Scripts for Remote WordPress Backup

For you to easily apply what I have shown so far, here are the full WordPress backup scripts. Each script creates a bzip-compressed backup of your WordPress site including its database, and uploads the archive to either Dropbox or Amazon AWS S3. For daily backups, the created backup archive is named after the current date.

WordPress Backup to Dropbox

#!/bin/sh
 
# MySQL information: replace it with your own
hostname=localhost
username=MY_USER
password=MY_PASS
database=MY_DB
 
# Dropbox information: replace it with your own
dropbox_uploader="~/bin/dropbox_uploader.sh"
dropbox_folder="Backup"
 
cur=`date +"%Y-%m-%d"`
dbfile=db.$cur.sql
wpfile=backup.$cur.tar.bz2
 
cd ~/public_html
 
echo "back up database"
mysqldump --add-drop-table -h$hostname -u$username -p$password $database > db.$cur.sql
 
echo "compress database"
bzip2 $dbfile
 
echo "back up wordpress"
tar -jcvf ../$wpfile --exclude='wp-content/cache/*' .
 
echo "transfer to dropbox"
$dropbox_uploader upload ../$wpfile $dropbox_folder
 
echo "done!"

WordPress Backup to Amazon AWS S3

#!/bin/sh
 
# MySQL information: replace it with your own
hostname=MY_HOST
username=MY_USER
password=MY_PASS
database=MY_DB
 
# AWS S3 information: replace it with your own
bucket=MY_BUCKET
s3Key=MY_S3_KEY
s3Secret=MY_S3_SECRET
 
cur=`date +"%Y-%m-%d"`
dbfile=db.$cur.sql
wpfile=backup.$cur.tar.bz2
 
cd ~/public_html
 
echo "back up database"
mysqldump --add-drop-table -h$hostname -u$username -p$password $database > db.$cur.sql
 
echo "compress database"
bzip2 $dbfile
 
echo "back up wordpress"
tar -jcvf ../$wpfile --exclude='wp-content/cache/*' .
 
echo "transfer to s3"
resource="/${bucket}/${wpfile}"
contentType="application/x-compressed-tar"
dateValue=`date -R`
stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
curl -X PUT -T "../${wpfile}" \
  -H "Host: ${bucket}.s3.amazonaws.com" \
  -H "Date: ${dateValue}" \
  -H "Content-Type: ${contentType}" \
  -H "Authorization: AWS ${s3Key}:${signature}" \
  https://${bucket}.s3.amazonaws.com/${wpfile}
 
echo "done!"

To make WordPress backup a daily routine, you can set up a daily CRON job to execute either script on a daily basis.
Conclusion

Regular website backup goes a long way to protect your long-term investment in your website. In this tutorial, I present a way to back up a WordPress website to remote cloud storage such as Dropbox and AWS S3.

Another approach to WordPress backup is performing a full backup followed by rsync-based incremental backups. During incremental backups, only changes made to your WordPress site (e.g., newly created, modified or deleted files) since the latest backup are archived. The incremental backups can save disk space (and bandwidth for offsite backup) if your website rarely gets updated. The price you pay however is when you want to "restore" a backup. You would have to have the last full backup and "every" incremental backup in-between, and restore them one by one in exact sequence. Also, the remote storage would need to talk the rsync protocol, which is not the case for existing cloud storage.

I personally prefer full WordPress backup to incremental snapshots due to the simplicity of the former. But everyone is entitled to his or her own opinion so feel free to share yours. Happy backup!