Linux Admin Daily Usage

Published on Author gryzli

Shell

 

Calculate the size of all files from a given type/extension

Recently I need something for doing such calculations and end up with the following command line (found on stackexchange), which will calculate the size summary for all “*.jpg” files in “some_directory”.

 

 

Using Linux moreutils 

Recently I found out the existence of moreutils . The package could be installed as an additional package in most of the linux distros from the standart repository. 

After installing it, you will get some really interesting programs accessible like:

 

Some nice examples about moreutils:

moreutils examples

 

 

Sponge – Redirect output from a file to the same file in bash

Many times I needed to do something like this in bash: 

But if you try to do this, you will end up with an empty file ! … So don’t do this ^^^^^^^ ! :) 

So here is where sponge can help, it is as simple as this: 

This works like a charm !

 

Some useful commands and bash tricks

For more extensive list of bash tips. take a look at my bash tips and tricks

 

 

Managing Big Number Of Hosts Through SSH

The following software is a golden piece: 

pssh

pscp

prsync

Combining it with ssh multiplexing, you will have some serious tool in your pocket. 

 

Speeding up SSH connections to remote servers

Some tips for speeding up your SSH connections to remote servers, especially if you are frequently reusing them.

 

Disable DNS resolve on SSHD server

( This should be executed on the host you are connecting to)

In Centos, SSHD by default has dns resolving enabled, which tries to get the reverse dns record from the connecting ip.

Put the following line in your config to disable it:

open /etc/ssh/sshd_config

Restart sshd server

systemctl restart sshd

/etc/init.d/sshd restart

 

Using SSH Multiplexing

Usefull technique when you connect to big number of servers and frequently re-connect to them.

Perfect for combining with pssh and pscp tools.

The idea of multiplexing is that once you open a ssh connection to a server, openssh will preserve your socket (until a timeout pasts) and every new connection to the same server will reuse the socket, which is very fast.

In order to use Multiplexing you need the following lines in your “~/.ssh/config” file

If you want to connect to some of the already connected hosts but with different settings (Xforwarding or -A for example), you could temporary disable ControlMaster for the new connection: 

 

 

 

Usefull settings for ~/.ssh/config

 

Removing files which filenames are starting with “–“

 

Search for all files that DOES NOT contain a given string

 

Change MOTD on Centos to a colorful message

 

Tools for working with Conntrack table

 

 

Nagios

Execute remote nrpe check

 

Dovecot

 

List parsed list of current Dovecot configuration

Sometimes you may need to parse Dovecot settings with external scripts. Going through the Dovecot config and it’s includes (+ nested syntax), could easily become automation hell.

One very helpful binary, that can help reading settings by other scripts/programs is “doveconf

 

 

MySQL 

Repairing MySQL databases/tables 

Repair single database
Repair single table of a database
Repair all databases

 

Repair all crashed tables, getting them from mysql error log

 

Usefull MySQL Queries

 

MySQL Hide Headers, Column Names And Formatting

By using the -N option, you can skip/hide the column/header information. 

By adding the ‘-B’ option, you could also remove the column formattings: 

 

SQLite 

Creating database in SQLite3

Creating tables in SQLite

 

Using Rowid As Primary Key Instead Of Autoincrement

In SQLite table, each row has internal identification number called “ROWID” and is 64 bit unsigned integer (unless you define your table with “WITHOUT ROWID” ). 

So you can use this ROWID instead of explicitly defining auto-increment field for primary key. 

In order to do it, you need to define your PK with ‘INTEGER primary key‘ : 

It will behave much like auto-increment, but one key difference is that the ID that will be taken for the new record, is not “+1 on whatever existed” but rather “+1 on what currently exists“. 

So if you have records with id’s “1,2,3” and you add new, it will get “4”. But if you delete the row with id=4, and then again create new one, it will again take “4” as id. 

 

Usefull commands

Show current tables .tables
Quit from sqlite console .quit or .q
Turn headers on/off  .headers on|off
Check table structure (describe table) .schema table 

 Executing batch commands 

Execute sqlite batch commands, without the need of going into the interactive sqlite shell.

 

Iptables 

 

Showing all current rules + interface + traffic counters 

 

Adding comments inside iptables rules 

Comments are very very usefull thing inside iptables. They not only could hint you why this rule is there, but also could be used as UNIQUE IDENTIFIERS, for checking if a rule exists, before trying to add it again (which could save you from duplicate rules). 

You can add multiple comments: 

The line above will add 2 comments inside the rule. 

 

Using multiport module for adding multiple ports to a rule

The multiport iptables module is another very useful module to use. By default iptables gives you the ability to define port ranges easily with the ‘–dports’ flag like ‘–dports 100:200’.

But sometime you want a rule for multiple ports, which are not continuous as a range. 
Let’s add rule for ports 55 and 77: 

 

Protecting from iptables xtables lock error (Another app is currently holding the xtables lock)

When you invoke iptables, it first tries to acquire a xtables lock in order to give you consistent result. If you use iptables a lot in scripts (cron jobs), there is a big chance to hit this problem, why you try to execute some rule. 

The worse situation is if you have multiple cron jobs, and they are hitting the xtables lock without you even know about it.

That’s why –  Always check the execution status of your iptables commands. 

If you have newer version of iptables, you should be able to use the lock-safe wait option:

Yum

List packages with lower versions inside yum:

Search for packages which provide some file by using “whatprovides

This could be used for both full path and wildcards.

 

 

NetBeans

Make smooth fonts in Netbeans 8.2 under Linux (Fedora in my case).

Just run netbeans with the following additional options:

 

AWK

Calculate the sum of a given column (from output). The current example calculates the sum for the first column.