Cache the passwords on your ssh private key(s) so you don't need to enter them again on servers where your public key is installed.

eval $(ssh-agent); ssh-add

Under Linux, check the number of CPUs, then prepare "xz" compression commands to utilize each one. Code them for all files in the current directory, moving from smallest to largest (to minimize temporary disk space usage). The commands are merely printed, not executed - after verification, pipe them to a shell.

C="xz -9e"; P=$(grep ^processor /proc/cpuinfo | wc -l); (ls -rS; echo /) | while read a; do if [[ "${a}" = "/" ]]; then j=0; while [[ "${j}" -lt "${P}" ]]; do echo "${C} ${f[j]} &"; ((j+=1)); done; break; fi; if [[ -f "${a}" ]]; then f[i]="${f[i]} '${a}'"; ((i+=1)); [[ "${i}" == "${P}" ]] && i=0; fi; done

Compress the contents of the current directory, but compress the smallest files first (handy when short of disk space for intermediate files):

ls -s | awk NR!=1 | sort -n | awk '{print $2}' | xargs bzip2 -9v

Use 7-Zip and tar together with standard input/output (best/highest compression settings for 32-bit Linux distribution binary [version 16.02], as discussed with author Igor Pavlov - if you prepare a 64-bit binary, try one of: -mx -mmt2 -md512m (or) -mx -mmt4 -md256m (or) -mx -mmt8 -md128m for best compression with much larger memory usage):

tar cf - (directory_path) | 7za a -si -mmt3 -md96m -mx=9 directory.tar.7z
7za x -so directory.tar.7z | tar xpf -

Cut a large ISO image into 1gb "chunks" that fit within the VFAT 2gb file size limit on most flash drives:

7za a -mx9 bigISO.7z /path/to/bigISO.iso -v1g

Fragment of sshd_config to lock in DJB ciphers, with the exception of aes256-ctr which will allow the use of Android ConnectBot (that sadly does not yet support DJB). Note that PuTTY might fail by default with this configuration since it doesn't support the MAC; this can be fixed by adjusting the "Connection / SSH / Cipher" setting in PuTTY's configuration pane, where it is necessary to push ChaCha20 above AES; save this per session/host to avoid repeated entry. ChaCha20 is an AEAD cipher, and bundles the poly1305 MAC; when ChaCha20 is active, all MAC settings in sshd_config are ignored (and PuTTY dispenses with them). DJB ciphers are generally superior to most alternatives (and the linked article speaks harshly of the GCM implementation of AES, the best alternative within ssh).


Fetch an X.509 cert from an https website:

openssl s_client -connect < /dev/null 2>&1 | awk '/^-.*CER/,/^-.*END/'

Find the largest file in a directory tree:

[old way] find /path -type f -exec ls -s {} \; | sort -n | tail -1

[gnu way] find /path -type f -print0 | xargs -0 ls -s {} \; | sort -n | tail -1

Find the largest file in a directory tree, using GNU Coreutils:

find /path -type f -ls | sort -n -k7 | tail -1

Print the number of regular files in each subdirectory of the current directory:

for x in *; do printf "%40s %g\n" $x $(find $x -type f -print | wc -l); done

Print the total bytes of files in a directory grouped by date (useful for Oracle archived log volume summaries):

ls -l|awk 'length($6)==3{x[$6" "$7]+=$5};END{for(i in x)printf"%s:\t%d\n",i,x[i]}'

Define a shell function to verify an MD5 signature for a single file, taking the first argument as the filename and the second argument as the expected/posted MD5 signature. After the function is defined, it can be called like so: md5v php-5.6.5.tar.xz 541a480e1f8747219074c99f3e9edbcc.

md5v(){ echo "$2  $1" | md5sum -c;}

Force a file to lowercase - useful for reading a line of input with head -1 in a script, then checking for y or n. Older dd doesn't have the status option - only tr is consistent in very old UNIX (but sometimes the lack of quotes is very helpful when writing scripts that run across multiple machines):

dd conv=lcase status=none if=/tmp/userinput
awk '{print tolower($0)}' /tmp/userinput
tr '[A-Z]' '[a-z]' < /tmp/userinput

Shell function to search the process table. Change -ef to aux below for the Berkeley form (better information) if your system supports it (on Solaris, look for /usr/ucb/ps). For example, "pfind ora_dbw" will search for all running Oracle database writer processes.

pfind(){ IFS='|';ps -ef|while read x;do [ -z "$y" ] && echo $x;y=1;case $x in *$1*)echo $x;;esac;done }

	or (Linux-specific version)

pfind() {
	unset qx qy
	if [[ -z "$COLUMNS" ]]
	ps -eo user:7,pid:5,%cpu,%mem,args |
	while read qx
		if [[ -z "$qy" ]]
			echo $qx
			case $qx in
				*$1*) echo ${qx:0:${COLUMNS}};;

	or (gawk, using regex instead of sh pattterns)

pfind(){ ps aux | gawk -v p="$1" 'NR==1;$0~p&&$2!=PROCINFO["pid"]'; }

Burn a CD of the current directory on Linux without an intermediate ISO file (check size with "du -sm ." first; also check "cdrecord -scanbus" for the right device ID):

nice --18 mkisofs -r -T -J -V "Volume Label" . | cdrecord -v fs=6m speed=8 dev=ATA:1,1,0 -eject -

Convert cdparanoia-style tracks to cdda2wav (for cdrecord's useinfo):

x=1;while true;do f=$(printf "track%02g.cdda.wav" $x);[ ! -f $f ] && break;mv $f $(printf "audio_%02g.wav" $x);((x++));done

Burn a DVD with a volume label:

growisofs -Z /dev/cdrom -V "Volume Label" -r -J .

Burn a pre-existing ISO file with growisofs:

growisofs -dvd-compat -Z /dev/cdrom=file.iso

Send a message to a remote syslog (works in bash and ksh93 [most commonly /usr/dt/bin/dtksh] - ksh seems to need an IP address, not a hostname). The remote syslog might need to be running with the "-r" option:

 echo message to remote syslog >> /dev/udp/

Remove spaces from filenames in the current directory (use with caution; examine the output before you pipe to the shell).

ls|awk '{a=$0;gsub(/ /,"",a);print "mv '"'"'"$0"'"'"' "a}'|sh

Remove blank lines and comments (lines beginning with #).

awk '$1!~/(^#|^$)/' /path/to/file.txt

Report all lines in file_1 that are not in file_2:

awk 'BEGIN{while(getline < "file_1" > 0) x[$0]=1; while(getline < "file_2" > 0) delete x[$0]; for(y in x) print y}'

Show what ports are open on your Linux system, and what progams are listening on them:

netstat -ap --numeric-ports | awk '$(NF-1)=="LISTEN"'

Send a file to a remote netcat with gawk:

server$ nc -l 12345
client$ gawk '{print |& "/inet/tcp/0/"}' < file.txt

Print out a list of all CVE numbers addressed by patches in your RPM database:

rpm -qa | while read x; do rpm -q "${x}" --changelog; done | awk 'BEGIN{IGNORECASE=1}; /cve-/ {i = $0; while(p = match(i, "cve-[0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9]")){print toupper(substr(i, p, 13)); i = substr(i, p + 1)}}' | sort -u | tee vul.txt

Print all expiring accounts from /etc/shadow in a format similar to "chage -l":

gawk 'function dt(x) { return strftime("%b %d %Y", x * 86400 + 86400) }; BEGIN { FS=":"; OFS="\t"; while (getline < "/etc/shadow") if($5 != 99999) print $1, dt($3), dt($3 + $5), dt($3 + $5 + $7), $4, $5, $6 }'

Sort accounts by password expiration (read from /etc/shadow):

gawk 'BEGIN { FS=":"; OFS="\t"; while (getline < "/etc/shadow") if($5 != 99999) print (($3 + $5) * 86400 - systime())/86400, $1, strftime("%b %d %Y", ($3 + $5) * 86400 + 86400); }' | sort -n

Get your SSH public key working on CentOS when SELinux is enabled:

chcon -t ssh_home_t ~/.ssh/authorized_keys

Generate an 8-character random password (does not include the letter "O" or the number "0" to avoid confusion):

while [[ $pw != [A-NP-Za-np-z]* ]] || [[ $pw != *[1-9]* ]] || [[ $pw == *[^A-NP-Za-np-z1-9]* ]]; do pw=$(openssl rand -base64 6); done; echo $pw; unset pw

Tell Oracle to let you see the time as well as the date:

alter session set nls_date_format = "RRRR/MM/DD HH24:MI"

Oracle invalid object report:

select owner, object_name || ' - '|| object_type from dba_objects where status='INVALID' and owner not in ('PUBLIC','OLAPSYS') order by owner, object_name;

Generate Oracle SQL for manual object recompiles of various types:

select 'alter view '||owner||'.'||object_name || ' compile;' from dba_objects where status='INVALID' and object_type='VIEW' order by owner, object_name;

select 'alter function '||owner||'.'||object_name || ' compile;' from dba_objects where status='INVALID' and object_type='FUNCTION' order by owner, object_name;

select 'alter procedure '||owner||'.'||object_name || ' compile;' from dba_objects where status='INVALID' and object_type='PROCEDURE' order by owner, object_name;

select 'alter trigger '||owner||'.'||object_name || ' compile;' from dba_objects where status='INVALID' and object_type='TRIGGER' order by owner, object_name;

select 'alter package '||owner||'.'||object_name || ' compile;' from dba_objects where status='INVALID' and object_type='PACKAGE' order by owner, object_name;

select 'alter package '||owner||'.'||object_name || ' compile body;' from dba_objects where status='INVALID' and object_type='PACKAGE BODY' order by owner, object_name;

How big is your Oracle database?

select, d.mb from v$database v, (select sum(bytes)/1024/1024 mb from dba_data_files) d;

Show Oracle index size and key compression status:

select s.owner||'.'||s.index_name, s.bytes, i.compression,i.prefix_length from
  (select owner, index_name, compression, prefix_length from dba_indexes) i,
  (select owner, segment_name index_name, sum(bytes) bytes
    from dba_segments where segment_type='INDEX' group by owner, segment_name) s
where i.owner = s.owner and i.index_name = s.index_name
order by s.bytes;

Extract SQL from the alert log to show objects impacted by (NOLOGGING) block corruption in a recovery session (also check rman's REPORT UNRECOVERABLE; command in the source database for impacted datafiles):

awk '/^Recovery is repairing media corrupt block/ && $7~/^[0-9]*$/ && $10~/^[0-9]*$/ {print "select owner||'"'"'.'"'"'||segment_name FROM dba_extents WHERE file_id="$10" AND "$7" between block_id and block_id+blocks-1 and rownum=1;"}' alert_orcl.log

Likewise extract SQL from the output of the DBVERIFY (dbv) utility to report (NOLOGGING) block corruption, as indicated by the DBV-00200 and DBV-00201 errors.

awk '/^DBV-002/ {b=$4; sub(/,$/,"",b); print "select owner||'"'"'.'"'"'||segment_name FROM dba_extents WHERE file_id=dbms_utility.data_block_address_file(" b ") AND dbms_utility.data_block_address_block(" b ") between block_id and block_id+blocks-1 and rownum=1;"}'

To allow an Oracle 8 client to logon to a 12c server (and silence the ORA-28040 error), add the following to your sqlnet.ora (found under network/admin):


Compute a checksum of a PL/SQL object (i.e. a trigger, function, procedure, or package). This is useful for verifying that a PL/SQL object is identical between development, QA, and production.

set numwidth 50
select avg(dbms_utility.get_hash_value(text,1000000000,power(2,30))) checksum from dba_source where owner='SCOTT' and name='TIGER_TRIGGER';

Under a command prompt on Windows, change the file directory listing to report file age in days, rather than the absolute time:

dir | \bin\gawk "{if($1~/^[0-9]+\/[0-9]+\/[0-9]+$/&&$2~/^[0-9]+:[0-9]+$/&&$3~/^[AP]M$/){split($1,d,\"/\");split($2,t,\":\");if($3==\"PM\"&&t[1]!=12)t[1]+=12;printf\"%4d\",int(10*(systime()-mktime(d[3]\" \"d[1]\" \"d[2]\" \"t[1]\" \"t[2]\" 0\"))/86400+0.5)/10;print substr($0, 21)}else print}"

Under a command prompt on Windows, find all files larger than 64 megabytes and print their size, date, and path delimited by semicolons (sorted by size). You can override the 64 megabyte threshold by specifying -v T=#BYTES (as shown in the second example):

dir /s /-c | \bin\gawk "BEGIN{T=(T?T:67108864)};{if($1==\"Directory\"&&$2==\"of\"){d=$0;gsub(/(^ Directory of |[\\]$)/,\"\",d)};if($3~/^(AM|PM)$/&&$4~/^[0-9]+$/&&$4>T){m=substr($0,1,20);gsub(/[ ]+/,\" \",m);f=substr($0,40);gsub(/\r/,\"\",f);f=d\"\\\\\"f;s[f]=$4+0;t[f]=m;k[$4]=1}};END{for(i in k)l[j++]=i+0;u=asort(l);OFS=\";\";for(i=1;i<u;i++)for(j in s)if(l[i]==s[j])print s[j],t[j],j}"
dir /s /-c | \bin\gawk -v T=40000000 "BEGIN{T=(T?T:67108864)};{if($1==\"Directory\"&&$2==\"of\"){d=$0;gsub(/(^ Directory of |[\\]$)/,\"\",d)};if($3~/^(AM|PM)$/&&$4~/^[0-9]+$/&&$4>T){m=substr($0,1,20);gsub(/[ ]+/,\" \",m);f=substr($0,40);gsub(/\r/,\"\",f);f=d\"\\\\\"f;s[f]=$4+0;t[f]=m;k[$4]=1}};END{for(i in k)l[j++]=i+0;u=asort(l);OFS=\";\";for(i=1;i<u;i++)for(j in s)if(l[i]==s[j])print s[j],t[j],j}"