JP's Linux Commands Lib

Commands

alias
cat
cd
chmod - change permissions
chown
cls
cp - copy files and directories
df - disk free - see how much space is left on the disk
du - see size of directories
fgrep
find
grep
gzip - compress files
ifconfig - equivalent of ipconfig
iptables - firewall config
last - see list of recent logins
less
ln - create symlinks
ls
mkdir - create a directory
mv - copy or rename
netstat - find out what's open on a machine
nohup
passwd - change a password
ps
pwd
rpm - Redhat package manager
rm
scp
su - change user
tail
tar - compress folders, equivalent of zip
time
touch - create empty file
whereis - locate files in certain locations
wget
who - see who is logged in
zip

Applications / Misc

Apache
append (a line of text to a file)
awk
bash
Chaining Commands
/dev/null
Close Hanging Window
Directory Data - drwxr-xr-x
Directory Structure - /etc /root
Environment Variables
Full path
Hostname change - eg when prompt [root@emedia-dev] is wrong
MySql
Network Services
Page through command result
Perl scripts
PHP
Redirectors - > or |
Run program in background
Search and Replace all files in a folder
See all users
See server load
Shortcuts
Standard Out and Standard Error
Switch Applications
Switch Terminals
Symlink
rc scripts
Which version of Linux?
vi

Keyboard Shortcuts

Ctrl A - go to begining of line
Ctrl D - logout (or exit)

Commands

alias

Examples:
alias dira='ls mnt/floppy'
alias s='ftp </home/myftp'
alias apache_stop="/opt/apache/bin/apachectl stop"
alias apache_start="/opt/apache/bin/apachectl start"
Ref
To make this permanent, edit the /ect/bashrc file:
vi /ect/bashrc
Ref

Top

cat

to open file with cat and only read the first 100 lines
cat jon_test.csv | head -n 100

Top

cd

to go to home directory of current user (ie /root for root user, /home/emanuel for user emanuel)
cd ~
to go up one directory level (don't forget the space!)
cd<space>..
put / before a partition
cd /etc
to go back into the directory you were in before
cd -

Top

chmod

changes permissions
chmod 640 index.html
Why 640? It's binary:
6 == 110 (file owner)
4 = 100 (file owner group)
0 == 000 (anyone)
corresponds to rw- r-- ---
640 , 740, 742 all normal combinations
640 is a safe config, BUT with 640, index.html is not world-readable, and apache is not in the file owner group, so file can't be seen in a browser

Top

chown

changes the owner to jon
chown jon test.txt
may be used with su

Top

cp

copy home directory to floppy
$ cp -f /mnt/floppy/* /home/simon
Ref
Quick Ref 2
-i : prompt before overwriting files
-r : copy recursively (copy subdirectories and their contents)
-u : overwrite destination files only when the source file is newer
-v : show what is being done
Note: cp *.* will only copy files, use cp -r * to copy directories too

Top

df

disk free - see how much space is left on the disk
df

/dev/sda2 4127108 3677512 239948 94% /
/dev/sda1 124427 8072 109931 7% /boot
none 1036312 0 1036312 0% /dev/shm
/dev/sda3 256665 9195 234217 4% /tmp
/dev/sda5 8262036 1867176 5975164 24% /var
/dev/sda6 18271560 12069184 5274224 70% /home

Top

du

to really see how big eg all the folders named ...tmp... are use
du -bc tmp*
-k will give you kilobytes
du -ka tmp*
-m megabytes
du -ms tmp_*
du -ms *
Byron used
du -skh
List all the subdirectories of the current directory sorted in ascending order by size
du -S .| sort -n
To get the size of all directories in the current directory (ref)
find . -maxdepth 1 -type d -print | xargs du -sk | sort -rn
To see the size of only the first level of subdirectories
du --max-depth=1
Good reference

Top

fgrep

fast grep
fgrep is quick but can't use reg exp
grep uses reg exp
-n show line numbers, then search string, then file
fgrep -n "22/Sep/2005:01:52:36 " bedlvnunet_log.2005.09.22
count number of matching lines in a grep, use the -c switch
fgrep google.fr access_log.2005.08.08 -c
Ctrl C to break
output ("pipe") to mytest.txt
fgrep -n "22/Sep/2005:01:52:36 " bedlvnunet_log.2005.09.22 > mytest.txt
fgrep -n -i "drop table" *vnunet_log.2005.09.22
fgrep -ni "drop table" *vnunet_log.2005.09.22

Top

find

Go to root:
cd /
find / -iname "index.php"
Reference
find . gives all files
find . | grep "tomcat" find all files with "tomcat" in it
find . | grep "tomcat" > myfile.txt and put the output in a file
To search entire hard disk for files containing "pattern":
find / -name '*' -exec grep pattern {} \;
Byron example:
find /export -name hf.css -ls | grep ANY | grep fr
687675 4 -rw-r--r-- 1 root root 3101 Feb 1 18:07 /export/eurodev/eurosite/resources/css/vnustyles/fr/ANY/hf.css
668536 4 -rw-r--r-- 1 root root 393 Nov 29 17:51 /export/eurodev/eurosite/resources/css/dwnstyles/fr/ANY/hf.css
568217 4 -rw-rw-r-- 1 root root 3072 Nov 14 20:14 /export/cruisecontrol/checkout/eurosite/resources/css/vnustyles/fr/ANY/hf.css
669228 4 -rw-rw-r-- 1 root root 393 Nov 29 18:11 /export/cruisecontrol/checkout/eurosite/resources/css/dwnstyles/fr/ANY/hf.css

From man find:
-ls True; list current file in `ls -dils' format on standard output. The block counts are of 1K blocks,
unless the environment variable POSIXLY_CORRECT is set, in which case 512-byte blocks are used. See the
UNUSUAL FILENAMES section for information about how unusual characters in filenames are handled.
find *
list all files at current level and below
find . -mtime +180 -delete;
find . -mtime +180 -exec rm -f {} \;
use find to delete all files older than 180 days (ref)
find . -size +1b -delete
find . -size +100k -delete
use find to delete all files bigger than 1byte or 100kB (ref)

Top

grep

grep might take 20 mins while fgrep takes 5
can use tail on the detective.txt file while it's being written in 2nd session
doesn't show the content of the files, just the filenames
to see which files have /home/yacapa/www
grep -rliz "\/home\/yacapa\/www" * | less
grep -rliz "infostore" *
grep -n -i "drop.+?table" *vnunet_log.2005.09.22
looks for drop + at least one character of shit + table
ie ignores droptable, dropFOOBARtab
finds drop table, dropFOOBARtable
+ means 1 or many times
grep -n -i "drop.*?table" *vnunet_log.2005.09.22
* means 0 or many times
ie finds droptable
grep -n -i "drop%+?table" *vnunet_log.2005.09.22
looks for 1 or more % after drop
matches drop%table, drop%%%%%%table
no match on drop%20table
grep -n -i "drop.+?table" *vnunet_log.2005.09.22 > detective.txt
Example - find all strings containing ServerNames in all files in a folder
grep "ServerName" *

telecharger.conf: ServerName www.yacapa.com
telecharger.conf: ServerName telecharger.yacapa.com
telecharger.conf: ServerName download.yacapa.com

grep ServerName * | awk '{print $3}' - prints the 3rd variable, as separated by spaces

www.yacapa.com
telecharger.yacapa.com
download.yacapa.com

grep "ServerAlias" *

Top

gzip

used for compressing files, not folders
if you use gzip -r images, you don't produce a zip of the /images folder, you zip every image within the /images folder!
gunzip -r images to undo that idiocy
see tar or zip for packaging a folder

Top

iptables

iptables configure firewalls - they say "allow this from here" etc

Top

less

less <filename>
"less" can only read, not edit - you need vi, emacs or something similar
for the next page
<space>
to go to end (opposite to go to start)
>
to quit
q or Esc
to move within doc
use page up/down or arrows
To search within less use a slash then the word
/DocumentRoot <return>
Can also use RegEx here
/Doc.*Root will find DocRoot, DocumentRoot etc
RewriteRule.*\.html will look for RewriteRule ..... .html (note the escaped dot)
For "Find Next" just use slash
/

Top

ln

to create a symlink called phpMyAdmin which links to directory phpMyAdmin-2.6.3-pl1
ln -s phpMyAdmin-2.6.3-pl1/ phpMyAdmin (note reverse order!)

Top

ls

list all
ls - la
gives details read/write/execute. NOTE: only gives the SIZE in bytes of the TOP level
Others that I saw Byron using:
ls -lrt
ls -lg
ls -lh
ls -lhrt - list all files in reverse (r) time (t) order, listing the file sizes in human (h) readable format ie not bytes
to get all files in current directory that have dlvnunet_log in the title
ls -la *dlvnunet_log.2005.09.22
to list all .cgi files in a directory
ls -la *.cgi
list files vertically in alphabetical order
ls -la | awk 'BEGIN { FS = "[0-9]+:[0-9]+ " } ; { print $2 }'
ls -1
list files in file-size order - the ls command has a -S flag which does this. Or -Sr to reverse the order
ls -larS
For a list of subdirs in the current directory:
ls -alF | grep /$ (could alias this to 'lsd')
For a list of all the files in /usr/bin that start with "mysql"
ls -la /usr/bin | grep "mysql"
to page through a long list of results
ls -la | less
use ls to count the number of files in a directory
ls -1 | wc -l

Top

mkdir

create a directory
mkdir dir1

Top

mv

rename a file or folder
mv <originalname>/ <new file name>
mv poofolder/ wibble - the slash after poofolder is optional
move a file or folder
mv <originalname>/ <destination path>
mv poofolder/ ../ - the slash after poofolder is optional
JP NOTE: remember mv *.* will only move files (as directories have no dot!) - don't confuse this with mv *

Top

netstat

To find out what's open on a machine - compare the Gentoo and a Windows box (same command will work)!
-netstat -an
Marian example of seeing how busy tools is:
- [root@tools-2] ~ $ netstat -ant | grep LISTEN
  tcp 0 0 0.0.0.0:18080 0.0.0.0:* LISTEN
  tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN
  tcp 0 0 0.0.0.0:8998 0.0.0.0:* LISTEN
  tcp 0 0 0.0.0.0:8888 0.0.0.0:* LISTEN
since Cocoon is running on 8888, he greps for that
- [root@tools-2] ~ $ netstat -ant | grep 8888
  tcp 0 0 0.0.0.0:8888 0.0.0.0:* LISTEN
  tcp 1 0 127.0.0.1:50591 127.0.0.1:8888 CLOSE_WAIT
  tcp 0 0 127.0.0.1:50589 127.0.0.1:8888 TIME_WAIT
  tcp 0 0 127.0.0.1:50606 127.0.0.1:8888 ESTABLISHED
  tcp 0 0 127.0.0.1:50605 127.0.0.1:8888 ESTABLISHED
  tcp 0 0 127.0.0.1:50602 127.0.0.1:8888 TIME_WAIT
  tcp 0 0 127.0.0.1:50601 127.0.0.1:8888 TIME_WAIT
  tcp 0 0 127.0.0.1:50599 127.0.0.1:8888 TIME_WAIT
and if there are tons of ESTABLISHED (each one is basically a user), you can grep for that number too
- [root@tools-2] ~ $ netstat -ant | grep 8888 | grep EST | wc
  4 24 324

Top

nohup

nohup is "no hangup", so you can close the window and the process keeps running
to run a command and log out while it is running:
nohup program
nohup %1
nohup top
normally output from the process goes into nohup.out, but if you don't want output, you could redirect output to /dev/null
nohup top > /dev/null &
Note you must exit before closing the terminal window
When you are in the shell, exit the shell by typing exit. When you do this the shell will send the signal called SIGHUP to child processes. This signal will cause the child processes to finish unless the program catches the SIGHUP and responds to it in some other way. Using nohup will prevent SIGHUP from reaching the process (your java app) and it will continue as if nothing happened. If you kill the xterm (eterm, gnome-term, ect.) with the window manager's button it may send a different signal, one that nohup does not block.
Ref

Top

ps

Byron's standard commands:
ps auxw (linux)
ps -ef (solaris)

Top

rm

delete forever
rm mytest.txt
delete all files beginning with "vnu"
rm vnu*

Top

scp

copy files/folders across machines
scp -r yacapa.sql root@62.140.213.96:/usr/local/mysql/bin
scp -r root@62.140.213.99:/xxx/yyy.sql root@62.140.213.96:/usr/local/mysql/bin
-r = recursively copy entire directories
http://www.stat.ubc.ca/~webmaste/howto/file/index.html#scp
cd to a directory which contains the folder
scp -r folder username@hostA.com:.
This will create a new folder if the folder is not made yet.
scp copies from X to Y. Example for copying from cms-4 to cms-1
- this exists on cms-4:
  [root@cms-4] /export/ivory/extra/test # ls
  test.txt
- Create the target folder on cms-1:
  [root@cms-1] /export/euro-ivory/extra $ mkdir test
- From within that folder on cms-1, copy across from cms-4:
  [root@cms-1] /export/euro-ivory/extra/test $ scp -r root@cms-4.lon.vnu.net:/export/ivory/extra/test .
  Password: <enter password>
  test.txt 100% 0 0.0KB/s 00:00

Top

su

after using su
exit = close current user session. Shell mmay revert to another user, if none are left shell will close

Top

tail

to monitor live changes to a file
tail -f <filename> f=(follow)
tail -f bedlvnunet_log.2005.09.22
CTRL C to quit

Top

tar

to compress/pack and then decompress/unpack folders (? also files?)
stands for Tape Archive
zip is the Windows world alternative
Jon:
- pack: tar -czvf INS-articles-20090121.tar.gz INS-articles-20090121/
- The z switch in the above tells it to put a .tar inside a .gz
- Note that when you pull down the resulting .tar.gz to Windows, it'll need to extract it twice to get the original files, prob both times with WinRAR. But note that after the first extract you'll get out a file with no suffix, so Windows is clueless. Be prepared to add a .gz suffix to get WinRAR to open that file a second time.
Emanuel:
- pack: tar -pczf name_of_your_archive.tar.gz /path/to/directory
- unpack: tar -xzf <filename.tar.gz>
Byron
- pack:
  tar cvf file.tar /etc (creates package of tar /etc and all files)
  tar cjvf etc.tar.bz /etc == "tar and compress with bzip2 /etc" (only really works on linux boxes or any box that uses GNU tar)
  tar czvf etc.tar /etc == "create, compress with gzip, view, file" (more globally available)
- unpack: tar xzvf etc.tar.gz (subsitute the c with x, x is for extract)
JP NOTE: WinZip can unpack .gz, not .bz (you need WinRar)
Count the number of files in a tar: tar -tvf RISK-supplementary-20090123.tar.gz | wc -l

Top

time

time ./mostread.sh
gives: (the first is the important one)
real 0m0.424s
user 0m0.096s
sys 0m0.008s

Top

touch

creates an empty file
touch foo.txt

Top

wget

To check http headers eg to see if PHP is installed: wget -S http://tools.vnu.net/tools

Top

whereis

locate files in certain locations
whereis sendmail
Only looks for scripts that are in the environment $path variable

Top

who

Top

zip

zip -r images images - produces an images.zip of the /images folder

Top

Applications / Misc

Apache

Stop / Start / Restart
rcapache2 restart
rcmysql status - tells you if it's started
rcmysql stop
Byron's trick to find all the hostnames in a config folder
grep "ServerAlias" *
grep ServerName * | awk '{print $3}'
Alternatively go to directory /etc/httpd/conf.d
grep DocumentRoot *conf
Check .conf files syntax:
/opt/apache/bin/httpd -t

Top

append

echo this is a test >> /home/myfile.txt
appends the line 'this is a test' to the file

Top

awk

ls -la | awk 'BEGIN { FS = "[0-9]+:[0-9]+ " } ; { print $2 }'
list files vertically in alphabetical order

Explanation:
FS = field separator for awk
RegEx translates as:
a number one or more times
followed by :
followed by a number one or more times
followed by a space
Look at ls -la output
rwxr-xr-x 2 emanuel users 128 Oct 20 17:08 .
drwxr-xr-x 3 emanuel users 72 Oct 20 11:36 ..
-rwxr-xr-x 1 emanuel users 5744 Oct 20 17:08 update-website.sh
-rwxr-xr-x 1 emanuel users 5143 Oct 20 16:03 update-website.sh~

we are splitting after the date!

Top

bash

While using bash, if you have typed a long command, and then realize you don't want to execute it yet, don't delete it. Simply append a # to the beginning of the line, and then hit enter. Bash will not execute the command, but will store it in history so later you can go back, remove the # from the front, and execute it.
To change prompt
PS1="sillyprompt "
Pier uses:
PS1='[\[\e[34m\]\u\[\e[00m\]@\[\e[31m\]\h\[\e[00m\]] \[\e[32m\]\w\[\e[00m\] \\$ '
with yellow instead of difficult-to-read blue:
PS1='[\[\e[33m\]\u\[\e[00m\]@\[\e[31m\]\h\[\e[00m\]] \[\e[32m\]\w\[\e[00m\] \\$ '
To make this permanent, copy it into /etc/bash/bashrc
if ${use_color} ; then
PS1='[\[\e[34m\]\u\[\e[00m\]@\[\e[31m\]\h\[\e[00m\]] \[\e[32m\]\w\[\e[00m\] \\$ '

Top

Chaining Commands

Greater than redirects to a file
ls > listing.txt
less listing.txt
Pipe redirects to NEXT program ie see output from one program in another program, eg to see output of la from within less
ls | less
ls -la | less
find . gives all files
find . | grep "tomcat" find all files with "tomcat" in it
find . | grep "tomcat" > myfile.txt and put the output in a file

Top

Close Hanging Window

ps -fu emanuel
ps -fu root
List all tasks from this user, shows the pid (processID), which we can kill
kill 8878
most vicious level
kill -9 8878
QN: what's equiv of "not responding??"

Top

Directory Data

Study this example:
drwxr-xr-x 1 emanuel users 8 2005-10=07 16:51 test.txt
first part is the Permissions Table
drwxr-xr-x
first letter "d" == directory, "-" == file
triplets: read - write - execute
first three for the owner (emanuel) rwx
next three for the group to which the owner belongs (users) r-x
final three for everyone r-x
NOTE: an x colors the file name green
Other parts:
1 (no idea, sth to do with directories/files)
emanuel (user)
users (user group)
8 (size)

Top

Directory Structure

/etc/ - Linux config files
/home/ is like my documents in windows
/var/log/ usually contains real time or regular interval messages from OS eg boot messages
/usr/local/bin - Like Program Files
/usr/bin - like Windows or Windows/System32

Top

Environment Variables

OS uses $PATH to make exe's here available from within any directory. Emanuel adds eg his Java compiler to $PATH to use it from anywhere
echo $PATH
Temporarily set env variable
JAVA_HOME=/home/usr/jdk1.3
Permanently set env variable (ref)
export JAVA_HOME=/home/usr/jdk1.3

Top

Full Path

as in windows, you can run an exe from anywhere by using the full path
/usr/bin/mysqladmin
equiv of: C:\Windows\notepad.exe

Top

Hostname Change

Want to change prompt from [root@emedia-dev] to [root@dev-yacapa]
1. Change /etc/hosts
  from 127.0.0.1 localhost.vnunet.com emedia-dev to 127.0.0.1 localhost.vnunet.com dev-yacapa
2. Run hostname command
  hostname dev-yacapa
Ref

Top

MySql

To check your DB connection from webserver A to MySql on webserver B (IP 10.75.51.37, db name: yacapa), execute following on A using the credentials for MySql on B:
mysql -uyacapa-webuser -p -h10.75.51.37 yacapa

Top

Network Services

xinetd
want to have as few as possible running for external viewers

Top

Page through command result

shift arrow up/down
NOTE: this doesn't seem to work for me! Try piping it through less!

Top

PHP

Location of php.ini file? See phpinfo(). On Pier's Gentoo install it's in /opt/php/etc/php.ini

Run php from command line

Use /opt/php/bin/php (Gentoo), /usr/bin/php (Fedora)
See modules
./php -m
Get help
./php -h
See phpinfo
./php -i
echo "<?phpinfo()?>" | /usr/local/bin/php | less
Ref

Top

Perl

to identify a perl script look at the first line - perl scripts always begin:
#!/user/bin/perl -w
tells linux with what exe it should run the file
to run a perl script
mt-check.cgi does not work
./mt-check.cgi works!
why? mt-check.cgi is a perl script ie a text file, not an exe
./ means "here execute"
to see text output of perl script use the pipe to chain the perl script to "less"
./mt-check.cgi | less

Top

Redirectors

Both | and > are "redirectors"
> redirect standard output to file (to get errors use &>)
| take standard output from command 1 as standard input to command 2 (can't get errors)

Top

Run progam in background

Add a & to the end of your command to run prog in background and get back control of your terminal
top &
nohup
If prog has already started can run it in the background like this:
CTRL-Z to suspend it
bg to run in the background
fg to get the program back in the foreground
Ref

Top

Search and Replace all files in a folder

find ./ -type f -exec sed -i 's/string1/string2/g' {} \;
Ref

Top

See all users

Equivalent of listing folders in Documents and Settings!
cd /home
ls

Top

See server load

top - to see current belastung, Q to quit
w gives load report (not LIVE, as top does)
free gives free resources

Top

Shortcuts

home directory
~/file.txt is the same as home/file.txt
cd ~ or even cd is the same as cd /home/user
cd - will return to the previous directory you were in

Top

Standard Out and Standard Error

Two Linux output modes: STANDARD OUT(PUT) and STANDARD ERROR
pipe shows standard out, not standard error, so this will not show "permission denied":
find . -regex ".*\.log" | less
... even when this DOES:
find . -regex ".*\.log"
REPORTS BOTH ERROR AND STANDARD
find . -regex ".*\.log"

find: ./news: Permission denied
./kdm.log
./XFree86.0.log
find: ./YaST2: Permission denied
./xinetd.log
./Xorg.0.log
./SaX.log
./tomcat5/base/start.log
find: ./apache2: Permission denied
./boot.log
REPORTS ONLY ERRORS, STANDARD GOES INTO THE FILE
find . -regex ".*\.log" > ~/file.txt

find: ./news: Permission denied
find: ./YaST2: Permission denied
find: ./apache2: Permission denied
CAN GET ERRORS INTO A FILE
find . -regex ".*\.log" &> ~/file.txt WORKS
CAN'T GET ERRORS WITH PIPE
find . -regex ".*\.log" &| less DOES NOT WORK

Top

Switch Applications

Ctrl + Z blends current task (eg less) into the background
fg brings it back

Top

Switch Terminals

alt F1 / F2

Top

Symlink

create a link from istore/daily-thisweek/ to important
ln - s istore/daily-thisweek/ important
now these are the same:
cd important
cd istore/daily-thisweek/ important
This is how the Amazon A/B switches were done: Apache pointed to a symlink, the SymLink was just flipped between a and b
Apache FollowSymLinks only a danger if you've created a stupid SymLink, eg from web folder to system folder

Top

rc

you can see these shell scripts (// .bat files) in /etc/rc.d/

Top

Which version of Linux?

to see kernel version:
uname -a
Linux www.yacapa.com 2.6.7-1.478smp #1 SMP Thu Jul 8 15:17:25 EDT 2004 i686 i686 i386 GNU/Linux
for more useful information, this will often work (eg on Fedora and Suze, but not on Gentoo):
cat /etc/issue or cat/etc/issue.net
Fedora Core release 2 (Tettnang)
Kernel \r on an \m
also try:
cat /proc/version
Linux version 3.10.0-427.10.1.lve1.4.22.el7.x86_64 (mockbuild@build.cloudlinux.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) ) #1 SMP Fri Oct 21 09:45:54 EDT 2016

Top

vi

vi <filename>
2 modes: insert & general (latter is default mode when you open file in vi, read only)
Be careful, esp. in general mode any key press can fuck things up
Press i to get into insert mode
Will see -- INSERT -- at the bottom of the screen
Press Esc to get out of insert mode
To save:
Esc to get out of insert mode
: this gets you to a command prompt
w<enter> saves (stands for write)
To quit:
Esc to get out of insert mode
: this gets you to a command prompt
q<enter> saves (stands for write)
Can also chain commands
Esc to get out of insert mode
: this gets you to a command prompt
wq<enter> saves (stands for write)

Top