/linux-scripting

Primary LanguageShell

This project is for learning the "Linux Basics and scripting"


Image

Basic Linux Commands

General Operations:

  • clear = Clears the terminal

Directory Operatings:

  • pwd = Show current directory. Example Output: /home/hotia
  • ls = List folders and files. Example Output: Desktop Downloads Pictures Documents
  • cd [dirname] = Change directory to [dir]
  • mkdir [dirname] = Make directory [dirname]
  • cd .. = Go up a directory

File Operations:

  • touch [filename] = Create [filename]
  • rm [filename] = Delete [filename]
  • rm -r [dirname] = Delete a non-empty directory and all the files in it
  • rm -d [dirname] or rmdir [dirname] = Delete an empty directory

Navigating in the File System:

  • cd usr/local/bin = Navigate multiple dirs (relative path - relative to current dir). Move to bin directory
  • cd ../.. = Move up 2 hierarchies, so go to 'usr' directory
  • cd /usr = Alternative to go to 'usr' directly (absolute path)
  • cd [absolute path] = Move to any location by providing the full path
  • cd /home/hotia = Go to my home directory (absolute path)
  • cd ~ = Shortcut alternative to go to home directory
  • ls /etc/network = List folders and files of 'network' directory

More File and Directory Operations

  • mv [filename] [new_filename] = Rename the file to a new file name
  • cp -r [dirname] [new_dirname] = Copy dirname to new_dirname recursively meaning including the files
  • cp [filename] [new_filename] = Copy filename to new_filename

Some more useful commands

  • ls -R [dirname] = Show dirs and files but also sub dirs and files
  • history = Gives a list of all past commands typed in the current terminal session
  • history 20 = Show list of last 20 commands
  • CTRL + r = Search history
  • CTRL + c = Stop current command
  • CTRL + SHIFT + v = Paste copied text into terminal
  • ls -a = See hidden files too
  • cat [filename] = Display the file content
  • cat .bash_history = Example 1: Display the file content
  • cat Documents/java-app/Readme.md = Example 2: Display the file content

Display OS Information

  • uname -a = Show system and kernel
  • cat /etc/os-release = Show OS information
  • lscpu = Display hardware information, e.g. how many CPU you have etc.
  • lsmem = Display memory information

Execute commands as superuser

  • sudo [some command] = Allows regular users to run programs with the security privileges of the superuser or root
  • su - admin = Switch from hotia user to admin
Package Manager - Installing Software on Linux

APT Package Manager:

  • sudo apt search [package_name] = Search for a given package
  • sudo apt install [package_name] = Install a given package
  • sudo apt install [package_name] [package_name2] = Install multiple packages with one command
  • sudo apt remove [package_name] = Remove installed package
  • sudo apt update = Updates the package index. Pulls the latest change sfrom the APT repositories

APT-GET Package Manager:

  • sudo apt-get install [package_name] = Install package with apt-get package manager

SNAP Package Manager:

  • sudo snap install [package_name] = Install a given package
Working with Vim Editor

Install Vim, if it's not available:

  • sudo apt install vim = Search for a given package

There are 2 Modes:

  • Command Mode: default mode, everything is interpreted as a command
  • Insert Mode: Allows to enter text

Vim Commands:

  • vim [filename] = Open file with Vim
  • Press i key = Switch to Insert Mode
  • Press esc key = Switch to Command Mode
  • Type :wq = Write File to disk and quit Vim
  • Type :q! = Quit Vim without saving the changes
  • Type dd = Delete entire line
  • Type d10d = Delete next 10 lines
  • Type u = Undo
  • Type A = Jump to end of line and switch to insert mode
  • Type 0 = Jump to start of the line
  • Type $ = Jump to end of the line
  • Type 12G = Go to line 12
  • Type 16G = Go to line 16
  • Type /pattern = Search for pattern, e.g. /nginx
    • Type n = Jump to next match
    • Type N = Search in opposite direction
  • Type :%s/old/new = Replace 'old' with 'new' throughout the file
Linux Accounts & Groups (Users & Permissions Part 1)

Locations of Access Control Files:

  • /etc/passwd
  • /etc/shadow
  • /etc/group
  • sudo adduser [username] = Create a new user
  • sudo passwd [username] = Change password of a user
  • su - [username] = Login as username ('su' = short for substitute or switch user)
  • su - = Login as root
  • sudo groupadd [groupname] = Create new group (System assigns next available GID)
  • sudo adduser [username] = Switch to Insert Mode

Note 2 different User/Group commands:
adduser, addgroup, deluser, delgroup = interactive, more user friendly commands
useradd, groupadd, userdel, groupdel = low-level utilities, more infos need provided by yourself

  • sudo usermod [OPTIONS] [username] = Modify a user account
  • sudo usermod -g devops tom = Assign 'devops' as the primary group for 'tom' user
  • sudo delgroup tom = Removes group 'tom'
  • groups = Display groups the current logged in user belongs to
  • groups [username] = Display groups of the given username
  • sudo useradd -G devops nicole = Create 'nicole' user and add nicole to 'devops' group (-G = secondary group, not primary)
  • sudo gpasswd -d nicole devops = Removes user 'nicole' from group 'devops'
File Ownership & Permissions (Users & Permissions Part 2)
  • ls -l = Print files in a long listing format, you can see ownership and permissions of the file

Ownership:

  • sudo chown [username]:[groupname] [filename] = Change ownership
  • sudo chown tom:admin test.txt = Change ownership of 'test.txt' file to 'tom' and group 'admin'
  • sudo chown admin test.txt = Change ownership of 'test.txt' 'admin' user
  • sudo chgrp devops test.txt = Make 'devops' group owner of test.txt file

Possible File Permissions (Symbolic):

  • r = Read
  • w = Write
  • x = Execute
  • '-' = No permission

Change File Permissions for different owners

File Permissions can be changed for:

  • u = Owner
  • g = Group
  • o = Other (all other users)

Minus (-) removes the permission

  • sudo chmod -x api = Takes 'execute' permission away for 'api' folder from all owners
  • sudo chmod g-w config.yaml = Takes 'write' permission away for 'config.yaml' file from the group

Plus (+) adds permission

  • sudo chmod g+x config.yaml = Add 'execute' permission for 'config.yaml' file to the group
  • sudo chmod u+x script.sh = Add 'execute' permission for 'script.sh' file to the user
  • sudo chmod o+x script.sh = Add 'execute' permission for 'script.sh' file to other

Change multiple permissions for an owner

  • sudo chmod g=rwx config.yaml = Assign 'read write execute' permissions to the group
  • sudo chmod g=r-- config.yaml = Assign only 'read' permission to the group

Changing permissions with numeric values

Set permissions for all owners with 3 digits, 1 digit for each owner Absolute vs Symbolic Mode

  • 0 = No permission
  • 1 = Execute
  • 2 = Write
  • 3 = Execute + Write
  • 4 = Read
  • 5 = Read + Execute
  • 6 = Read + Write
  • 7 = Read + Write + Execute
  • sudo chmod 777 script.sh = rwx (Read, Write and Execute) permission for everyone for file 'script.sh'
  • sudo chmod 740 script.sh = Give user all permissions (7), give group only read permission (4), give other no permission (0)
Pipes & Redirects

Pipe & Less:

Pipe Command:

  • | = Pipe command = Pipes the output of the previous command as an input to the next command

Less Command:

  • less [filename] = Displays the contents of a file or a command output, one page at a time. And allows to navigate forward and backward through the file

Different piping examples/use cases:

  • cat /var/log/syslog | less = Pipes the output of 'syslog' file to less program.
  • ls /usr/bin | less = Pipes the output of ls command to less program.
  • history | less = Pipes the output of history command to less program.

Pipe & Grep:

Grep Command:

  • grep [pattern] = Searches for a particular pattern of characters and displays all lines that contain that pattern

More piping examples/use cases:

  • history | grep sudo = Look for any commands of history commands, which have 'sudo' word in it
  • history | grep "sudo chmod" = Look for any commands of history commands, which have 'sudo chmod' phrase in it
  • history | grep sudo | less = History output will pass output to grep and filter for 'sudo' and this output will again be piped or passed to less program
  • ls /usr/bin/ | grep java = Filter ls output for java
  • cat Documents/java-app/config.yaml | grep ports = See all 'ports' occurences in config.yaml file

Redirects in Linux:

  • > = Redirect Operator = Takes the output from the previous command and sends it to a file that you give

Different redirects examples/use cases:

  • history | grep sudo > sudo-commands.txt = Redirect output into a 'sudo-commands.txt' file
  • cat sudo-commands.txt > sudo-rm-commands.txt = Redirect output of 'sudo-commands.txt' file into 'sudo-rm-commands.txt' file
  • history | grep rm > sudo-rm-commands.txt = Redirect output of filtered history commands into existing 'sudo-rm-commands.txt' file. Note: Contents of file will be overwritten
  • history | grep rm >> sudo-rm-commands.txt = Redirect output of filtered history commands into existing 'sudo-rm-commands.txt' file. Note: Contents of file will be appended
Environment Variables

Variables store information. Environment variables are available for the whole environment. An environment variable consists of name=value pair.

Existing Environment Variables:

  • SHELL=/bin/bash= default shell program, in this case bash
  • HOME=/home/hotia= current user's home directory
  • USER=hotia = currently logged in user
  • printenv = List all environment variables
  • printenv | less = List all environment variables with less program
  • printenv [environment variable] = Display value of given environment variable, e.g. printenv USER
  • printenv | grep USER = Filter environment variables, which have 'USER' in the name
  • echo $USER = Print value of USER environment variable

Create own Environment Variables:

  • export DB_USERNAME=dbuser = Set environment variable 'DB_USERNAME' with value 'dbuser'
  • export DB_PASSWORD=secretpwdvalue = Set environment variable 'DB_PASSWORD' with value 'secretpwdvalue'
  • export DB_NAME=mydb = Set environment variable 'DB_NAME' with value 'mydb'
  • printenv | grep DB = Filter environment variables for 'DB' characters
  • export DB_NAME=newdbname = Set environment variable 'DB_NAME' to new value 'newdbname'

Delete Environment Variables:

  • unset DB_NAME = Delete variable with name 'DB_NAME'

Persisting Environment Variables:

Persisting Environment Variables with shell specific configuration file: Environment variables set in terminal are only available in the current terminal session.

Add environment variables to the '~/.bashrc' file or your specific shell 'rc' file. Variables set in this file are loaded whenever a bash login shell is entered.

  • export DB_USERNAME=dbuser
  • export DB_PASSWORD=secretvl
  • export DB_NAME=mydb In terminal again:
  • source ~/.bashcrc = Load the new env vars into the current shell session

Persisting Environment Variables system wide:

  • ~./bashrc = user specific
  • /etc/environment = system wide, meaning all users will have access to the variables

PATH Environment Variable:

  • PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin = List of directories to executible files, separated by ':'. Tells the shell which directories to ssearch for the executable in response to our executed command
  • PATH=$PATH:/home/hotia = Appending /home/hotia folder to the existing $PATH value to execute for example a scipt from /home/hotia anywhere
Networking

Useful Networking Commands:

  • ip= one of the basic commands. For setting up new systems and assigning IPs to troubleshooting existing systems. Can show address information, manipulate routing, plus display network various devices, interfaces, and tunnels.
  • ifconfig= for configuring and troubleshooting networks. It has since been replaced by the ip command
  • netstat= tool for printing network connections, routing tables, interface statistics, masquerade connections, and multicast memberships
  • ps aux =
    • ps = displays information about a selection of the active processes
    • a = show processes for all users
    • u = display the process's user/owner
    • x = also show processes not attached to a terminal
  • nslookup = Find DNS related query
  • ping = To check connectivity between two nodes
SSH - Secure Shell

Connecting via SSH: ssh username@SSHserver

  • ssh root@IP_ADRESSE= Connect with root user to server address
  • ssh-keygen -t rsa= Create SSH Key Pair with 'rsa' algorithm. SSH Key Pair is stored to the default location ~/.ssh
  • ls .ssh/= Display contents of .ssh folder, which has:
    • id_rsa = Private Key
    • id_rsa.pub = Public Key
  • ssh -i .ssh/id_rsa root@IP_ADRESSE = Connect with root user to server with specified private key file location (.ssh/id_rsa = default, but you can specify a different one like this)

Two Files used by SSH:

  • ~/.ssh/known_hosts = lets the client authenticate the server to check that it isn't connecting to an impersonator
  • ~/.ssh/authorized_keys = lets the server authenticate the user