DavidLzCoding/MissingSemesterOfCS

Learning notes about the MIT course "Missing Semester Of CS"

MissingSemesterOfCS

Brief Introduction

The "Missing Semester Of CS" course wants to tech us "computing ecosystem literacy" that is how to master tools like the command-line, text-editor, version control system and so on.

Mastering these tools not only enables you to spend less time on figuring out how to bend your tools to your will, but it also lets you solve problems that would previously seem impossibly complex.

• Course homepage: https://missing.csail.mit.edu/
• Lecture video resource link: https://www.bilibili.com/video/BV1x7411H7wa

Watch!

This doc only covered the essential useful skill&knowledge about the terminal, the git and so on. If you want to get more information about these tools, like how to beautify the terminal fonts and colors and others, please google it :)

Terminal & Shell Scripts

Terminal,also called shell,is a tool for computer users to type-in commands and control the operating system. For example, we can type shutdown nowto shut down my linux computer immediately.

Type commands on the terminal

In myself conclusion, there are three steps to type commands:

1. Make sure what you want to do with commands? For example, I want to let terminal show the current UTC time.
2. Figure out which command and parameters will satisfy the task. For example, date -u is the one possible command; and "-u" is the parameter to told the date command:"we need the UTC time"
3. Type commands on the terminal and check the return information.

String value in Shell

We can use quotes to contain string value,like:

echo "Hello world!"

Variable Assignment

Be careful that the variable assignment can't work with blanks between "=".

Use Variables

Shell grammar specified '$' symbol for variable usages:

echo $PATH

Use Command as variables which contained return values

pwd is a command which return current working directory. "$(pwd)" is a syntax that uses commands as a variable which contains the command return value.

Variable replacement in double quote string

If you need replace variables in string, please use double quote string, and don't use single quote string.

Define a shell script and call it

The first line in the shell script which started with "#!" told shell that the script should be run with python program.

Inside the shell script,we use"$1","$2"..."$n" get the input parameters.

Make a shell command through calling "source xxx.sh"

Use "shellcheck" for your scripts checking

For example:

Feel Free to use 'man' or 'tldr' commands

Working with terminals, it is important that you should feel free to use 'man' commands, which can print help docs.

# print help doc about ls
man ls 

# print help doc about shutdown
man shutdown

# print simple example doc about shutdown
tldr shutdown

Simple File System Operation

Using 'pwd' to get current path of file system

Using 'ls' to list files under current path

Using 'cd' to change path

Using 'which' to search binary runnable file

Using 'find' to find file

Using 'mv' to move file

# move current hello.md file to up level and rename to helloworld
mv hello.md ../helloWorld.md

Using 'cp' to copy file

# copy hello.md to hello2.md
cp hello.md hello2.md

File Standard I/O Operators

We can use '<' symbol for input redirection,and use '>' or '>>' symbol for output redirection:

• '<' and '>' use covered writing mode.
• '>>' uses appending writing mode.

Using 'tee' command to check the content which we wrote into file:

Error output Operator

Symbol "$?" is a error output operator and the error code 0 is stand for no error, the error code 1 is stand for has errors.

File System Search Operation

Command "find" is a powerful operation which help you find any type of files, such as special name of file, special type of file, special modification status of file and so on.

Network Operation

The "curl" is a network command which can fetch http result from designated url parameters.

We can use "ping" to check the connection between our computer and opposite website server.

Extra references: Linux Shell Tutorials

• Chinese Version: https://linux.vbird.org/linux_basic/
  • I think this website is the best linux tutorial in Chinese.
• English Version:

Data Wrangling

Data Wrangling is mean for processing, transforming the original data.

Using grep to extract text data

The img shows that we use "grep" to extract text line which contains "sshd". After that, using symbol ">" to output result into file "tsp.log".

And, whatever, if you need to filter text data twice continuously.

Using sed to do text search,replace,count,filter

For example, at below img, 's/.*Disconnected from//' means that searching any text starts with any word and end with 'Disconnected from', and replace with blank.

In addition,pipeline symbol "|" can forward the result of sed to another text processing commands like "uniq", "wc", and so on.

Please debug regexp with tools

Regexp is too complicated to debug with your eyes, so, please debug it with regexp debugging tools,for example:

• regex101.com

Version Control Tools

The version control tool can save you from disaster, help you collaborate with others, and quickly find and isolate problematic changes.

Git graph that show how to branch and merge

For example, we can do feature adding and bug fixing based on the same parent node,concurrently. After that, merging these two nodes into one node.

New Git repo will create a hidden git folder

When a new git repository has been created, the git will help us create a hidden folder which is a git metadata store.

Use "git status" to check what's going on git

For example:

Use "git add" to add files which should be in the next snapshot

For example, if you want file "hello.txt" be included in the next snapshot, you should input "git add hello.txt".

Use "git log" to check all history commit operations

For example:

Use "git cat-file" to check git decimal version code

Use "git checkout" to move the head of node

If you want to change your current working nodes, you can use "git checkout". This git command will set node,which you want,as a head node and change contents to that time.

Use "git diff" show difference between two commit version of codes.

For example:

Use "git branch" create a new branch

For example:

Use "git merge" to merge branch into current branch

For example, we check out to master branch, and merge cat branch into the master branch.

Debugging and Profiling

A golden rule in programming is that code does not do what you expect it to do, but what you tell it to do. Bugs are gaps that between your expectation and your codes.

• A first approach to debug a program is to add print statements around where you have detected the problem, and keep iterating until you have extracted enough information to understand what is responsible for the issue.

• A second approach is to use logging in your program, instead of ad hoc print statements.

The log file place

In UNIX systems, it is commonplace for programs to write their logs under /var/log.

On linux, use "journalctl" to check system log:

# On Linux
journalctl --since "1m ago" | grep Hello

There are also some tools like lnav, that provide an improved presentation and navigation for log files.

The debuggers

• Program language specified debugger:
  When printf debugging is not enough you should use a debugger. For example: pdb for python, adb for Android, lldb for c/cpp.

• System call debugger:
  Even if what you are trying to debug is a black box binary there are tools that can help you with that. Whenever programs need to perform actions that only the kernel can, they use System Calls. There are commands that let you trace the syscalls your program makes. In Linux there’s strace and macOS and BSD have dtrace.

strace example:

The network tracer

Under some circumstances, you may need to look at the network packets to figure out the issue in your program. Tools like tcpdump and Wireshark are network packet analyzers that let you read the contents of network packets and filter them based on different criteria.

The browser debuger

For web development, the Chrome/Firefox developer tools are quite handy.