OS
init == systemd(in modern)
responsible for transitioning the system between runlevels.
runlevel
there are different run levels that define the state or mode in which the system operates; single-user mode, multi-user mode, graphical mode. The init process switches between these run levels as needed.
Order of operating system boot process (after powering on)
- Self Test(Power on self Test) > Loading OS > Loading System Utilities
Safe Mode
a diagnostic mode that starts OS with a limited set of files and drivers to debug system malfunctions.
Fence Register(memory fence / memory barrier)
a hardware mechanism that ensures that memory operations are performed in a specific order, especially in multi-threaded or multi-core environments. The primary purpose is to ensure proper synchronization and memory consistency for CPU operations. It primarily protects the CPU.
Threads
- the execution units within a process.
- each thread has its own execution context, state, stack, program counter.
- within the same process, they share the heap, stack, code section, and global/environment variables.
Addressed generated by CPU
- logical addresses used in virtual memory systems.
Kernel
- a core component of the OS responsible for managing HW resources, providing system services and mediating interactions between SW and HW.
- remains in memeory throughout the entire computer session.
- the first part of the OS to load into memory during booting.
- Power-On-Self-Test and BIOS/UEFI Initialization
- consists of various modules and not all of them are loaded into memery at once when the OS is running
- critical code is loaded into a separate area of memory.
Trouble shoot Too Many Open Files Error
- sudo lsof : lists Open Files to identify the process that's generating "Too many open files" error
- ulimit -n : Check the resource limits for the user and the system.
- ulimit -n 9999 : Increate Resource Limits
CPU
- speed is measured in Gigaherts(GHz)
Fully-distributed mode
- all daemons execute the process in separate nodes.
Memory
Code Section
- a section where the executable code of the process resides
Data Section
- a section where global and static variables used by the process reside. These variables are shared within a process.
Heap
- a region of memory where dynamically allocated data is stored.
- All threads within the same process share the same heap.
- meaning that memory allocated on the head by one thread can be accessed and modified by other threads in the same process.
Stack
- a region of memory used for function call and local variable storage.
Memory-Mapped Files
- processes can use memory-mapped files to map files into their memory space. use the data as if it were in memory
Shared Memory
- processes can share data using shared memory, pipes, sockets, interprocess communition(IPC)
Cache memory
Mapping methods
- direct mapping
- associative mapping
- set-associative mapping
When to use caching
- Short running queries on databases : △
- Computation intensive processing : O
- Working with inconsistent data : X
- high latency network requests : O
- long running queries on databases : O
- Frequently accessed data : O
Multi-tasking/processing
Multi-tasking (Single-Threaded)
- run tasks/processes concurrently on a single CPU core (rapidly switching between tasks > illusion!)
- share the resources of a single CPU core. consumtion of a task can affect the other.
- slower/unstable
Multi-processing
- run multiple processes on multiple CPU cores.
- each CPU core can execute its own process independently
- no need to share the resources of a single CPU core as each has its own CPU core.
- faster/stable
CLI
Chmod 755 file.txt
- Change mode.
- User Group Others
- rwx → 421
ps aux / top(real-time)
- to view a list of running processes
dmesg
- provides system-level information related to kernel messages including HW, drivers and system events.
- not detailed CPU info.
lscpu
- gets detailed CPU info.
netstat
- provides information about network connections, routing tables, interface statistics, masquerade connections, and multicast memberships.
ss(socket statistics)
- display detailed socket statistics, including information about network connections and their state.
Network
Checking port on a server
- "Ping" directly checks whether the server itself is reachable and responsive using ICMP(Internet Control Message Protocol). NOT directly checking the status of a specific port on a server.
- telnet/nc/nmap (commands) can be used to check a specific port on a server.
- When using telnet/nc etc to check the status of a port on a remote server, TCP(Transmission Control Protocol) is commonly used.
SNMP
- a widely used protocol for monitoring and managing network devices such as routers, switches and servers.
Get an IP from DHCP
- Discover → Offer → Request → Acknowledge(Confirming IP Lease)
※ lease is a concept for the DHCP IP assignment.
CIDR
- private
- 10.0.0.0 ~ 10.255.255.255
- 172.16.0.0 ~ 172.31.255.255
- 192.168.0.0 ~ 192.168.255.255
- public
- 0.0.0.0 ~ 127.255.255.255
- 128.0.0.0 ~ 191.255.255.255
- 192.0.0.0 ~ 233.255.255.255
- 224.0.0.0 ~ 239.255.255.255
- 240.0.0.0 ~ 255.255.255.255
IPv4 VS IPv6
- Anycast Address is specific to IPv6 and not present in IPv4
A network with 5 subnets, each with at least 16 hosts. which subnet masks should be used ?
- Subnet mask: 255.255.255.240 (or /28 in CIDR notation)
Port numbers
- SFTP(22; SSH File Transfer Protocol)
- SFTP and SCP(Secure Copy Protocal) use SSH(Secure Shell) protocol to secure the traffic and data transfer.
OSI Model
- Application > Presentation > Session > Transport > Network > Data Link > Physical Layer
TCP Sliding Window
- a fundamental mechanism used to optimize and control the flow of data between two endpoints(devices.)
- crucial for TCP's reliability and efficiency.
DNS
- RD(Recursion Desired) flag in the DNS message header is used to determine if a query is recursive. (client sends the message to DNS server)
Trouble shoot Intermittent network
- Run a Packet Capture to Analyze the Traffic : packet loss/excessive latency, unusual traffic patterns or malformed packets.
- Review the system and network device logs
- User Follow the Path : check others' connections.
ping
- 상대 서버 점검 : ping X.X.X.X
- 내 랜카드 점검 : ping localhost / 127.0.0.1
- 유무선 공유기 점검 : ipconfig > ping 192.168.0.1
- 통신사(ISP) DNS 점검 : nslookup > exit > ping 168.126.63.1
- 인터넷망 점검 : nslookup dns.google > ping 8.8.8.8
when unable to communicate on the network.
- check the host local logs.
When an HTTP request to a host is failing
- Review Logs on All devices
- Perform tcpdump on the host(or a packet capture tool)
When Ping is OK, SSH not working
- SSH Client key is Invalid
- Only ICMP Ports are Allowed(TCP Ports) : review the server's firewall
CORS(Cross-Origin Resource Sharing)
- related to missing or incorrectly configured headers in an HTTP response
- there are specific HTTP headers that needdto be set in the response from the server to allow the origins to access resources from the server.
- Access-Control-Allow-Origin/Methods/Headers/Credentials
traceroute
- traceroute/tracepath uses UDP(User Datafram Protocol) so it is one way.
nslookup
- used mainly to resolve domain → IP or vice versa
Check for DB connection
- telnet db.ip 3306
- connect to db using psql/mysql
- ping db.ip
Purpose of ARP(Address Resolution Protocol)
- maps an IP to a MAC(Media Access Control) address on a local network.
TCP
- SSH, FTP, SMTP uses TCP
- DHCP, ICMP does not use TCP
SFP
- SFP(Sender Policy Framework) is a DNS-based email authentication protocol used to help prevent email spoofing and phshing attacks.
MX(Mail Exchanger)
- responsible for accepting incoming email for a domain.
NTFS ;;;
Layer responsible for encryption and decryption
- Transport Layer(4), Network Layer(3), Application Layer(7)
Database
DDL
- CREATE, ALTER, DROP, TRUNCATE, COMMENT, RENAME, GRANT/REVOKE
RDBMS
- designed with transactional processing in mind
- ACID (Atomicity, Consistency, Isolation, Durabiolity) transactions to ensure data integrity and reliability.
NoSQL
- typically use key-value stores
- non-relational Database
Secondary indexing
- technique that creates an additional index structure for a non-primary key column
- should be a 유일성 + 최소성 == candidate key field
Querying 500 people from over 200 countries
- partition with low volume of data and Bucket data
Keys
- super key : 유일성
- candidate key : 유일성 + 최소성
- primary key : 유일성 + 최소성
- foreign key : there can be multiple FKs
DB caching types
- Intergrated caches
- Distributed caches : on multiple nodes
- Remote caches : to share cached data
- Local caches : to reduce latency
RDBMS the adequate normal form.
- the most commonly aimed-for normal forms : 1NF, 2NF, 3NF
Role of a parser in the query lifecycle
- a crucial role in the initial stages of query processing
- Syntax Analysis
Both runtime skewness and ata skwness have impact on query performances.
which table columns should be avoided when distributing table data
- foreign key : often used as join keys.
- join column : can lead to data skew(uneven data distributions)
- columns with skewed data
※ PK is usually good.
so having duplicates in the distributing column is not good.
DW
- maintaining de-normalized tables in a DW is recommended for improved query performance, aggregation efficiency, reduced query complexity.
- Catesian product joins(cross joins) are not used in a DW.
Processing order
- Verify Syntax → Create Query Plan → Acquire Locks
Buffer(DB Cache)
- minimizes the number of block transfers between the disk and memory in a DB system.
Web
A Three-Tier Architecture
- Presentation Tier (Front-end)
- Application Tier (Middle)
- Data Tire (Back-End)
HTTP Status codes
- HTTP 3XX : indicated client request was redirected.
- 503 : Server Overload/Server Failed
- 443 :
- 500 : Internal Server Error
- 400 : Bad Request