Multiprocessing and multithreading are purposed for better computing power of a system.
Multiprocessing
A system that has more than one or two processors.
CPUs are added to increase the computing the power.
There are two types of multiprocessing: symmetric multiprocessing and asymmetric multiprocessing.
Symmetric multiprocessing
A multiprocessor computer hardware & software architecture where two or more identical processors are connected to a single, shared main memory w/ full access to all input and output devices.
A type of multiprocessing where each processor is self-scheduling.
- Each individual processor does the task of OS
- All processors communicate w/ each other by a shared memory
- More expensive
- Requires synchronization to maintain load balance
- Processor failure directly leads to reduced computing capacity
Asymmetric multiprocessing
Multiprocessor computer system where NOT all of the multiple interconnected CPUs are treated equally. Only a master processor runs the tasks of the OS.
Processors are in a master-slave relationship, where only one serves as the master, responsible for assigning tasks to the slave processors.
- Master processor does the job of OS
- No communication between processors except the master processor
- Cheaper
- If the master processor malfunctions, the slave processor continues execution and turns into a master processor.
- When a slave processor fails, other processors take over the task
Disadvantage: unequal load placed on processors may cause some processors to be idle while others have a huge job queue.
Multithreading
A system in which multiple threads are created of a process to increase the computing power.
Many threads of a process are executed simultaneously.
Multiprocessing vs. Multithreading
- How it increases computing power
- Multiprocessing) CPUs are added
- Multithreading) many threads are created of a single process
- Simultaenous execution
- Multiprocessing) many processes are executed simultaneously
- Multithreading) many threads of a process are executed simultaneously
- Address space
- Multiprocessing) each process owns a separate address space
- Multithreading) all threads share a common address space.
Global Interpreter Lock
GIL: a mutex that protects access to Python objects & prevents multiple threads frome executing Python bytecodes at once
- necessary b/c CPython's memory management is not thread-safe
- only one thread can access the interpreter at a time
- In a multithread environment, the process of each thread obtaining lock of the interpreter may become a bottleneck and/or reduce performance.
Because Python uses reference counting to manage memory, each object saves the number of variables that reference itself and memory release is done when the number hits 0. While this is a efficient method in terms of memory space, it is inefficient in other terms because the reference count has to be updated every time a reference is added or released and the reference count must be an atomic operation to prevent problems. To keep this operation atomic, each operation needs a lock, causing potential problems in performances and may be a dead lock.
Thus, CPython uses a lock on the interpreter and guarantees atomicity.
GIL and Java
Since Java uses the Mark and Sweep algorithm for GC, which only operates when the memory is filled up to a certain extent, Java does not have to do atomic operations every time a reference is added or removed.
References
