안녕하세요, 주니어 개발자 Eon입니다.
오늘 포스팅 내용은 서버 트래픽 지연에 대한 테스트 방법입니다.
단편성 포스트이며, 서버 과부하에 대한 테스트는 아닙니다.
서버 트래픽 지연 테스트는 말 그대로 서버의 트래픽을 제한함으로써 수행 가능합니다.
'트래픽 제한(제어)'인 셈입니다.
Why do we have to test server traffic restrictions? : 트래픽 제한 테스트는 왜 하는 걸까?
극한의 환경을 가정
대표적으로, 트래픽이 지연되는 상황을 가정할 수 있습니다.
글로벌 서비스를 하다 보면 불가피하게 네트워크 품질이 떨어집니다.
그에 대한 연결 안정성 보장을 위해 낮은 품질의 네트워크 환경에서도 서비스가 잘 동작하는지 확인하고 테스트할 필요가 있습니다.
화상회의, 온라인 게임 등에서도 필요한 테스트입니다.
화상회의는 트래픽 지연 이슈가 크면 클수록 상대방이 잘 보이지 않고, 영상이 끊기며, 자주 회의에서 퇴장처리되는 등의 문제가 발생할 수 있습니다.
온라인 게임은 유저들 간에 싱크가 맞지 않음으로써 격리 현상이 생길 수 있고, 오픈 마켓 같은 게임 내 시장 거래 시스템 이용에 어려움이 발생할 수 있습니다.
보통 지역이 서로 다른 곳에 서비스를 할 경우에 발생 가능성이 높습니다.
한국 국내에 한정해서는 발생 가능성이 낮으나, 일본이나 중국으로 바다 건너 국가와의 통신에서는 지연이 발생하기 쉽습니다.
이제 아래에 한 가지 트래픽 제어 방법을 소개합니다.
Network interface : 네트워크 인터페이스
ifconfig출력값 중 'eth0'를 포함한 같은 종류가 네트워크 인터페이스이다.
주의 사항
아래의 Command를 실행한 후에 중지하지 않으면 계속해서 트래픽이 제한될 수 있으므로 테스트를 마친 이후에는 항상 중지할 수 있도록 한다.
Traffic Control. wondershaper : 트래픽 제어
트래픽 제어를 하는, wondershaper를 활용하는 방법이며 잘 동작하는 것을 확인했습니다.
ubuntu 18.04 docker bridge network 환경에 대해 수행했습니다.
아래의 example 외에도 wondershaper의 README.md와 man.page를 첨부합니다.
dependecies
sudo apt-get install wondershaperusage
#traffic limit wondershaper {network interface} {download_speed / kbps} {upload_speed / kbps} #traffic unlimit wondershaper clear {network interface}example
#traffic limit sudo wondershaper br-*-*-*-*-*-*- 100 100 #traffic unlimit sudo wondershaper clear br-*-*-*-*-*-*-
wondershaper README.md
The Wonder Shaper 1.1a
bert hubert <ahu@ds9a.nl>
http://lartc.org/wondershaper
(c) Copyright 2002
Licenced under the GPL - see 'COPYING'
This document is a bit long, I'll split it up later.
The very short summary is: edit the first few lines of 'wshaper' and run it.
GOALS
-----
I attempted to create the holy grail:
* Maintain low latency for interfactive traffic at all times
This means that downloading or uploading files should not disturb SSH or
even telnet. These are the most important things, even 200ms latency is
sluggish to work over.
* Allow 'surfing' at reasonable speeds while up or downloading
Even though http is 'bulk' traffic, other traffic should not drown it out
too much.
* Make sure uploads don't harm downloads, and the other way around
This is a much observed phenomenon where upstream traffic simply destroys
download speed. It turns out that all this is possible, at the cost of a
tiny bit of bandwidth. The reason that uploads, downloads and ssh hurt
eachother is the presence of large queues in many domestic access devices
like cable or DSL modems.
* Have the ability to mark certain hosts/ports as 'low priority'
If you *know* which hosts or ports are hogging your outgoing link, be able
to deprioritize it.
The next section explains in depth what causes delays, and how we can fix
them. You can safely skip it and head straight for the script if you don't
care how the magic is performed.
Before emailing me or the mailinglist PLEASE read the 'known problems'
section.
Why it doesn't work well by default
-----------------------------------
ISPs know that they are benchmarked solely on how fast people can download.
Besides available bandwidth, download speed is influenced heavily by packet
loss, which seriously hampers TCP/IP performance. Large queues can help
prevent packetloss, and speed up downloads. So ISPs configure large queues.
These large queues however damage interactivity. A keystroke must first
travel the upstream queue, which may be seconds (!) long and go to your
remote host. It is then displayed, which leads to a packet coming back,
which must then traverse the downstream queue, located at your ISP, before
it appears on your screen.
This HOWTO teaches you how to mangle and process the queue in many ways, but
sadly, not all queues are accessible to us. The queue over at the ISP is
completely off-limits, whereas the upstream queue probably lives inside your
cable modem or DSL device. You may or may not be able to configure it. Most
probably not.
So, what next? As we can't control either of those queues, they must be
eliminated, and moved to your Linux router. Luckily this is possible.
Limit upload speed somewhat
---------------------------
By limiting our upload speed to slightly less than the truly available rate,
no queues are built up in our modem. The queue is now moved to Linux.
Limit download speed
--------------------
This is slightly trickier as we can't really influence how fast the internet
ships us data. We can however drop packets that are coming in too fast,
which causes TCP/IP to slow down to just the rate we want. Because we don't
want to drop traffic unnecessarily, we configure a 'burst' size we allow at
higher speed.
Now, once we have done this, we have eliminated the downstream queue totally
(except for short bursts), and gain the ability to manage the upstream queue
with all the power Linux offers.
Let interactive traffic skip the queue
--------------------------------------
What remains to be done is to make sure interactive traffic jumps to the
front of the upstream queue. To make sure that uploads don't hurt downloads,
we also move ACK packets to the front of the queue. This is what normally
causes the huge slowdown observed when generating bulk traffic both ways.
The ACKnowledgements for downstream traffic must compete with upstream
traffic, and get delayed in the process.
We also move other small packets to the front of the queue - this helps
operating systems which do not set TOS bits, like everything from Microsoft.
Allow the user to specify low priority traffic (new in 1.1!)
------------------------------------------------------------
Sometimes you may notice low priority OUTGOING traffic slowing down
important traffic. In that case, the following options may help you:
NOPRIOHOSTSRC
Set this to hosts or netmasks in your network that should have low
priority
NOPRIOHOSTDST
Set this to hosts or netmasks on the internet that should have low
priority
NOPRIOPORTSRC
Set this to source ports that should have low priority. If you have
an unimportant webserver on your traffic, set this to 80
NOPRIOPORTDST
Set this to destination ports that should have low priority.
See the start of wshaper and wshaper.htb
Results
-------
If we do all this we get the following measurements using an excellent ADSL
connection from xs4all in the Netherlands:
Baseline latency:
round-trip min/avg/max = 14.4/17.1/21.7 ms
Without traffic conditioner, while downloading:
round-trip min/avg/max = 560.9/573.6/586.4 ms
Without traffic conditioner, while uploading:
round-trip min/avg/max = 2041.4/2332.1/2427.6 ms
With conditioner, during 220kbit/s upload:
round-trip min/avg/max = 15.7/51.8/79.9 ms
With conditioner, during 850kbit/s download:
round-trip min/avg/max = 20.4/46.9/74.0 ms
When uploading, downloads proceed at ~80% of the available speed. Uploads
at around 90%. Latency then jumps to 850 ms, still figuring out why.
What you can expect from this script depends a lot on your actual uplink
speed. When uploading at full speed, there will always be a single packet
ahead of your keystroke. That is the lower limit to the latency you can
achieve - divide your MTU by your upstream speed to calculate. Typical
values will be somewhat higher than that. Lower your MTU for better effects!
A small table:
Uplink speed | Expected latency due to upload
--------------------------------------------------
32 | 234ms
64 | 117ms
128 | 58ms
256 | 29ms
So to calculate your effective latency, take a baseline measurement (ping on
an unloaded link), and look up the number in the table, and add it. That is
about the best you can expect. This number comes from a calculation that
assumes that your upstream keystroke will have at most half a full sized
packet ahead of it.
This boils down to:
mtu * 0.5 * 10
-------------- + baseline_latency
kbit
The factor 10 is not quite correct but works well in practice.
Your kernel
-----------
If you run a recent distribution, everything should be ok. You need 2.4 with
QoS options turned on.
If you compile your own kernel, it must have some options enabled. Most
notably, in the Networking Options menu, QoS and/or Fair Queueing, turn at
least CBQ, PRIO, SFQ, Ingress, Traffic Policing, QoS support, Rate
Estimator, QoS classifier, U32 classifier, fwmark classifier.
In practice, I (and most distributions) just turn on everything.
The scripts
-----------
The script comes in two versions, one which works on standard kernels and is
implemented using CBQ. The other one uses the excellent HTB qdisc which is
not in the default kernel. The CBQ version is more tested than the HTB one!
See 'wshaper' and 'wshaper.htb'.
Tuning
------
These scripts need to know the 'real' rate of your ISP connection. This is
hard to determine upfront as different ISPs use different kinds of bits it
appears. People report success using the following technique:
Estimate both your upstream and downstream at half the rate your ISP
specifies. Now verify if the script is functioning - check interactivity
while uploading and while downloading. This should deliver the latency as
calculated above. If not, check if the script executed without errors.
Now slowly increase the upstream & downstream numbers in the script until
the latency comes back. This way you can find optimum values for your
connection. If you are happy, please report to me so I can make a list of
numbers that work well. Please let me know which ISP you use and the name of
your subscription, and its reputed specifications, so I can list you here
and save others the trouble.
Installation
------------
If you dial in, you can copy the script to /etc/ppp/ip-up.d and it will be
run at each connect.
If you want to remove the shaper from an interface, run 'wshaper stop'. To
see status information, run 'wshaper status'.
KNOWN PROBLEMS
--------------
If you get errors, add an -x to the first line, as follows:
#!/bin/bash -x
And retry. This will show you which line gives an error. Before contacting
me, make sure that you are running a recent version of iproute!
Recent versions can be found at your Linux distributor, or if you prefer
compiling, here: ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz
More information
----------------
Information on how this all works can be found on http://lartc.org
The Linux Advanced Routing & Traffic Control HOWTO site.wondershaper.man
WONDERSHAPER(8) System Manager's Manual WONDERSHAPER(8)
NAME
wondershaper - simple traffic shaping script
SYNOPSIS
wondershaper [interface]
wondershaper clear [interface]
wondershaper [interface] [downlink] [uplink]
DESCRIPTION
This manual page documents briefly the wondershaper script. This man‐
ual page was written for the Debian distribution because the original
script does not have a manual page.
wondershaper is a traffic shaping script that provides low latency,
prioritizes bulk transfers below normal web traffic, prioritizes
interactive shells above normal web traffic, and attempts to prevent
upload and download traffic from affecting each other's ack packets.
Put simply, the wondershaper makes your internet connection more
"responsive"
SYNTAX
A summary of wondershaper syntax is included below. For a complete
description, see the files in /usr/share/doc/wondershaper.
wondershaper [ interface ]
Shows the status of traffic shaping on that interface.
wondershaper clear [ interface ]
Removes all traffic shaping from that interface.
wondershaper [ interface ] [ downlink ] [ uplink ]
Configures the wondershaper on the specified interface, given
the specified downlink speed in kilobits per second, and the
specified uplink speed in kilobits per second.
SEE ALSO
The programs are documented fully by:
/usr/share/doc/wondershaper/README.Debian.gz
/usr/share/doc/wondershaper/README.gz
AUTHOR
This manual page was written by Vince Mulhollon <vlm@debian.org>, for
the Debian project (but may be used by others).
May 30, 2004 WONDERSHAPER(8)이번 포스트에서는 wondershaper를 활용한 트래픽 제어를 소개했습니다.
다양한 용도의 여러 환경을 가정하고 테스트할 수 있는 도구입니다.
다음 포스팅은 docker 시리즈입니다.
docker를 다루기 위한 기초부터 조금 더 편하게 사용할 수 있는 명령어까지 알아보겠습니다.
이것으로 트래픽 제어에 대한 포스팅을 마치겠습니다.
감사합니다.👍
주니어 개발자