• Be part of the TeaParty!
    Click here for more information

Pending Voice freeze on 70+ users on channel

eduardoroeder

Active member
I've got this setup in Digital Ocean.

After 60+ players on a channel the whole Server starts gettings a voice freezing every couple seconds. Anyone experiencing the same thing?

---- Automatically Merged Double Post ----

I can send the logs, but I don't know what is causing. It can be the bandwidth or other things as well. I didn't notice any packet loss... But the ping variation went from 1 to 200-300, with most of time being 20-40
 
Last edited:

farhadhelix

Fanatic member
Premium User
I've got this setup in Digital Ocean.

After 60+ players on a channel the whole Server starts gettings a voice freezing every couple seconds. Anyone experiencing the same thing?

---- Automatically Merged Double Post ----

I can send the logs, but I don't know what is causing. It can be the bandwidth or other things as well. I didn't notice any packet loss... But the ping variation went from 1 to 200-300, with most of time being 20-40

hi . did you check your vps CPU usage when this happens? post the TOP command output here if you dont know what it does
 

eduardoroeder

Active member
The CPU was ranging from 10 to 100%, it continued even after I closed the CPU-consuming programs. RAM had about 6,5GB free. Forgot to mention it because I was looking deep into the problem hehe

I've switched hosts and it still didn't happen yet. But the same situation didn't happen aswell, so can't confirm that this is a network related issue (looks like it is tho).
 

eduardoroeder

Active member
No. TeaSpeak was low in CPU usage. Just as a normal instance running.

By "it continued" I meant the symptom, not the 10%-100% CPU usage. After I closed those programs the CPU shrinked to it's default usage for a standard TeaSpeak server. Same thing happened with RAM.
 

farhadhelix

Fanatic member
Premium User
if it is not Teaspeak process then you have an issue in your vps

what proccess are there which you killed?
 

yamano

TeaFanatic
Premium User
If you're using the teaspeak server v1.4.1-beta1, downgrade it to v1.4.0-beta or v1.4.1-beta ;)
 
Last edited:

WolverinDEV

TeaSpeak Team
Staff member
Administrator
If you're using the teaspeak server v1.4.1-beta1, downgrade it to v1.4.0-beta or update it to v1.4.1-beta2 ;)
Use 1.4.1-beta0
Beta 1 and 2 are fucked up currently

---- Automatically Merged Double Post ----

may use beta 3 :D

---- Automatically Merged Double Post ----

Well the main issue is (i think) 70 ppl in one channel and a lot of them talking.
Depending on your config.yml and your instance you should may change some settings.

As a question: How many VS do you have with how many users?
As well how many cores & threads does your VPS have?
 
Last edited:

eduardoroeder

Active member
Like I said on the main topic, I've now changed the host provider just after those issues and they seem to have disappeared.

All the TeaServer described here are the 1.4.1-beta (no 0, 1, 2 or 3)

I had initially a VPS on Digital Ocean with 3 shared CPU core, 1gb ram, running TeaSpeak with 7 virtual servers, 300 slot each, 100~ clients/VS + 7proccesses which peaks to 100% cpu usage several times per minute, having about 100mb ram left and then 4gb swap space. This was working fine for 2-3 weeks.

I needed to open more of those proccesses (it's a bot instance which uses 80-110mb ram each, and extensivelly does some crawling on a game site to catch information and display on the TS3 server channels), so I decided to make a upgrade: I switched on 8/11/2019 to a VPS with 2 Dedicated CPU core + 8gb RAM so i could increase my clients amount. I proceeded then to add one more Virtual Server with 400 slots, with more or less 280 people connecting. Plus one more Bot instance. This made my CPU usage to go almost always over 80%, going some times to 100% (looked like it was the same behaviour as before, but a little stronger). In this situation, I joined all server instances to watch out what were happening: on 4 of those 8 instances they were doing some kind of meeting. One server had 200 player on one channel and 80 on another; second server had 80 players on a single channel; third and fourth servers had 50~ players on a single channel.

Sometimes it just lagged one instance, other times all of the instances (even the other 4) lagged also.

I further tryied to resize my VPS from 2 dedicated to 4 shared CPU cores, and from 8gb ram to 6GB ram, but it didn't fix aswell.

Then I started looking that this happened just after this last instance joined. I talked to them and they had some similar situations on the last 2 hosting providers. I couldn't be that a simple server cant handle 200 players on a single channel, since I've done this several times on other people TS servers and never had any trouble at it.

So, it could be two things: either someone was bugging the server (DDoS, some kind of invalid packet, some kind of exploit, idk) or the server network could not handle everything together, since the more people in one channel the more bandwith the server needs. 200 players accross 100 channels (2 players in each channel) requires little-to-no bandwith, while 200 players in one channel requires massive bandwith.

Then I decided to change the host. In couple of hours I switched to OVH (where my concorrent hosts their TS and never had this kind of issue, and where I know they have DDoS protections) and the problem vanished. Just copied the DB, reinstalled everything etc and it was all good. The whole weekend it's been some meeting with well over 100 players on some servers and yet I(they) couldn't notice any voice lagging.

Probably it isn't related to TeaSpeak, but yet it can still be.
 

REDOSS

Fanatic member
OVH is a good, if not the best option to a host provider!
I'm a OVH client since 2004/2005 ;)
But in any case, 50, 100, 200 clients in one channel it requires good channel bandwidth and good processor. Most likely, his previous hoster does this not provide.
 

eduardoroeder

Active member
But in any case, 50, 100, 200 clients in one channel it requires good channel bandwidth and good processor. Most likely, his previous hoster does this not provide.
Bandwidth of course, but processor didnt play a thing there, since I tryied turning off all the "bots" while the issue were happening, lowering the CPU usage down to ~15%(or less).

---- Automatically Merged Double Post ----

I also set up the default channel codec to Opus Voice with 5 of quality
 
Last edited:

WolverinDEV

TeaSpeak Team
Staff member
Administrator
Well I love to see such detailed investigation.
Normally users just blame the software for errors :D

And you really got the main point.
100 users spread across a lot of channel make less traffic than 100 users in one channel.
Simple formula: 300bps * (<user count> - 1) ^ <concurrent speakers>

Another big thing if you're having servers of that scale would be to play around with the thread settings within the config.yml :)
The second option which could be adjusted is the encryption. On this scale it makes a huge difference if voice packets get encrypted or not.

Some further questions:
How many users do you have as well what is the resource usage?
CPU, RAM, Traffic and ofc which CPU you're using.

And as a takeaway for me:
I never thought really that channels with 100+ users had a real use case, but I might should do some code optimizations to improve performance at that scale. Specially within the IO handler I've some ideas, but I'm afraid to apply them because this could have some quite unexpected side effects.