BloodyIron

Avorion is a fun game, in my opinion, that's a reality. Back in March, when the game launched 1.0, and the Pandemic stuff started being a thing, I took it upon myself to mobilise myself and my org's staff to whip up an updated Avorion server and promote it to try and build a community, plus maybe help some people keep sanity in this madness. That was March. I didn't realise that the server software was going to perform as abysmally as it did. We have seen drastic performance issues with only 5-12 people, and that's with absurdly over-provisioned specs. I posted and reported on this matter as early as March 22nd : https://www.avorion.net/forum/index.php/topic,6443.0.html Between now and then we've seen a few Beta updates (which we haven't tested, because beta in production is bad practice), and a few aspects of those updates have mentioned server performance. I don't know if those would address the issues we've experienced, but the optimist in me says there's a good chance it would. Fastforward, it is now May 26th. Yes, I know, there's a literal global pandemic going on, but I'm not going to wait around forever running my server with the devs not releasing something that is literally crippling my server, and many other servers. Nobody wants to play on servers with the lag we've experienced, and it most certainly is not anything we're doing wrong, nor how many resources we're allocating. So when exactly are we going to get any real updates that fix server performance? It's been months, and supposedly code is already written. I'm a patient person, but at the end of the day, I'm not going to wait around forever. If the devs aren't going to fix the performance issues, I'm eventually going to nuke my server and probably move onto other games. I want to run a massive server, and with the way the server code works now, that's literally impossible. This game could be awesome in scale. So devs, when? I don't think we've seen a single update since release that wasn't beta.

Not sure what you're asking for about anything else in particular, so unsure how to answer that question. In regards to the storage, I've monitored it from within the system itself (and yes, this is inside a VM) but also at the storage level. i do not see IOPS contention at the storage level, and I have not observed iowait on the Avorion side (in the VM). The amount of storage traffic that is happening is nowhere near saturation of the network links, nor the IOPS that the storage can serve. I may not have mentioned that I used that server config link, was more pointing out that I exhausted multiple resources that I dug up online (including that one) before I posted here, as I would have preferred to solve this issue by myself, however the devs do need to get involved here.

1. We were already having performance issues with sectors at 5, and as we upped it to 7, and then later to 12, the problems stayed about the same. I only upped it because our player base was requesting it, and we observed no change in how bad the server was performing as we gradually upped it. 2. ZFS is on a dedicated NAS, not local to the hypervisor itself. FreeNAS is running the ZFS zpool, so yeah. 3. I take it you didn't read the model of CPUs we're using. I guarantee it has HT ;) 4. We had a much higher worker count, but saw zero benefit from it. When trying to profile it and look at the worker thread graphics that were generated, a lot of worker threads were doing nothing, so we lowered the count in an effort to increase efficiency. Either way, nothing changed. 5. At this point, I am 100% confident this is bad code, and I'm very optimistic about the next patch helping, as the beta patch notes already talk about server performance fixes. 6. Also, I turned profiling off to try and improve performance and reduce spam to the console (so I could actually read the console). Not sure why you're recommending it to be on in a situation that involves poor performnace. 7. I had already, multiple times, referenced the link you mention, as we were building the server.

Oooo, thanks! :D

One of the gamers on my server is requesting I whitelist a client-side only mod (or so it looks), but I'm not really finding how to whitelist a clientside-only mod, only how to enable server-side mods. Can someone please point me in the right direction? :)

How is your server load so much lower than mine? How many CPU cores did you throw at it, and what is the model of the CPU? Is your setup bare metal or a VM, or what? What is the type of storage backing your Avorion Server?

Can't wait for server performance improvements. CHUG CHUG CHUG :(

The server performance improvements can't come fast enough :(

Also would really like it if borderless worked more appropriately. I'm on Ubuntu 19.10 using the Gnome DE. When I go borderless there's always something cut-off on the top or bottom, no matter where I place the window. The window really needs to resize to the usable space for whatever DE/OS/GUI is used. I have to use fullscreen to see everything and alt-tabbing minimises, which is pain.

I really would like a better way to queue (and save) ordered commands for AI ships. Accidentally not pressing shift when issuing a command, wiping complex commands, is extremely frustrating, and there's better ways to do this. Such as a dedicated interface, or something like that. If I have to go do something to one of my ships doing loops, having to remake those loops is painful now, and as soon as I enter a ship, it gets wiped.

I don't know what sectors people are in while logged off though, more hoping for a baked-in feature for the server here. But I like your attitude here :P

Until I hear from the devs I'm going to hold off on doing that. What you propose is logical, however I don't know if there are abnormal outcomes of that, such as if players are in that space while I regen/wipe it, what happens then?

So my alerting thresholds weren't quite ideal (95% usage, with a few percentage of inaccuracy, UGH) and my server ran out of space over night. I've since grown the disk plenty, and free disk space is readily available. Looks like a few factions were generated as new, and on startup it complains saying "reading of file ... fialed, moving on to the next instance, error: unexpected end of file", which are completely reasonable error messages, as the factions probably weren't fully written to disk. In scenarios like this, as an option for an admin, I'd like to be able to force re-gen (or complete generation of remaining) of the problematic factions. Since they are new factions, I would be okay with regenning them as people haven't really become reliant on them for very long (3 hours now?). This is an infrequent scenario indeed, but it would be nice to have an option (or it just do it) to do this, so we can reduce errors at the console log going forward, instead of increase them. Thoughts?

The Server load didn't seem to increase until a particular player logged on, destroyed 5-6 ships and started salvaging them. So that seems like a probable candidate for a cause of the drastic server load increase. And an area where efficiency increases probably would help lots. Not 100% sure, but IMO worth looking into, devs. ;P

So, curious situation, increased "aliveSectorsPerPlayer" to 12 (was 7), decreased "workerThreads" to 16 (was 32) & "generatorThreads" to 8 (was 16), leaving "scriptBackgroundThreads" at 16. And now the server seems to be running better. Right now just two players online, so we'll have to see how it holds up when more players jump back on, but the "server 100% load" alert for players is not currently happening (again, 2 players). Still think this game needs actual programmatic tuning, but figured I'd share this possible improvement.