Server crash during backup

[brunobf]

Hello Devs,

I decided to attempt to setup the backup feature on my server again, and I'm still getting crashes when it's enabled. This has been happening for a long time and I just gave up last time after a lot of debugging.

For reference, I'm using the server inside of a docker container (specifically this image: rfvgyhn/avorion:stable). I've mapped both the galaxy path to a volume, and the backup path to another volume. I've ensured that both paths have exactly the same user ownership (uid 1000)  as well as permissions:

drwxr-xr-x   11 1000     1000          4096 Feb 14 04:29 avorion
drwxr-xr-x    2 1000     1000             6 Feb 14 00:43 avorion_backup

During file server startup, I can see that it's setup to write to the correct path. I tried both leaving backupsPath empty or hardcoded to the same path. I can see from the initialization log that both are mapping to the same path:

2023-02-14 04-38-48| Backup creation enabled. Path: "/home/steam/.avorion/backups"
2023-02-14 04-38-48| Please ensure that the server has write-access to that path.

What I also see in the log is the following:

2023-02-14 04-39-24| Creating emergency backup, waiting for database ... (0s); collecting data ... (1.56s); saving backup ...
2023-02-14 04-39-25| Server frame took over 1 second.
2023-02-14 04-39-25| 
2023-02-14 04-39-36| Caught signal 15. Initiating shutdown.
2023-02-14 04-39-37| Cleaning up entity transfers ...

If I search for the thread that exits last, and which I'm assuming is the one dumping the stack trace, I see the following:

2023-02-14 04-38-48| Starting thread [Game WorkerPool Thread 1]...
2023-02-14 04-38-48| Started thread [Game WorkerPool Thread 1] with id 140224147474176
...
2023-02-14 04-39-43| Thread 140224147474176 Received signal SIGSEGV
2023-02-14 04-39-43| Total memory in use by process: 5.88GB
2023-02-14 04-39-43| Thread Name [140224172652288]: Save Sectors WorkerPool Thread 0
2023-02-14 04-39-43| Workpacket [140224147474176]: BackupModule::createBackup()
...
2023-02-14 04-39-43| Factions In Memory: 0
2023-02-14 04-39-43| Sectors Loaded: 0
2023-02-14 04-39-43| Total memory in use (whole system): 45.84GB
2023-02-14 04-39-43| Runtime: 50 seconds
2023-02-14 04-39-43| 
2023-02-14 04-39-43| === STACKTRACE =====================================
2023-02-14 04-39-43| #1: ?? [0xd01781]
2023-02-14 04-39-43| #2: [ext] ?? [0x7f88b7624d60]
2023-02-14 04-39-43| #3: [ext] ?? [0x7f88b774d30e]
2023-02-14 04-39-43| #4: ?? [0xc646bc]
2023-02-14 04-39-43| #5: ?? [0xc6503c]
2023-02-14 04-39-43| #6: ?? [0x3e3d0f]
2023-02-14 04-39-43| #7: ?? [0xc655c0]
2023-02-14 04-39-43| #8: ?? [0xc6608c]
2023-02-14 04-39-43| #9: ?? [0xc655c0]
2023-02-14 04-39-43| #10: ?? [0xc5a5e8]
2023-02-14 04-39-43| #11: ?? [0x484f6]
2023-02-14 04-39-43| #12: ?? [0x460d5]
2023-02-14 04-39-43| #13: ?? [0x46c10]
2023-02-14 04-39-43| #14: ?? [0xd19456]
2023-02-14 04-39-43| #15: [ext] ?? [0x7f88b79f01de]
2023-02-14 04-39-43| #16: [ext] ?? [0x7f88b7d56ea7]
2023-02-14 04-39-43| #17: [ext] ?? [0x7f88b76e8a2f]
2023-02-14 04-39-43| ===================================================
2023-02-14 04-39-43| 
2023-02-14 04-39-43| Sending crash report ... 
2023-02-14 04-39-44| Crash report sent successfully!
2023-02-14 04-39-44| Thread 140224147474176 Received signal SIGSEGV

With regards to the standard info requested to be included in the posts:

Operating System: debian:bullseye seems to be the base image OS (cm2network/steamcmd) https://github.com/CM2Walki/steamcmd/blob/master/bullseye/Dockerfile

CPU: AMD Ryzen 9 3900X

GPU: None being used

I have included 3 examples logs of the crashes (the server pretty much crash loops when it's enabled)

Let me know if any other information can help on this.

serverlog 2023-02-14 04-38-46.txt serverlog 2023-02-14 04-36-44.txt serverlog 2023-02-14 04-32-41.txt